pgdevkit 0.15.0__tar.gz → 0.15.2__tar.gz

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pgdevkit
-Version: 0.15.0
+Version: 0.15.2
 Summary: A helper for developing with Postgres
 Requires-Python: >=3.14
 Requires-Dist: docker>=7.1.0
@@ -471,10 +471,22 @@ Install with the `db` extra: `pip install pgdevkit[db]`.
   `SELECT` privileges, and consider `statement_timeout=at_most(...)`.
 - **`ComplexHelper`** — handles composite, enum and JSONB columns for the CRUD functions: pass
   `complex_helper=ComplexHelper(con)` (optionally with `normalizers`), or `complex_helper="auto"` for a default one,
-  to any `pg_*` function that takes the argument. What it learns from the catalog (types, column types, adapter
-  registrations) is cached per connection, so a helper created per call queries the catalog only the first time a
-  connection is used (the SELECT/INSERT itself is then the only query). The cache cannot see later DDL, nor DDL that
-  was rolled back after it had been cached: call `helper.clear_cache()` after `CREATE TYPE`/`ALTER TYPE`/column
+  to any `pg_*` function that takes the argument. **Nothing is registered on the connection:** psycopg's type
+  registration is global to a connection (and so to the next user of a pooled one: a plain `SELECT` of an enum would
+  return psycopg's auto-numbered `Enum` member instead of the label, #54). The helpers therefore register the
+  enum/composite adapters only on the *cursor* that runs a write statement (`pg_insert`, `pg_update`/`pg_update_dict`,
+  `pg_upsert`/`pg_upsert_dict`, the `*_many` variants), and only for the columns where a dict/list value had to
+  be converted. Reads (`pg_retrieve*`) select such columns as `to_jsonb(...)` and need no registration. After any
+  `pg_*` call the connection reads enums as label strings and composites as text (e.g. `'(1,2)'`), as before the call.
+  `RETURNING` rows of `pg_insert`/`pg_update_dict`/`pg_upsert_dict` (and so `pg_update`/`pg_upsert`) select such
+  columns as `to_jsonb(...)` too, so with a helper they are plain Python values (a composite as a dict, an enum as its
+  label) whichever columns were written; without one it is a plain `RETURNING *`.
+  If you call `recursive_convert` yourself, register the types on the cursor that sends the values, with the `info`
+  from `load_all_complex_types`/`load_complex_type`: `await helper.register_on(cur, info)`, then execute the converted
+  value on that same `cur` (passing the connection instead would restore psycopg's global behaviour).
+  What it learns from the catalog (types and column types) is cached per connection, so a helper created per call
+  queries the catalog only the first time a connection is used (the SELECT/INSERT itself is then the only query). The cache cannot see later DDL, nor
+  DDL that was rolled back after it had been cached: call `helper.clear_cache()` after `CREATE TYPE`/`ALTER TYPE`/column
   changes or rollbacks of those (typical for tests that create types inside a rolled-back transaction on a reused
   connection). `ComplexHelper(con, cache=False)` gives the helper a private cache and the pre-cache behaviour.
 - **CRUD functions** — `pg_retrieve`, `pg_retrieve_many` (equality `filters`, plus optional `where=` as a t-string or
@@ -556,7 +568,7 @@ for a `Response`, and the rows are *not* validated against the model.
   starts and released when it ends. A `con` must stay open until the response has been sent, i.e. come from a `Depends`
   with `yield` (FastAPI >= 0.118); `async with pool.connection() as conn: return PostgresJsonResponse(q, con=conn)`
   closes it too early. Prefer the pool.
-- **Query** — a literal string, a `psycopg.sql` composable, a sqlglot expression or a t-string, with values bound
+- **Query** — a literal string, a `psycopg.sql` composable, a sqlglot expression (any `exp.Expr`, so Select, Union, Subquery and other `exp.Query` values are fine) or a t-string, with values bound
   through `params` (`%(name)s`; a t-string carries its own values and takes no `params`). Passing only literals
   is enforced by the type checker, **not at runtime**: the query is wrapped as a subquery, never parsed, so never
   build it from user input. For a data-modifying CTE, or custom JSON, pass `query_produces_json=True` and return one

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/README.md

@@ -446,10 +446,22 @@ Install with the `db` extra: `pip install pgdevkit[db]`.
   `SELECT` privileges, and consider `statement_timeout=at_most(...)`.
 - **`ComplexHelper`** — handles composite, enum and JSONB columns for the CRUD functions: pass
   `complex_helper=ComplexHelper(con)` (optionally with `normalizers`), or `complex_helper="auto"` for a default one,
-  to any `pg_*` function that takes the argument. What it learns from the catalog (types, column types, adapter
-  registrations) is cached per connection, so a helper created per call queries the catalog only the first time a
-  connection is used (the SELECT/INSERT itself is then the only query). The cache cannot see later DDL, nor DDL that
-  was rolled back after it had been cached: call `helper.clear_cache()` after `CREATE TYPE`/`ALTER TYPE`/column
+  to any `pg_*` function that takes the argument. **Nothing is registered on the connection:** psycopg's type
+  registration is global to a connection (and so to the next user of a pooled one: a plain `SELECT` of an enum would
+  return psycopg's auto-numbered `Enum` member instead of the label, #54). The helpers therefore register the
+  enum/composite adapters only on the *cursor* that runs a write statement (`pg_insert`, `pg_update`/`pg_update_dict`,
+  `pg_upsert`/`pg_upsert_dict`, the `*_many` variants), and only for the columns where a dict/list value had to
+  be converted. Reads (`pg_retrieve*`) select such columns as `to_jsonb(...)` and need no registration. After any
+  `pg_*` call the connection reads enums as label strings and composites as text (e.g. `'(1,2)'`), as before the call.
+  `RETURNING` rows of `pg_insert`/`pg_update_dict`/`pg_upsert_dict` (and so `pg_update`/`pg_upsert`) select such
+  columns as `to_jsonb(...)` too, so with a helper they are plain Python values (a composite as a dict, an enum as its
+  label) whichever columns were written; without one it is a plain `RETURNING *`.
+  If you call `recursive_convert` yourself, register the types on the cursor that sends the values, with the `info`
+  from `load_all_complex_types`/`load_complex_type`: `await helper.register_on(cur, info)`, then execute the converted
+  value on that same `cur` (passing the connection instead would restore psycopg's global behaviour).
+  What it learns from the catalog (types and column types) is cached per connection, so a helper created per call
+  queries the catalog only the first time a connection is used (the SELECT/INSERT itself is then the only query). The cache cannot see later DDL, nor
+  DDL that was rolled back after it had been cached: call `helper.clear_cache()` after `CREATE TYPE`/`ALTER TYPE`/column
   changes or rollbacks of those (typical for tests that create types inside a rolled-back transaction on a reused
   connection). `ComplexHelper(con, cache=False)` gives the helper a private cache and the pre-cache behaviour.
 - **CRUD functions** — `pg_retrieve`, `pg_retrieve_many` (equality `filters`, plus optional `where=` as a t-string or
@@ -531,7 +543,7 @@ for a `Response`, and the rows are *not* validated against the model.
   starts and released when it ends. A `con` must stay open until the response has been sent, i.e. come from a `Depends`
   with `yield` (FastAPI >= 0.118); `async with pool.connection() as conn: return PostgresJsonResponse(q, con=conn)`
   closes it too early. Prefer the pool.
-- **Query** — a literal string, a `psycopg.sql` composable, a sqlglot expression or a t-string, with values bound
+- **Query** — a literal string, a `psycopg.sql` composable, a sqlglot expression (any `exp.Expr`, so Select, Union, Subquery and other `exp.Query` values are fine) or a t-string, with values bound
   through `params` (`%(name)s`; a t-string carries its own values and takes no `params`). Passing only literals
   is enforced by the type checker, **not at runtime**: the query is wrapped as a subquery, never parsed, so never
   build it from user input. For a data-modifying CTE, or custom JSON, pass `query_produces_json=True` and return one

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/pgdevkit/db/complex_types.py

@@ -1,10 +1,14 @@
 from __future__ import annotations
 
+import warnings
 import weakref
 from dataclasses import dataclass, field
 from typing import Any, Callable
 
 from psycopg import AsyncConnection
+from psycopg import adapters as _psycopg_adapters
+from psycopg.abc import AdaptContext
+from psycopg.adapt import AdaptersMap
 from psycopg.rows import dict_row
 from psycopg.sql import Identifier
 from psycopg.types.composite import CompositeInfo, register_composite
@@ -28,12 +32,11 @@ ComplexTypeInfo = CompositeInfo | EnumInfo | type[Jsonb] | type[JsonbArray] | No
 @dataclass
 class _TypeCache:
     """What a `ComplexHelper` has learned about one connection's database: the catalog's complex types, the
-    CompositeInfo/EnumInfo objects built from them (they carry that database's OIDs), which of those are registered
-    on the connection (an adapter registration lives and dies with the connection), and each table column's type."""
+    CompositeInfo/EnumInfo objects built from them (they carry that database's OIDs) and each table column's type.
+    Never "what is registered where": nothing is registered on the connection (see `ComplexHelper.register_on`)."""
 
     system_complex_type_dict: dict[Any, tuple[str, str]] | None = None
     complex_types: dict[tuple[str, str], CompositeInfo | EnumInfo] = field(default_factory=dict)
-    registered: set[CompositeInfo | EnumInfo] = field(default_factory=set)
     columns: dict[tuple[tuple[str, str], bool], dict[str, ComplexTypeInfo]] = field(default_factory=dict)
     column: dict[tuple[tuple[str, str], str], ComplexTypeInfo] = field(default_factory=dict)
     shared: bool = False  # whether other helpers on the same connection use it too (then column lookups are cached)
@@ -52,12 +55,28 @@ def _cache_for(con: AsyncConnection) -> _TypeCache:
         return _TypeCache()
 
 
+def _make_composite_object(values: Any, info: CompositeInfo) -> Any:
+    # One module-level function, so psycopg's cache of generated loader classes (keyed on it) hits on re-registration
+    # instead of growing by two classes per registration.
+    assert info.python_type is not None
+    return info.python_type(*values)
+
+
+class _Scope:
+    """An `AdaptContext` of its own (a copy of psycopg's defaults, shared with nothing)."""
+
+    connection = None
+
+    def __init__(self) -> None:
+        self.adapters = AdaptersMap(_psycopg_adapters)
+
+
 class ComplexHelper:
     """psycopg adapter for PostgreSQL composite types, enums, and JSONB.
 
     Detects a table's non-scalar columns (composite types, enums, JSONB) and
-    converts plain dict/list Python values into the psycopg-registered types
-    those columns need, recursing into nested composite fields.
+    converts plain dict/list Python values into the Python objects (enum members, composite namedtuples)
+    psycopg sends for those columns, recursing into nested composite fields.
 
     `normalizers` lets a caller reshape a composite value before conversion,
     keyed by composite type name (e.g. a project with a `locale_labels`
@@ -65,7 +84,14 @@ class ComplexHelper:
     this is intentionally the only project-specific extension point; nothing
     else about a project's types is hardcoded here.
 
-    What it learns from the catalog (the types, the columns' types, which ones are registered on the connection) is
+    **It never registers anything on the connection.** psycopg's adapter registration is global for the connection
+    (and so for the next user of a pooled connection: a plain `SELECT` of an enum would start returning psycopg's
+    `Enum` members instead of the label, see issue #54). What `recursive_convert` returns is sent through a cursor
+    that has the types registered for itself only (`register_on(cursor, info)`; the `pg_*` helpers do this for the
+    cursor that runs their statement), and the connection behaves exactly as before. Reads need no registration at
+    all: the `pg_retrieve*` helpers select complex columns as `to_jsonb(...)`.
+
+    What it learns from the catalog (the types and the columns' types) is
     cached per connection, for as long as that connection lives: a helper created for every call, on a connection
     that is reused (e.g. from a pool), only queries the catalog the first time. The cache can't see schema changes
     made afterwards (`CREATE TYPE`, `ALTER TYPE ... ADD VALUE`, altered columns), nor DDL that was rolled back after
@@ -106,17 +132,29 @@ class ComplexHelper:
 
     @property
     def registered(self) -> set[CompositeInfo | EnumInfo]:
-        return self._cache.registered
+        """Deprecated: nothing is registered on the connection any more, so this is always empty (a new set each
+        time). Use `register_on(cursor, info)` to register types for a statement."""
+        warnings.warn(
+            "ComplexHelper.registered is deprecated: types are no longer registered on the connection; "
+            "use `await helper.register_on(cursor, info)` to register them on a cursor",
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        return set()
 
     @registered.setter
     def registered(self, value: set[CompositeInfo | EnumInfo]) -> None:
-        self._cache.registered = value
+        warnings.warn(
+            "ComplexHelper.registered is deprecated and ignored (types are no longer registered on the connection; "
+            "see `register_on`)",
+            DeprecationWarning,
+            stacklevel=2,
+        )
 
     def clear_cache(self) -> None:
         """Forget what was learned about this connection's database (see the class docstring)."""
         self._cache.system_complex_type_dict = None
         self._cache.complex_types.clear()
-        self._cache.registered.clear()
         self._cache.columns.clear()
         self._cache.column.clear()
 
@@ -125,7 +163,10 @@ class ComplexHelper:
             await cur.execute("""
         SELECT t.oid,
                 pg_catalog.format_type ( t.oid, NULL ) AS obj_name,
-                t.typtype
+                -- an array type reports its element type's typtype, so an enum[] is known to be an enum
+                CASE WHEN t.typcategory = 'A'
+                    THEN ( SELECT et.typtype FROM pg_catalog.pg_type et WHERE et.oid = t.typelem )
+                    ELSE t.typtype END AS typtype
             FROM pg_catalog.pg_type t
             JOIN pg_catalog.pg_namespace n
                 ON n.oid = t.typnamespace
@@ -154,9 +195,7 @@ class ComplexHelper:
             return None
         udt_schema: str = res["udt_schema"]
         udt_name: str = res["udt_name"]
-        c = await self._get_complex_type(f"{udt_schema}.{udt_name}", res["is_enum"], self.con)
-        await self._recurse_register(c, self.con)
-        return c
+        return await self._get_complex_type(f"{udt_schema}.{udt_name}", res["is_enum"], self.con)
 
     async def load_all_complex_types(
         self, table_name: tuple[str, str], include_generated: bool = False
@@ -179,7 +218,8 @@ class ComplexHelper:
               from information_schema.columns c
                  left join enum_types e on e.enum_schema=c.udt_schema
                      and (e.enum_name=c.udt_name or (c.data_type='ARRAY' and c.udt_name='_'||e.enum_name))
-              where table_schema=%(schema)s and table_name = %(tbl)s and (is_generated <> 'ALWAYS' or %(include_generated)s)"""
+              where table_schema=%(schema)s and table_name = %(tbl)s and (is_generated <> 'ALWAYS' or %(include_generated)s)
+              order by c.ordinal_position"""
         async with self.con.cursor(row_factory=dict_row) as cur:
             await cur.execute(
                 colquery,
@@ -236,24 +276,51 @@ class ComplexHelper:
         if (schema, type_name) not in self.complex_types:
             ci = await (EnumInfo if is_enum else CompositeInfo).fetch(con, Identifier(schema, type_name))
             assert ci is not None, f"{'Enum' if is_enum else 'Complex'} type {name} not found in database"
+            # `recursive_convert` needs the Python types (`ci.enum`, `ci.python_type`) that psycopg only creates when
+            # a type is registered: do that in a scratch scope that no connection or cursor ever sees.
+            scratch = _Scope()
+            if isinstance(ci, EnumInfo):
+                register_enum(ci, scratch)
+            else:
+                register_composite(ci, scratch, make_object=_make_composite_object)
             self.complex_types[(schema, type_name)] = ci
         return self.complex_types[(schema, type_name)]
 
-    async def _recurse_register(self, info: CompositeInfo | EnumInfo, con: AsyncConnection) -> None:
+    async def register_on(self, context: AdaptContext, info: ComplexTypeInfo) -> None:
+        """Register the psycopg adapters for `info` (and every composite/enum nested in it) in `context`, so that
+        what `recursive_convert` returns for it can be sent by statements run through that context, and values of
+        those types come back as those Python objects.
+
+        `info` is what `load_all_complex_types(table)` / `load_complex_type(table, column)` return for the column
+        you are writing, and the value `recursive_convert(value, info, con)` returns must be executed on the same
+        `context` (the cursor) that you registered it on.
+
+        Pass the **cursor** that runs the statement: psycopg gives each cursor its own copy of the adapters, so the
+        registration lasts as long as the cursor and nothing else on the connection (hence on a pooled one) is
+        affected. Passing the connection itself registers globally for that connection, for good (psycopg's
+        default); that is not what the `pg_*` helpers do. A no-op for JSONB (`info` being `Jsonb`/`JsonbArray`) and
+        for `None`."""
+        if isinstance(info, (CompositeInfo, EnumInfo)):
+            if self.system_complex_type_dict is None:
+                await self.load_complex_type_dict()
+            await self._recurse_register(info, context, set())
+
+    async def _recurse_register(
+        self, info: CompositeInfo | EnumInfo, context: AdaptContext, seen: set[CompositeInfo | EnumInfo]
+    ) -> None:
         assert self.system_complex_type_dict is not None, "System complex type dictionary not loaded"
-        if info not in self.registered:
-            if isinstance(info, EnumInfo):
-                register_enum(info, con)
-            else:
-                register_composite(info, con)
-            self.registered.add(info)
+        if info in seen:
+            return
+        seen.add(info)
         if isinstance(info, EnumInfo):
+            register_enum(info, context, enum=info.enum)
             return
+        register_composite(info, context, factory=info.python_type, make_object=_make_composite_object)
         for t in info.field_types:
             if t in self.system_complex_type_dict:
                 name, typtype = self.system_complex_type_dict[t]
-                ci = await self._get_complex_type(name, typtype == "e", con)
-                await self._recurse_register(ci, con)
+                ci = await self._get_complex_type(name, typtype == "e", self.con)
+                await self._recurse_register(ci, context, seen)
 
     async def recursive_convert(
         self,

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/pgdevkit/db/crud.py

@@ -3,6 +3,7 @@ from __future__ import annotations
 from string.templatelib import Interpolation, Template
 from typing import Any, Callable, Literal, Mapping, Optional, Sequence, Type, TypeVar
 
+from psycopg import AsyncCursor
 from psycopg.connection_async import AsyncConnection
 from psycopg.rows import dict_row
 from psycopg.sql import Literal as SqlLiteral
@@ -26,7 +27,7 @@ def _resolve_helper(con: AsyncConnection, complex_helper: ComplexHelperArg) -> C
 async def _select_list(
     con: AsyncConnection, table_name: tuple[str, str], complex_helper: ComplexHelperArg
 ) -> Composable:
-    """Column list for a SELECT, wrapping composite/enum/JSONB columns in
+    """Column list for a SELECT or RETURNING, wrapping composite/enum/JSONB columns in
     `to_jsonb(...)` so psycopg gets back plain Python values. Falls back to
     `SELECT *` (no extra query) when no ComplexHelper is given."""
     helper = _resolve_helper(con, complex_helper)
@@ -44,12 +45,14 @@ async def _select_list(
 
 async def _convert_complex_values(
     con: AsyncConnection,
+    cur: AsyncCursor[Any],
     table_name: tuple[str, str],
     data: dict,
     complex_helper: ComplexHelperArg,
 ) -> dict:
     """Convert dict/list values destined for composite/enum columns into the
-    psycopg-registered types those columns need. A no-op (no extra query)
+    psycopg types those columns need, and register those types on `cur` (the cursor that will run the
+    statement) only, never on the connection. A no-op (no extra query)
     unless `data` actually contains dict/list values."""
     helper = _resolve_helper(con, complex_helper)
     if helper is None:
@@ -61,12 +64,14 @@ async def _convert_complex_values(
     for k in candidate_keys:
         info = await helper.load_complex_type(table_name, k)
         if info is not None:
+            await helper.register_on(cur, info)
             converted[k] = await helper.recursive_convert(data[k], info, con)
     return converted
 
 
 async def _convert_complex_values_many(
     con: AsyncConnection,
+    cur: AsyncCursor[Any],
     table_name: tuple[str, str],
     rows: Sequence[dict],
     complex_helper: ComplexHelperArg,
@@ -81,6 +86,8 @@ async def _convert_complex_values_many(
     complex_keys = {k for k, info in infos.items() if info is not None}
     if not complex_keys:
         return rows
+    for k in complex_keys:
+        await helper.register_on(cur, infos[k])
     converted_rows = []
     for row in rows:
         new_row = dict(row)
@@ -224,14 +231,19 @@ async def pg_insert(
     *,
     complex_helper: ComplexHelperArg = None,
 ) -> dict[str, Any]:
-    """Insert one row and return the full row (RETURNING *)."""
-    data = await _convert_complex_values(con, table_name, data, complex_helper)
-    query = SQL("INSERT INTO {tbl} ({cols}) VALUES ({vals}) RETURNING *").format(
-        tbl=Identifier(*table_name),
-        cols=SQL(", ").join(Identifier(k) for k in data),
-        vals=SQL(", ").join(Placeholder(k) for k in data),
-    )
+    """Insert one row and return the full row (`RETURNING *`).
+
+    With a `complex_helper`, composite/enum/JSONB columns are returned as `to_jsonb(...)`, exactly like
+    `pg_retrieve` does: plain Python values (a composite as a dict, an enum as its label), whichever columns were
+    written. Without one it is a plain `RETURNING *` as psycopg reads it."""
     async with con.cursor(row_factory=dict_row) as cur:
+        data = await _convert_complex_values(con, cur, table_name, data, complex_helper)
+        query = SQL("INSERT INTO {tbl} ({cols}) VALUES ({vals}) RETURNING {returning}").format(
+            tbl=Identifier(*table_name),
+            cols=SQL(", ").join(Identifier(k) for k in data),
+            vals=SQL(", ").join(Placeholder(k) for k in data),
+            returning=await _select_list(con, table_name, complex_helper),
+        )
         await cur.execute(query, data)
         row = await cur.fetchone()
     assert row is not None
@@ -246,22 +258,26 @@ async def pg_update_dict(
     *,
     complex_helper: ComplexHelperArg = None,
 ) -> Any | None:
-    """Update a row identified by primary_keys. Returns the raw row tuple.
+    """Update a row identified by primary_keys. Returns the raw row tuple (in column order).
+
+    With a `complex_helper`, composite/enum/JSONB columns in it are plain Python values, like `pg_retrieve` returns
+    them (a composite as a dict, an enum as its label), whichever columns were written; without one, plain psycopg
+    behaviour (`RETURNING *`).
 
     Pass `complex_helper` when the table has composite/enum columns among the
-    values being set — omitted, this behaves exactly as before (plain values
-    passed straight through to psycopg)."""
-    data = await _convert_complex_values(con, table_name, data, complex_helper)
-    set_parts = [
-        SQL("{col} = {val}").format(col=Identifier(k), val=Placeholder(k)) for k in data if k not in primary_keys
-    ]
-    where_parts = [SQL("{col} = {val}").format(col=Identifier(pk), val=Placeholder(pk)) for pk in primary_keys]
-    query = SQL("UPDATE {tbl} SET {sets} WHERE {where} RETURNING *").format(
-        tbl=Identifier(*table_name),
-        sets=SQL(", ").join(set_parts),
-        where=SQL(" AND ").join(where_parts),
-    )
+    values being set — omitted, values are passed straight through to psycopg."""
     async with con.cursor() as cur:
+        data = await _convert_complex_values(con, cur, table_name, data, complex_helper)
+        set_parts = [
+            SQL("{col} = {val}").format(col=Identifier(k), val=Placeholder(k)) for k in data if k not in primary_keys
+        ]
+        where_parts = [SQL("{col} = {val}").format(col=Identifier(pk), val=Placeholder(pk)) for pk in primary_keys]
+        query = SQL("UPDATE {tbl} SET {sets} WHERE {where} RETURNING {returning}").format(
+            tbl=Identifier(*table_name),
+            sets=SQL(", ").join(set_parts),
+            where=SQL(" AND ").join(where_parts),
+            returning=await _select_list(con, table_name, complex_helper),
+        )
         await cur.execute(query, data)
         return await cur.fetchone()
 
@@ -283,20 +299,25 @@ async def pg_upsert_dict(
     *,
     complex_helper: ComplexHelperArg = None,
 ) -> dict:
-    """INSERT ... ON CONFLICT ... DO UPDATE, returns the row as a dict."""
-    data = await _convert_complex_values(con, table_name, data, complex_helper)
-    fields = list(data)
-    updates = [SQL("{col} = EXCLUDED.{col}").format(col=Identifier(k)) for k in fields]
-    query = SQL(
-        "INSERT INTO {tbl} ({cols}) VALUES ({vals}) ON CONFLICT ({pks}) DO UPDATE SET {updates} RETURNING *"
-    ).format(
-        tbl=Identifier(*table_name),
-        cols=SQL(", ").join(Identifier(k) for k in fields),
-        vals=SQL(", ").join(Placeholder(k) for k in fields),
-        pks=SQL(", ").join(Identifier(pk) for pk in primary_keys),
-        updates=SQL(", ").join(updates),
-    )
+    """INSERT ... ON CONFLICT ... DO UPDATE, returns the row as a dict.
+
+    With a `complex_helper`, composite/enum/JSONB columns in it are plain Python values, like `pg_retrieve` returns
+    them (a composite as a dict, an enum as its label), whichever columns were written; without one, plain psycopg
+    behaviour (`RETURNING *`)."""
     async with con.cursor(row_factory=dict_row) as cur:
+        data = await _convert_complex_values(con, cur, table_name, data, complex_helper)
+        fields = list(data)
+        updates = [SQL("{col} = EXCLUDED.{col}").format(col=Identifier(k)) for k in fields]
+        query = SQL(
+            "INSERT INTO {tbl} ({cols}) VALUES ({vals}) ON CONFLICT ({pks}) DO UPDATE SET {updates} RETURNING {returning}"
+        ).format(
+            tbl=Identifier(*table_name),
+            cols=SQL(", ").join(Identifier(k) for k in fields),
+            vals=SQL(", ").join(Placeholder(k) for k in fields),
+            pks=SQL(", ").join(Identifier(pk) for pk in primary_keys),
+            updates=SQL(", ").join(updates),
+            returning=await _select_list(con, table_name, complex_helper),
+        )
         await cur.execute(query, data)
         row = await cur.fetchone()
     assert row is not None
@@ -328,30 +349,34 @@ async def pg_upsert_many_dict(
     want a missing row to be a silent no-op rather than create one."""
     if not data:
         return
-    data = await _convert_complex_values_many(con, table_name, data, complex_helper)
-    fields = list(data[0])
-    if must_exist:
-        update_assignments = [
-            SQL("{col} = {val}").format(col=Identifier(k), val=Placeholder(k)) for k in fields if k not in primary_keys
-        ]
-        target_eq = SQL(" AND ").join(
-            SQL("t.{col} = {val}").format(col=Identifier(pk), val=Placeholder(pk)) for pk in primary_keys
-        )
-        query = SQL("UPDATE {tbl} t SET {updates} WHERE {target_eq}").format(
-            tbl=Identifier(*table_name),
-            updates=SQL(", ").join(update_assignments),
-            target_eq=target_eq,
-        )
-    else:
-        updates = [SQL("{col} = EXCLUDED.{col}").format(col=Identifier(k)) for k in fields if k not in primary_keys]
-        query = SQL("INSERT INTO {tbl} ({cols}) VALUES ({vals}) ON CONFLICT ({pks}) DO UPDATE SET {updates}").format(
-            tbl=Identifier(*table_name),
-            cols=SQL(", ").join(Identifier(k) for k in fields),
-            vals=SQL(", ").join(Placeholder(k) for k in fields),
-            pks=SQL(", ").join(Identifier(pk) for pk in primary_keys),
-            updates=SQL(", ").join(updates),
-        )
     async with con.cursor() as cur:
+        data = await _convert_complex_values_many(con, cur, table_name, data, complex_helper)
+        fields = list(data[0])
+        if must_exist:
+            update_assignments = [
+                SQL("{col} = {val}").format(col=Identifier(k), val=Placeholder(k))
+                for k in fields
+                if k not in primary_keys
+            ]
+            target_eq = SQL(" AND ").join(
+                SQL("t.{col} = {val}").format(col=Identifier(pk), val=Placeholder(pk)) for pk in primary_keys
+            )
+            query = SQL("UPDATE {tbl} t SET {updates} WHERE {target_eq}").format(
+                tbl=Identifier(*table_name),
+                updates=SQL(", ").join(update_assignments),
+                target_eq=target_eq,
+            )
+        else:
+            updates = [SQL("{col} = EXCLUDED.{col}").format(col=Identifier(k)) for k in fields if k not in primary_keys]
+            query = SQL(
+                "INSERT INTO {tbl} ({cols}) VALUES ({vals}) ON CONFLICT ({pks}) DO UPDATE SET {updates}"
+            ).format(
+                tbl=Identifier(*table_name),
+                cols=SQL(", ").join(Identifier(k) for k in fields),
+                vals=SQL(", ").join(Placeholder(k) for k in fields),
+                pks=SQL(", ").join(Identifier(pk) for pk in primary_keys),
+                updates=SQL(", ").join(updates),
+            )
         await cur.executemany(query, data)
 
 
@@ -378,14 +403,14 @@ async def pg_insert_many(
     if not data:
         return
     dict_data = [d if isinstance(d, dict) else d.model_dump() for d in data]
-    dict_data = await _convert_complex_values_many(con, table_name, dict_data, complex_helper)
-    fields = list(dict_data[0])
-    query = SQL("INSERT INTO {tbl} ({cols}) VALUES ({vals})").format(
-        tbl=Identifier(*table_name),
-        cols=SQL(", ").join(Identifier(k) for k in fields),
-        vals=SQL(", ").join(Placeholder(k) for k in fields),
-    )
     async with con.cursor() as cur:
+        dict_data = await _convert_complex_values_many(con, cur, table_name, dict_data, complex_helper)
+        fields = list(dict_data[0])
+        query = SQL("INSERT INTO {tbl} ({cols}) VALUES ({vals})").format(
+            tbl=Identifier(*table_name),
+            cols=SQL(", ").join(Identifier(k) for k in fields),
+            vals=SQL(", ").join(Placeholder(k) for k in fields),
+        )
         await cur.executemany(query, dict_data)
 
 

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/pgdevkit/db/fetch.py

@@ -28,7 +28,9 @@ QueryParams = Mapping[str, Any] | Sequence[Any] | None
 # The only things a query may be: a literal string (e.g. from `SqlLoader.load_sql()`), a sqlglot
 # expression, or psycopg's own safe-composition types (`psycopg.sql` / a t-string). A plain `str`
 # is rejected by the type checker on purpose, so user-controlled text can't be concatenated in.
-type SqlQueryNoTemplate = LiteralString | exp.Expression | Composable
+# The sqlglot type is `exp.Expr` (any query: Select, Union, Subquery...), not `exp.Expression`: in
+# sqlglot 30 `exp.Query` is an `exp.Expr` but not an `exp.Expression` (issue #56), so don't narrow it back.
+type SqlQueryNoTemplate = LiteralString | exp.Expr | Composable
 type SqlQuery = SqlQueryNoTemplate | Template
 
 

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/pgdevkit/testdb/schema.py

@@ -260,12 +260,15 @@ async def _insert_test_data(
                     await copy.write_row(tuple(row[c] for c in col_names))
             return
 
+        for col in col_names:
+            # registered on this cursor only, never on the (possibly pooled) connection
+            await complex_helper.register_on(cur, complex_types.get(col))
         for row in rows:
             for col in col_names:
                 info = complex_types.get(col)
                 if info is not None and isinstance(row[col], (dict, list)):
-                    # composite/enum/JSONB: needs psycopg-registered-type
-                    # conversion. Anything else (plain scalars, and native
+                    # composite/enum/JSONB: needs conversion to the
+                    # Python objects registered on `cur` above. Anything else (plain scalars, and native
                     # Postgres arrays like text[], which aren't "complex" —
                     # psycopg already adapts a Python list to those natively)
                     # is left untouched.

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/pyproject.toml

@@ -11,7 +11,7 @@ packages = ["pgdevkit"]
 
 [project]
 name = "pgdevkit"
-version = "0.15.0"
+version = "0.15.2"
 description = "A helper for developing with Postgres"
 readme = "README.md"
 requires-python = ">=3.14"

{pgdevkit-0.15.0 → pgdevkit-0.15.2}/skills/pgdevkit/SKILL.md

@@ -234,7 +234,7 @@ deleted = await execute("DELETE FROM sessions WHERE user_id = %(id)s", {"id": 1}
 - Passing `con=` runs on *your* connection: `fetch_all` never commits, closes or releases it. Without it, a connection is borrowed from `pool=` (or the `set_default_pool()` pool) and released afterwards.
 - `statement_timeout=<seconds>` (optional) lets Postgres abort the query after that long, raising `QueryCanceled`; it is applied with `SET LOCAL` semantics for this call only (a borrowed `con` gets its previous value back). A plain value *replaces* the connection's own setting in both directions, so a per-call 900 s overrides a database-level safety net of 600 s; `statement_timeout=at_most(120)` (`from pgdevkit.db import at_most`) never loosens it: the effective timeout is `min(current, 120)`.
 - `cancel=` takes an `asyncio.Event`: set it (e.g. when the HTTP client disconnects) and the running query is cancelled on the server, raising `psycopg.errors.QueryCanceled`. If the awaiting task is cancelled instead, the server-side query is cancelled too and `CancelledError` propagates as usual (not `QueryCanceled`), so no query keeps running unattended. Don't use `cancel=` on a `con` that concurrent tasks share (it aborts whatever statement is running on it); if the cancel request itself can't be delivered, the connection is closed. After `QueryCanceled` a borrowed `con` is usable again once you `await con.rollback()` (autocommit connections need nothing). To stream a whole result as JSON from FastAPI (it wires the disconnect itself) see `PostgresJsonResponse` below.
-- `query` is typed `SqlQuery`: a literal string (`SqlLoader.load_sql()`), a sqlglot expression, a `psycopg.sql` composable or a t-string. A plain `str` fails type checking on purpose, so don't build SQL with f-strings/concatenation. A sqlglot expression is only as safe as the strings it was built from (its builders parse plain strings as SQL): pass user values as `exp.Placeholder` + `params`, never as literals/raw text. A literal `%` in a sqlglot expression (`LIKE 'a%'`, modulo) just works, with or without `params`; in a plain string or `psycopg.sql` composable it is psycopg's rule: write `%%` when you pass `params`. A t-string carries its own values, so don't also pass `params`.
+- `query` is typed `SqlQuery`: a literal string (`SqlLoader.load_sql()`), a sqlglot expression (any `exp.Expr`, so Select, Union, Subquery and other `exp.Query` values are fine), a `psycopg.sql` composable or a t-string. A plain `str` fails type checking on purpose, so don't build SQL with f-strings/concatenation. A sqlglot expression is only as safe as the strings it was built from (its builders parse plain strings as SQL): pass user values as `exp.Placeholder` + `params`, never as literals/raw text. A literal `%` in a sqlglot expression (`LIKE 'a%'`, modulo) just works, with or without `params`; in a plain string or `psycopg.sql` composable it is psycopg's rule: write `%%` when you pass `params`. A t-string carries its own values, so don't also pass `params`.
 
 ### Dynamic SQL
 
@@ -262,13 +262,25 @@ See [`references/temporal-tables.md`](references/temporal-tables.md) for row-lev
 ### Custom Postgres types (composites, enums)
 
 `pgdevkit.db.complex_types.ComplexHelper` detects a table's composite/enum/JSONB
-columns and converts plain dict/list values into the psycopg-registered types
-those columns need. `pgdevkit.testdb.schema`'s test-data seeding uses it automatically; the `pg_*` CRUD helpers only do
+columns and converts plain dict/list values into the Python objects psycopg sends
+for those columns. `pgdevkit.testdb.schema`'s test-data seeding uses it automatically; the `pg_*` CRUD helpers only do
 when you pass `complex_helper="auto"` (or `complex_helper=ComplexHelper(con, normalizers={...})` to reshape a value
 before conversion, keyed by composite type name, e.g. backfilling missing locale keys); without it values are not
 converted and reads are a plain `SELECT *`.
 
-What the helper learns from the catalog (types, column types, adapter registrations) is cached **per connection**, so
+**The helper never registers types on the connection** (psycopg's registration is connection-global and would stick
+to a pooled connection: a plain `SELECT` of an enum would return psycopg's `Enum` member instead of the label, #54).
+Writes register the enum/composite adapters on the one cursor that runs the statement, only for the columns where a
+dict/list value needed converting; reads (`pg_retrieve*`) use `to_jsonb(...)` and register nothing. So after any
+`pg_*` call the connection reads enums as label strings and composites as text (`'(1,2)'`), as before the call. `RETURNING` rows of
+`pg_insert`/`pg_update_dict`/`pg_upsert_dict` (hence `pg_update`/`pg_upsert`) also use `to_jsonb(...)` for such
+columns: with a helper they are plain Python values (a composite as a dict, an enum as its label) whichever columns
+were written; without one it is a plain `RETURNING *`.
+Using `recursive_convert` yourself: take `info` from `load_all_complex_types`/`load_complex_type`, call
+`await helper.register_on(cur, info)` on your cursor, and execute the converted value on that same `cur` (the
+connection as context would be a global registration again).
+
+What the helper learns from the catalog (types and column types) is cached **per connection**, so
 creating a `ComplexHelper(con)` per call (`pg_retrieve(con, M, pks, complex_helper=ComplexHelper(con))`) queries the
 catalog only on that connection's first use, not every time. The cache cannot see later schema changes
 (`CREATE TYPE`, `ALTER TYPE ... ADD VALUE`, new columns), nor DDL that was rolled back after it had been cached (test