matrx-orm 3.1.2__tar.gz → 3.1.4__tar.gz

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/CLAUDE.md

@@ -20,6 +20,7 @@ matrx-orm is a standalone async Postgres ORM (`Model`, `QueryBuilder`, migration
 - **`MatrxEntity`** (`matrx_orm/entity.py`) — **the platform entity contract as an ORM primitive.** A table registered in `platform.entity_types` generates as a `MatrxEntity` (everything else stays a plain `Model`) and gets the behavior instead of every consumer hand-wiring it: `await page.versions()` (not a query on `history.row_versions`), `await page.restore(3)` (not a `version_restore` RPC + a cache bust — `restore()` busts the cache itself), `Note.alive()` / `.dead()` / `.soft_delete()` / `.undelete()`, and `Page.capabilities()`. Capability flags (`_entity_token`, `_is_versioned`, `_has_soft_delete`, `_is_org_scoped`, `_rls_variant`) are **stamped by the generator from the LIVE DATABASE** (does the table really carry the `_version_capture` trigger? a `deleted_at` column?) — never from `entity_types`' own flags, which are wrong on 61 of its 242 rows. **The database is truth; a flag is a claim.** Using a capability an entity does not declare RAISES (`EntityCapabilityError`) and names the fix — it never returns silently-wrong data, and `is_deleted` refuses to answer from a column a partial fetch never loaded. Drift guard: `scripts/check_entity_drift.py` (loud, non-blocking, in `release.sh`).
 - **Change tracking** — every instance records which columns were **loaded** (a SELECTed row, or the caller's kwargs) and which were **changed**. `save()` writes only those, so `.only("id","email")` + `save()` can never NULL the columns it never fetched (it used to — silent data loss). Mutable containers (jsonb/arrays) are always rewritten when loaded, because `obj.metadata["k"]=1` cannot be seen by `__setattr__`; **reassigning** (`obj.metadata = {...}`) is tracked precisely and is the sanctioned way. `Model._hydrate()` is the ONE row→model entry point.
 - **Read-only models** (`_read_only = True`): a model that can be queried but NEVER written through the ORM — every write path raises `ReadOnlyModelError` (enforced at `QueryExecutor.insert/bulk_insert/upsert/update/delete`, so it covers all of `create`/`save`/`update`/`delete`/`update_where`/`delete_where`/`upsert`/`bulk_*`). Used for DB-managed tables the app must not mutate. The canonical **`auth.users`** model is baked into the schema generator (`get_string_user_model`) as the full, curated Supabase auth schema (safe columns only — no password/token fields), `_read_only`, bound per-database. Read a user's profile/metadata there; mutate users only through Supabase Auth (GoTrue).
+- **Connection-level type codecs** (`async_db_manager._init_connection_default`): every pooled connection decodes **uuid → `str`**, **vector → `list[float]`**, and **json/jsonb → `dict`/`list`** at the driver boundary — the TYPE decides the decode, never the call path. So `call_function` scalar/rows, raw fetches, and field hydration all agree; a caller must **never `json.loads` a fetched json/jsonb value** (it's already decoded; the 2026-07-13 `build_agent_context` crash was a jsonb RPC fetched pre-codec as a raw string). Encoding: a pre-serialised JSON `str` passes through untouched; a native dict/list is dumped (`_encode_json`). Pinned by `tests/level1/test_connection_codecs.py`.
 - **Query layer**: `QueryBuilder`, expressions (`F`, `Q`), window functions, CTEs, subqueries.
 - **Migrations**: `MigrationDB`, `MigrationLoader`, `MigrationExecutor`, `makemigrations`, `migrate` — migrations declare explicit `dependencies` and apply in topological order (no multi-head/merge support yet).
 - **Admin router** (FastAPI): `admin_router` exposes read/write endpoints for any registered model.

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: matrx-orm
-Version: 3.1.2
+Version: 3.1.4
 Summary: Async-first PostgreSQL ORM with bidirectional migrations, schema introspection, many-to-many relationships, and built-in state caching
 Project-URL: Homepage, https://github.com/AI-Matrix-Engine/aidream-current
 Project-URL: Repository, https://github.com/AI-Matrix-Engine/aidream-current

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/matrx_orm/core/async_db_manager.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import asyncio
+import json
 import logging
 import os
 import traceback
@@ -143,6 +144,23 @@ def _decode_vector(value: str) -> list[float]:
     return [float(x) for x in inner.split(",")]
 
 
+def _encode_json(value: Any) -> str:
+    """Encode a Python value bound to a ``json``/``jsonb`` parameter.
+
+    A ``str`` passes through untouched — every existing ORM write path
+    (``JSONField.get_db_prep_value``, ``JsonbMerge``, ``call_function``'s
+    ``_bind_args``) pre-serialises to a JSON string, and asyncpg's built-in
+    codec (which this replaces) also required ``str``. Anything else (dict,
+    list, int, ...) is serialised here with the same coercion hook the field
+    layer uses, so binding a native container directly now also works.
+    """
+    if isinstance(value, str):
+        return value
+    from matrx_orm.core.fields import JSONBField
+
+    return json.dumps(value, default=JSONBField._json_default)
+
+
 async def _init_connection_default(conn: asyncpg.Connection) -> None:
     """Register custom type codecs on each new connection.
 
@@ -166,6 +184,28 @@ async def _init_connection_default(conn: asyncpg.Connection) -> None:
         format="text",
     )
 
+    # json/jsonb → dict/list (decode) and str-or-native → json text (encode).
+    #
+    # Without this codec asyncpg hands json/jsonb values back as RAW JSON
+    # STRINGS. The field layer papered over that (``JSONField.to_python`` does
+    # ``json.loads``), but every fetch path that bypasses field hydration —
+    # ``call_function`` scalar/rows, ``field=`` dereference, sql_executor rows
+    # — returned a ``str`` where the caller expected a ``dict``. That is
+    # exactly the 2026-07-13 ``build_agent_context`` crash ("'str' object has
+    # no attribute 'get'"): a jsonb-returning RPC moved from PostgREST (which
+    # decodes) to ``call_function`` (which didn't). Decoding at the driver
+    # boundary kills the whole class: the DRIVER knows the value is jsonb, so
+    # the type — not the call path — decides the decode. ``to_python`` already
+    # passes non-str values through, so field hydration is unaffected.
+    for _json_type in ("json", "jsonb"):
+        await conn.set_type_codec(
+            _json_type,
+            encoder=_encode_json,
+            decoder=json.loads,
+            schema="pg_catalog",
+            format="text",
+        )
+
     # pgvector → list[float]
     #
     # The ``vector`` type lives in whichever schema the pgvector extension was

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/matrx_orm/core/db_function.py

@@ -14,7 +14,11 @@ hand-write `SELECT schema.func(...)` itself — call this instead.
 
 Three return modes:
   * ``mode="scalar"`` (default) — the function returns one usable value
-    (text, integer, boolean, ...); we ``fetchval`` it.
+    (text, integer, boolean, ...); we ``fetchval`` it. A ``json``/``jsonb``
+    return value arrives as a decoded ``dict``/``list`` (never a raw JSON
+    string) — the pool's connection-level codec
+    (``async_db_manager._init_connection_default``) decodes at the driver
+    boundary. Callers must NOT ``json.loads`` a result.
   * ``mode="none"`` — the function's return value is discarded (e.g. it
     returns a composite/row type the caller doesn't need decoded). We
     ``execute`` the statement instead of ``fetchval``, which never attempts
@@ -37,12 +41,12 @@ composite; the single field still comes back scalar, through ``fetchval``.
 alongside a row-set, is a caller bug, not a valid combination).
 
 A ``dict``/``list`` positional argument is JSON-serialised and its
-placeholder is cast explicitly (``$N::jsonb``) — asyncpg has no jsonb
-encoder, so a bare dict would bind as an unsupported parameter type and
-raise. This mirrors the same trick ``JsonbMerge`` already uses for
-ORM-generated ``UPDATE`` statements (see
-``matrx_orm.core.expressions.JsonbMerge``) — it is not a special case, it is
-how every jsonb-typed function parameter must be bound.
+placeholder is cast explicitly (``$N::jsonb``). The pool's connection-level
+json codec (``async_db_manager._encode_json``) passes the pre-serialised
+string through untouched, so this never double-encodes. This mirrors the
+same trick ``JsonbMerge`` already uses for ORM-generated ``UPDATE``
+statements (see ``matrx_orm.core.expressions.JsonbMerge``) — it is not a
+special case, it is how every jsonb-typed function parameter is bound.
 
 A genuine Postgres **array** parameter (``uuid[]``, ``text[]``, ...) is a
 different shape from jsonb — asyncpg natively encodes a Python ``list`` as a

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/matrx_orm/core/expressions.py

@@ -708,9 +708,10 @@ class JsonbMerge(Func):
     sibling, expressible directly on ``Model.update_where`` (no manager
     needed).
 
-    ``value`` is JSON-serialised here (never left to the caller) so it binds
-    as a plain string param cast to ``jsonb`` — asyncpg has no jsonb codec
-    registered, so a bare dict param would bind as text and fail the cast.
+    ``value`` is JSON-serialised here (never left to the caller) and binds
+    as a string param cast to ``jsonb``. The pool's connection-level json
+    codec (``async_db_manager._encode_json``) passes pre-serialised strings
+    through untouched, so this never double-encodes.
 
     Usage::
 

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/matrx_orm/query/executor.py

@@ -2,6 +2,7 @@ from __future__ import annotations
 
 import json
 import re
+from types import NotImplementedType
 from typing import TYPE_CHECKING, Any
 
 from matrx_utils import vcprint
@@ -27,6 +28,55 @@ _SKIP_PARAM = object()
 
 _PARAM_RE = re.compile(r"\$(\d+)")
 
+_TOP_LEVEL_WHERE_STOPPERS = (
+    "ORDER BY",
+    "GROUP BY",
+    "HAVING",
+    "LIMIT",
+    "OFFSET",
+    "FOR UPDATE",
+)
+
+
+def _strip_top_level_trailing_clauses(sql_fragment: str) -> str:
+    """Drop outer-query ORDER BY / LIMIT / … suffixes from a WHERE tail.
+
+    ``_extract_where_clause`` peels the WHERE tail off a built SELECT. The tail
+    must not include the SELECT's own trailing clauses — but it MUST keep any
+    LIMIT/ORDER BY that lives *inside* a subquery (``id IN (SELECT … LIMIT n)``).
+    Naïve ``.split("LIMIT")`` truncates at the inner LIMIT and breaks bounded
+    batch updates (admin_persistence dismiss/resolve).
+    """
+    if not sql_fragment:
+        return sql_fragment
+    depth = 0
+    cut_at: int | None = None
+    i = 0
+    n = len(sql_fragment)
+    while i < n:
+        ch = sql_fragment[i]
+        if ch == "(":
+            depth += 1
+            i += 1
+            continue
+        if ch == ")":
+            depth = max(0, depth - 1)
+            i += 1
+            continue
+        if depth == 0:
+            for stopper in _TOP_LEVEL_WHERE_STOPPERS:
+                slen = len(stopper)
+                if sql_fragment[i : i + slen] == stopper:
+                    before = sql_fragment[i - 1] if i > 0 else " "
+                    if before.isalnum() or before == "_":
+                        break
+                    cut_at = i if cut_at is None else min(cut_at, i)
+                    break
+        i += 1
+    if cut_at is not None:
+        return sql_fragment[:cut_at].strip()
+    return sql_fragment.strip()
+
 
 def _escape_like(value: Any) -> str:
     """Escape LIKE/ILIKE metacharacters so user input matches LITERALLY.
@@ -820,11 +870,7 @@ class QueryExecutor:
         where_clause = ""
         if "WHERE" in base_query:
             after_where = base_query.split("WHERE", 1)[1].strip()
-            # Strip trailing clauses
-            for stopper in ("ORDER BY", "GROUP BY", "HAVING", "LIMIT", "OFFSET", "FOR UPDATE"):
-                if stopper in after_where:
-                    after_where = after_where.split(stopper)[0].strip()
-            where_clause = after_where
+            where_clause = _strip_top_level_trailing_clauses(after_where)
         return where_clause, where_params
 
     async def _execute(self) -> list[dict[str, Any]]:
@@ -904,15 +950,14 @@ class QueryExecutor:
 
     async def _try_write_dict(
         self, query: dict[str, Any]
-    ) -> list[dict[str, Any]] | type[NotImplemented]:
+    ) -> list[dict[str, Any]] | NotImplementedType:
         """Call execute_write_dict on the adapter.
 
         Returns ``NotImplemented`` if the adapter does not override the hook
         (default ``BaseAdapter`` behaviour) so callers can fall through to the
         SQL path.
         """
-        result = await self.db.execute_write_dict(query)
-        return result  # type: ignore[return-value]
+        return await self.db.execute_write_dict(query)
 
     async def insert(self, query: dict[str, Any]) -> dict[str, Any]:
         """INSERT a single row, RETURNING *."""

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "matrx-orm"
-version = "3.1.2"
+version = "3.1.4"
 description = "Async-first PostgreSQL ORM with bidirectional migrations, schema introspection, many-to-many relationships, and built-in state caching"
 readme = "README.md"
 license = { text = "MIT" }

matrx_orm-3.1.4/tests/level1/test_connection_codecs.py

@@ -0,0 +1,93 @@
+"""Level 1: connection-level type codecs (``_init_connection_default``).
+
+The pool registers codecs so driver-boundary decoding is uniform across EVERY
+fetch path — model hydration, ``call_function``, sql_executor. The json/jsonb
+codec exists because asyncpg's default returns jsonb as a RAW JSON STRING;
+only the field layer decoded it, so any path that bypassed field hydration
+(e.g. ``call_function(mode="scalar")`` on a jsonb-returning RPC) handed
+callers a ``str`` where they expected a ``dict`` — the 2026-07-13
+``build_agent_context`` crash. These tests pin the registration and the
+encoder's str-pass-through contract so that class of bug stays extinct.
+"""
+
+from __future__ import annotations
+
+import json
+from unittest.mock import AsyncMock
+
+import pytest
+
+from matrx_orm.core.async_db_manager import (
+    _encode_json,
+    _init_connection_default,
+)
+
+
+class _FakeConn:
+    def __init__(self) -> None:
+        self.set_type_codec = AsyncMock()
+        # The vector-schema lookup in _init_connection_default; None = pgvector
+        # not installed, the only legitimate skip.
+        self.fetchval = AsyncMock(return_value=None)
+
+
+class TestJsonCodecRegistration:
+    @pytest.mark.asyncio
+    async def test_registers_json_and_jsonb_decoders(self) -> None:
+        conn = _FakeConn()
+        await _init_connection_default(conn)  # type: ignore[arg-type]
+        registered = {
+            call.args[0]: call.kwargs for call in conn.set_type_codec.await_args_list
+        }
+        for type_name in ("json", "jsonb"):
+            assert type_name in registered, (
+                f"{type_name} codec not registered — jsonb values fetched outside "
+                f"field hydration (call_function scalar/rows) will come back as "
+                f"raw JSON strings again (the build_agent_context crash class)."
+            )
+            kwargs = registered[type_name]
+            assert kwargs["decoder"] is json.loads
+            assert kwargs["encoder"] is _encode_json
+            assert kwargs["schema"] == "pg_catalog"
+
+    @pytest.mark.asyncio
+    async def test_uuid_codec_still_registered(self) -> None:
+        conn = _FakeConn()
+        await _init_connection_default(conn)  # type: ignore[arg-type]
+        assert "uuid" in {c.args[0] for c in conn.set_type_codec.await_args_list}
+
+
+class TestEncodeJson:
+    def test_str_passes_through_untouched(self) -> None:
+        # Every existing write path (JSONField.get_db_prep_value, JsonbMerge,
+        # call_function._bind_args) pre-serialises to str; double-encoding
+        # would corrupt every jsonb write in the platform.
+        assert _encode_json('{"a": 1}') == '{"a": 1}'
+        assert _encode_json("plain") == "plain"
+
+    def test_dict_and_list_serialised(self) -> None:
+        assert json.loads(_encode_json({"a": 1})) == {"a": 1}
+        assert json.loads(_encode_json([1, "b"])) == [1, "b"]
+
+    def test_non_native_values_degrade_via_field_hook(self) -> None:
+        # Same coercion hook the field layer uses (JSONBField._json_default):
+        # a weird value degrades instead of crashing the write.
+        class Weird:
+            __slots__ = ()
+
+            def __repr__(self) -> str:
+                return "<weird>"
+
+        # The exact degrade shape belongs to _json_default's own contract;
+        # what THIS codec guarantees is: never crash, always emit valid JSON.
+        out = _encode_json({"w": Weird()})
+        assert isinstance(json.loads(out), dict)
+
+
+class TestDecodeContract:
+    def test_decoder_yields_native_containers(self) -> None:
+        # json.loads is the registered decoder — pin the shapes callers rely on.
+        assert json.loads('{"variables": {}}') == {"variables": {}}
+        assert json.loads("[1, 2]") == [1, 2]
+        assert json.loads('"scalar"') == "scalar"
+        assert json.loads("null") is None

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/tests/level1/test_subquery_in_filter.py

@@ -51,10 +51,7 @@ class TestSubqueryInFilter:
     def test_renders_in_not_equals(self):
         Model_ = _make_model()
         inner = (
-            Model_.filter(recovered_at__isnull=True)
-            .order_by("-failed_at")
-            .limit(50)
-            .select("id")
+            Model_.filter(recovered_at__isnull=True).order_by("-failed_at").limit(50).select("id")
         )
 
         qb = Model_.filter(id__in=Subquery(inner))
@@ -98,3 +95,18 @@ class TestSubqueryInFilter:
         assert "recovered_at IS NULL" in cond
         assert "table_target = $" in cond
         assert "cx_message" in params
+
+    def test_extract_where_clause_preserves_inner_limit(self):
+        """Bounded batch dismiss/resolve: update_where(id__in=Subquery(...))
+        must not truncate the inner SELECT at its LIMIT."""
+        Model_ = _make_model()
+        inner = (
+            Model_.filter(recovered_at__isnull=True).order_by("-failed_at").limit(50).select("id")
+        )
+        qb = Model_.filter(id__in=Subquery(inner))
+        executor = qb._get_executor()
+        where_clause, where_params = executor._extract_where_clause()
+        assert where_clause.endswith(")")
+        assert "LIMIT $" in where_clause
+        assert "ORDER BY failed_at DESC" in where_clause
+        assert where_params == [50]

{matrx_orm-3.1.2 → matrx_orm-3.1.4}/tests/schema/test_composite_pk_fk_generation.py

@@ -1,7 +1,7 @@
 # Pins the generator behavior for composite primary keys whose columns are ALSO
 # foreign keys (live shape: platform.association_types — PK (source_type,
 # target_type), both FKs to platform.entity_types(token)). Regression context:
-# KNOWN_DEFECTS 2026-07-06 — a regenerate emitted AssociationTypes with no
+# FOUND_DEFECTS 2026-07-06 — a regenerate emitted AssociationTypes with no
 # primary_key=True and every db.models import crashed at boot. Root cause was a
 # PK-less live table; these tests pin BOTH layers: (1) FK+PK columns must emit
 # primary_key=True, (2) a PK-less table must fail generation loudly instead of