matrx-orm 3.1.32__tar.gz → 3.1.34__tar.gz

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/CLAUDE.md

@@ -23,6 +23,7 @@ Every table that becomes a Model needs row identity: a single-column primary key
 - **Core model layer**: `Model`, `BaseManager`, `BaseDTO`, `ModelView`, `model_registry`, plus 50+ field types (`CharField`, `IntegerField`, `ForeignKey`, `ManyToManyField`, `JSONField`, …).
 - **`MatrxEntity`** (`matrx_orm/entity.py`) — **the platform entity contract as an ORM primitive.** A table registered in `platform.entity_types` generates as a `MatrxEntity` (everything else stays a plain `Model`) and gets the behavior instead of every consumer hand-wiring it: `await page.versions()` (not a query on `history.row_versions`), `await page.restore(3)` (not a `version_restore` RPC + a cache bust — `restore()` busts the cache itself), `Note.alive()` / `.dead()` / `.soft_delete()` / `.undelete()`, and `Page.capabilities()`. Capability flags (`_entity_token`, `_is_versioned`, `_has_soft_delete`, `_is_org_scoped`, `_rls_variant`) are **stamped by the generator from the LIVE DATABASE** (does the table really carry the `_version_capture` trigger? a `deleted_at` column?) — never from `entity_types`' own flags, which are wrong on 61 of its 242 rows. **The database is truth; a flag is a claim.** Using a capability an entity does not declare RAISES (`EntityCapabilityError`) and names the fix — it never returns silently-wrong data, and `is_deleted` refuses to answer from a column a partial fetch never loaded. Drift guard: `scripts/check_entity_drift.py` (loud, non-blocking, in `release.sh`).
 - **Change tracking** — every instance records which columns were **loaded** (a SELECTed row, or the caller's kwargs) and which were **changed**. `save()` writes only those, so `.only("id","email")` + `save()` can never NULL the columns it never fetched (it used to — silent data loss). Mutable containers (jsonb/arrays) are always rewritten when loaded, because `obj.metadata["k"]=1` cannot be seen by `__setattr__`; **reassigning** (`obj.metadata = {...}`) is tracked precisely and is the sanctioned way. `Model._hydrate()` is the ONE row→model entry point.
+- **Upsert writes only what the caller supplied.** `upsert` / `bulk_upsert` / `upsert_with_conflict` materialize every field's Python default so the **INSERT** half is complete — but `ON CONFLICT DO UPDATE SET` covers **only the columns present in the caller's `data`** (an explicit `update_fields` still wins). Letting defaults reach the UPDATE half silently RESET every defaulted column on each re-upsert of an existing row: it once reset scraped-and-analyzed `research.rs_source` rows to `scrape_status='pending', server_attempts=0` (a watchdog then finalized them `'failed'`) and undid users' manual `is_included=false`. A caller-supplied `None` is excluded too — it is dropped from the INSERT, and `EXCLUDED.<col>` on an omitted column resolves to that column's **DEFAULT**. An upsert of only conflict keys means "ensure this row exists" and reconciles to a no-op self-assignment, never a raise (a bare `DO NOTHING` would break `RETURNING *`). Pinned by `tests/level1/test_upsert_default_leak.py`.
 - **Read-only models** (`_read_only = True`): a model that can be queried but NEVER written through the ORM — every write path raises `ReadOnlyModelError` (enforced at `QueryExecutor.insert/bulk_insert/upsert/update/delete`, so it covers all of `create`/`save`/`update`/`delete`/`update_where`/`delete_where`/`upsert`/`bulk_*`). Used for DB-managed tables the app must not mutate. The canonical **`auth.users`** model is baked into the schema generator (`get_string_user_model`) as the full, curated Supabase auth schema (safe columns only — no password/token fields), `_read_only`, bound per-database. Read a user's profile/metadata there; mutate users only through Supabase Auth (GoTrue).
 - **Connection-level type codecs** (`async_db_manager._init_connection_default`): every pooled connection decodes **uuid → `str`**, **vector → `list[float]`**, and **json/jsonb → `dict`/`list`** at the driver boundary — the TYPE decides the decode, never the call path. So `call_function` scalar/rows, raw fetches, and field hydration all agree; a caller must **never `json.loads` a fetched json/jsonb value** (it's already decoded; the 2026-07-13 `build_agent_context` crash was a jsonb RPC fetched pre-codec as a raw string). Encoding: a pre-serialised JSON `str` passes through untouched; a native dict/list is dumped (`_encode_json`). Pinned by `tests/level1/test_connection_codecs.py`.
 - **Query layer**: `QueryBuilder`, typed predicates/projections (`F`, `Q`,
@@ -37,7 +38,7 @@ Every table that becomes a Model needs row identity: a single-column primary key
 - **Signals**: `pre_create`, `post_create`, `pre_save`, `post_save`, `pre_delete`, `post_delete`.
 - **Schema builder** (`matrx_orm.schema_builder`): code generation for Python + TypeScript type definitions from the DB schema. Postgres **VIEWs** are emitted as **read-only `Model` classes** into the same schema models file (on by default; per-entry `output.include_views: false` opts a schema out, `output.exclude_views: [...]` skips individual views). View models get no manager/DTO/base-class/auto_config (views are not writable), synthesize `_primary_keys` from an `id` column (else the first column, for cache-key/`load_by_id` only), and emit plain typed fields with **no `ForeignKey`** (views carry no FK metadata — nothing cross-schema to resolve). Never hand-write a view model again.
 - **Secrets battery** (`matrx_orm.secrets_battery`): the ONE canonical user-credential vault — `users.user_secrets` CRUD + the platform's single Fernet primitive, keyed by `CREDENTIALS_ENCRYPTION_KEY`. Any package: `configure_secrets(db_config_name=...)` once, then `await require_user_secret(user_id, key)`. No package ever calls another service to resolve a credential. Contract: [`matrx_orm/secrets_battery/FEATURE.md`](matrx_orm/secrets_battery/FEATURE.md).
-- **Polymorphic associations** (`matrx_orm.associations`): `EntityRegistry` (token→Model) + `Associations` — a registry-driven, **ownership-gated** edge engine over a host's polymorphic edge table. Every `source→target` hop is org-scoped *and* the resolved target's `org_id` is re-verified (fail-closed, screams on mismatch), so a hop can't widen access past the tenant boundary. Hardcodes no table names — the host wires its own edge + token-registry models.
+- **Polymorphic associations** (`matrx_orm.associations`): `EntityRegistry` (token→Model) + `Associations` — a registry-driven, **ownership-gated** edge engine over a host's polymorphic edge table. Every `source→target` hop is org-scoped *and* the resolved target's `org_id` is re-verified (fail-closed, screams on mismatch), so a hop can't widen access past the tenant boundary. **System vocabulary is exempt from the org-equality check** (existence still required): a model generated with `_rls_variant="system"` is platform-shared dictionary data (seo.keyword/topic) every tenant legitimately links to; the edge still carries the caller's org. Hardcodes no table names — the host wires its own edge + token-registry models.
 
 Public API is defined explicitly in `matrx_orm/__init__.py` (100+ exports). Treat changes there as semver-breaking.
 

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: matrx-orm
-Version: 3.1.32
+Version: 3.1.34
 Summary: Async-first PostgreSQL ORM with bidirectional migrations, schema introspection, many-to-many relationships, and built-in state caching
 Project-URL: Homepage, https://github.com/AI-Matrix-Engine/aidream-current
 Project-URL: Repository, https://github.com/AI-Matrix-Engine/aidream-current

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/matrx_orm/associations.py

@@ -400,12 +400,22 @@ class Associations:
         return await self._load_gated(source_type, ids, org_id)
 
     async def _verify_in_org(self, model: type[Model], ref: EntityRef, org_id: str) -> None:
-        """Write-side gate: the endpoint must exist (RLS-visible) and be in ``org_id``."""
+        """Write-side gate: the endpoint must exist (RLS-visible) and be in ``org_id``.
+
+        **System vocabulary is exempt from the org-equality check** (existence
+        is still required): a model generated with ``_rls_variant == "system"``
+        is platform-shared dictionary data (e.g. seo.keyword / seo.topic) that
+        every tenant legitimately links to — its rows live in the system org by
+        design, so org-equality would refuse every registered edge to it. The
+        edge row itself still carries the CALLER's org (the tenancy boundary).
+        """
         entity = await model.load_by_id_or_none(ref.id)
         if entity is None:
             raise CrossOrgAssociationError(
                 f"cannot associate {ref.type}:{ref.id} — not found or not accessible in org {org_id}"
             )
+        if getattr(model, "_rls_variant", None) == "system":  # orm-getattr-ok: generator-stamped capability flag
+            return
         entity_org = getattr(entity, self._org, None)
         if entity_org is not None and str(entity_org) != str(org_id):
             raise CrossOrgAssociationError(
@@ -419,6 +429,9 @@ class Associations:
             entity = await model.load_by_id_or_none(rid)
             if entity is None:
                 continue  # layer 1: RLS-filtered, soft-deleted, or absent
+            if getattr(model, "_rls_variant", None) == "system":  # orm-getattr-ok: generator-stamped capability flag
+                out.append(entity)  # system vocabulary: platform-shared, org check N/A
+                continue
             entity_org = getattr(entity, self._org, None)
             if entity_org is not None and str(entity_org) != str(org_id):
                 # layer 2 fired => layer 1 / RLS already failed. Scream.

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/matrx_orm/operations/conflict_writes.py

@@ -460,8 +460,21 @@ async def upsert_with_conflict(
     placeholders = [f"${i + 1}" for i in range(len(values))]
 
     resolved_set_fields: dict[str, Any] = dict(set_fields or {})
+    # ONLY columns the CALLER supplied — `prepared` also carries materialized
+    # model defaults, which belong in the INSERT half but must never reach
+    # DO UPDATE SET (they silently reset real state on every re-upsert of an
+    # existing row). This matches the documented contract above: "Any column in
+    # `data` NOT given an explicit entry defaults to Excluded(col)".
     for col in columns:
-        resolved_set_fields.setdefault(col, Excluded(col))
+        if col in data:
+            resolved_set_fields.setdefault(col, Excluded(col))
+    if not resolved_set_fields:
+        # The caller supplied nothing but conflict keys and no set_fields —
+        # exactly one meaning ("ensure this row exists"), so reconcile to a
+        # no-op self-assignment rather than emitting an empty SET clause. A
+        # bare DO NOTHING is wrong here: it would return no row, breaking the
+        # RETURNING * contract this function promises.
+        resolved_set_fields[columns[0]] = Excluded(columns[0])
     for col in resolved_set_fields:
         if not isinstance(col, str) or not _IDENT_RE.match(col):
             raise ValueError(f"set_fields column must be a plain identifier; got {col!r}")

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/matrx_orm/operations/create.py

@@ -38,6 +38,40 @@ def _normalise_on_conflict(on_conflict: OnConflictArg) -> tuple[list[str], Confl
     )
 
 
+def _conflict_update_fields(
+    model_cls: type[Model],
+    caller_keys: set[str],
+    conflict_fields: list[str],
+) -> list[str]:
+    """Which columns an upsert's ``ON CONFLICT DO UPDATE SET`` may touch.
+
+    ONLY the columns the caller actually supplied. The INSERT half of an upsert
+    legitimately materializes every model default (a brand-new row must get
+    them), but letting those defaults reach the UPDATE half silently RESETS real
+    state on every re-upsert of an existing row — no error, no log.
+
+    That is not hypothetical: `research/search.py` upserts search hits while
+    deliberately omitting `scrape_status` / `server_attempts` / `is_included`,
+    expecting DO UPDATE to leave them alone. Instead every re-discovery of a
+    known URL reset a scraped, analyzed source back to 'pending'/0, and a
+    user's manual "exclude" was silently undone. `save()` already had this
+    discipline (it writes only loaded/changed columns); `upsert()` never did.
+
+    When the caller supplied nothing BUT conflict keys there is exactly one
+    thing they can mean — "ensure this row exists" — so reconcile to a no-op
+    self-assignment rather than raising. It keeps `RETURNING *` returning the
+    row, which a bare `DO NOTHING` would not.
+    """
+    updatable = [
+        name
+        for name in caller_keys
+        if isinstance(model_cls._fields.get(name), Field) and name not in conflict_fields
+    ]
+    if updatable:
+        return sorted(updatable)
+    return [conflict_fields[0]] if conflict_fields else []
+
+
 async def create(model_cls: type[Model], **kwargs: Any) -> Model:
     instance = model_cls(**kwargs)
     return await save(instance)
@@ -165,7 +199,17 @@ async def upsert(
     query["conflict_fields"] = conflict_fields
     if target is not None:
         query["conflict_target"] = target
-    query["update_fields"] = update_fields
+    # `prepared` carries materialized defaults so the INSERT half is complete;
+    # the UPDATE half must see ONLY what the caller supplied. See
+    # _conflict_update_fields.
+    query["update_fields"] = update_fields or _conflict_update_fields(
+        model_cls,
+        # Intersect with `prepared`: a caller-supplied None is already dropped
+        # from the INSERT, and `EXCLUDED.<col>` on a column absent from the
+        # INSERT list resolves to that column's DEFAULT — reintroducing the bug.
+        {key for key in data if key in prepared},
+        conflict_fields,
+    )
 
     executor = QueryExecutor(query)
     result = await executor.upsert(query)
@@ -198,6 +242,7 @@ async def bulk_upsert(
     conflict_fields, target = _normalise_on_conflict(on_conflict)
 
     data_list: list[dict[str, Any]] = []
+    supplied_keys: set[str] = set()
     for obj_data in objects_data:
         instance = model_cls(**obj_data)
         prepared: dict[str, Any] = {}
@@ -209,13 +254,19 @@ async def bulk_upsert(
                 if value is not None:
                     prepared[field_name] = field.get_db_prep_value(value)
         data_list.append(prepared)
+        supplied_keys.update(key for key in obj_data if key in prepared)
 
     query = QueryBuilder(model_cls)._build_query()
     query["data"] = data_list
     query["conflict_fields"] = conflict_fields
     if target is not None:
         query["conflict_target"] = target
-    query["update_fields"] = update_fields
+    # Union of the keys the caller actually supplied across all rows — a column
+    # nobody supplied must never be reset from a materialized default. See
+    # _conflict_update_fields.
+    query["update_fields"] = update_fields or _conflict_update_fields(
+        model_cls, supplied_keys, conflict_fields
+    )
 
     executor = QueryExecutor(query)
     created_instances = await executor.bulk_upsert(query)

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "matrx-orm"
-version = "3.1.32"
+version = "3.1.34"
 description = "Async-first PostgreSQL ORM with bidirectional migrations, schema introspection, many-to-many relationships, and built-in state caching"
 readme = "README.md"
 license = { text = "MIT" }

{matrx_orm-3.1.32 → matrx_orm-3.1.34}/tests/level1/test_associations.py

@@ -90,15 +90,25 @@ class ProjectModel(_FakeBase):
     _table_name = "ctx_projects"
 
 
+class VocabModel(_FakeBase):
+    """System-shared dictionary data (e.g. seo.keyword / seo.topic) — the
+    generator stamps `_rls_variant = "system"` on such models."""
+
+    _table_name = "keywords"
+    _rls_variant = "system"
+
+
 @pytest.fixture(autouse=True)
 def _reset_rows():
-    for cls in (EntityTypeRow, AssociationRow, NoteModel, ProjectModel):
+    for cls in (EntityTypeRow, AssociationRow, NoteModel, ProjectModel, VocabModel):
         cls._rows = {}
     yield
 
 
 def _token_registry():
-    return EntityRegistry(token_map={"note": NoteModel, "project": ProjectModel})
+    return EntityRegistry(
+        token_map={"note": NoteModel, "project": ProjectModel, "keyword": VocabModel}
+    )
 
 
 def _assoc():
@@ -223,6 +233,35 @@ async def test_associate_verify_rejects_absent_endpoint():
         await a.associate(EntityRef("note", "n1"), EntityRef("project", "p1"), org_id="org1")
 
 
+@pytest.mark.asyncio
+async def test_associate_allows_system_vocabulary_target_cross_org():
+    # seo.keyword / seo.topic pattern: the target lives in the SYSTEM org by
+    # design (_rls_variant="system"); the org-equality gate must not refuse it.
+    a = _assoc()
+    NoteModel.seed(id="n1", org_id="org1")
+    VocabModel.seed(id="k1", org_id="system-org")
+    edge = await a.associate(EntityRef("note", "n1"), EntityRef("keyword", "k1"), org_id="org1")
+    assert edge.org_id == "org1"  # the edge carries the CALLER's org
+
+
+@pytest.mark.asyncio
+async def test_associate_system_vocabulary_still_requires_existence():
+    a = _assoc()
+    NoteModel.seed(id="n1", org_id="org1")
+    with pytest.raises(CrossOrgAssociationError):
+        await a.associate(EntityRef("note", "n1"), EntityRef("keyword", "ghost"), org_id="org1")
+
+
+@pytest.mark.asyncio
+async def test_load_gated_returns_system_vocabulary_cross_org():
+    a = _assoc()
+    NoteModel.seed(id="n1", org_id="org1")
+    VocabModel.seed(id="k1", org_id="system-org")
+    await a.associate(EntityRef("note", "n1"), EntityRef("keyword", "k1"), org_id="org1")
+    targets = await a.targets_from(EntityRef("note", "n1"), "keyword", org_id="org1")
+    assert [t.id for t in targets] == ["k1"]
+
+
 @pytest.mark.asyncio
 async def test_associate_verify_false_skips_endpoint_check():
     a = _assoc()  # no endpoints seeded at all

matrx_orm-3.1.34/tests/level1/test_upsert_default_leak.py

@@ -0,0 +1,307 @@
+"""Level 1: an upsert's ``DO UPDATE SET`` must touch ONLY caller-supplied columns.
+
+REGRESSION — 2026-07-25 silent platform-wide data corruption.
+
+``upsert`` / ``bulk_upsert`` / ``upsert_with_conflict`` materialize every model
+field's Python-level default so the INSERT half of the statement is complete.
+Those defaults were then also landing in ``ON CONFLICT DO UPDATE SET``, so every
+re-upsert of an EXISTING row silently reset each defaulted column back to its
+default — no error, no log.
+
+Real damage: ``research/search.py`` upserts each search hit while deliberately
+omitting ``scrape_status`` / ``server_attempts`` / ``is_included``, expecting
+DO UPDATE to leave them alone. Instead, every re-discovery of an already-known
+URL reset a successfully scraped AND analyzed source back to
+``scrape_status='pending', server_attempts=0`` — after which a stuck-row
+watchdog finalized it as ``'failed'``. A user's manual "exclude this source"
+was silently undone by the same mechanism.
+
+``save()`` already had this discipline (it writes only loaded/changed columns);
+the upsert paths never did.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any
+
+import pytest
+from matrx_orm import ConflictTarget
+from matrx_orm.core.fields import BooleanField, IntegerField, TextField
+
+
+class _Meta:
+    def __init__(self, qualified: str) -> None:
+        self.qualified_table_name = qualified
+
+
+def _source_model() -> type:
+    """Mirrors the shape of ``research.rs_source`` that actually got corrupted:
+    caller-supplied identity columns plus defaulted state columns."""
+    fields: dict[str, Any] = {
+        "topic_id": TextField(),
+        "url": TextField(),
+        "title": TextField(),
+        "description": TextField(),
+        # The state columns. Every one carries a Python-level default, which is
+        # exactly what leaked into DO UPDATE SET.
+        "scrape_status": TextField(default="pending"),
+        "server_attempts": IntegerField(default=0),
+        "server_gave_up": BooleanField(default=False),
+        "is_included": BooleanField(default=True),
+        "is_stale": BooleanField(default=False),
+    }
+
+    class _RsSource:
+        _meta = _Meta("research.rs_source")
+        _fields = fields
+        _read_only = False
+        _database = "test_db"  # QueryBuilder.__init__ resolves this before _build_query
+
+        def __init__(self, **kw: Any) -> None:
+            for name in fields:
+                setattr(self, name, None)
+            for k, v in kw.items():
+                setattr(self, k, v)
+
+        @classmethod
+        def get_database_name(cls) -> str:
+            return "test_db"
+
+    return _RsSource
+
+
+def _set_clause(sql: str) -> str:
+    match = re.search(r"DO UPDATE SET (.*?) RETURNING", sql, re.DOTALL)
+    assert match, f"no DO UPDATE SET found in: {sql}"
+    return match.group(1)
+
+
+def _set_columns(sql: str) -> set[str]:
+    """Assignment targets in the SET clause. Splits on TOP-LEVEL commas only —
+    bulk_upsert renders `col = COALESCE(EXCLUDED.col, table.col)` for
+    partially-populated columns, whose inner comma must not split an entry."""
+    clause = _set_clause(sql)
+    parts: list[str] = []
+    depth = 0
+    current = ""
+    for char in clause:
+        if char == "(":
+            depth += 1
+        elif char == ")":
+            depth -= 1
+        if char == "," and depth == 0:
+            parts.append(current)
+            current = ""
+            continue
+        current += char
+    parts.append(current)
+    return {part.split("=")[0].strip() for part in parts if part.strip()}
+
+
+# The columns whose silent reset caused the incident.
+STATE_COLUMNS = {
+    "scrape_status",
+    "server_attempts",
+    "server_gave_up",
+    "is_included",
+    "is_stale",
+}
+
+
+@pytest.fixture
+def captured_sql(monkeypatch: pytest.MonkeyPatch) -> list[tuple[str, tuple[Any, ...]]]:
+    """Intercept at the executor boundary so create.py's real logic runs."""
+    calls: list[tuple[str, tuple[Any, ...]]] = []
+
+    from matrx_orm.query import executor as executor_mod
+
+    class _DBStub:
+        async def execute_write(self, sql: str, *params: Any) -> list[dict[str, Any]]:
+            calls.append((sql, params))
+            return [{"id": "row-1"}]
+
+    def _fake_init(self: Any, query: dict[str, Any]) -> None:
+        self.query = query
+        self.model = query.get("model")
+        self.db = _DBStub()
+
+    async def _no_dict(self: Any, _query: dict[str, Any]) -> Any:
+        return NotImplemented
+
+    def _guard(self: Any, _op: str) -> None:
+        return None
+
+    def _make(self: Any, **row: Any) -> Any:
+        return row
+
+    monkeypatch.setattr(executor_mod.QueryExecutor, "__init__", _fake_init)
+    monkeypatch.setattr(executor_mod.QueryExecutor, "_try_write_dict", _no_dict)
+    monkeypatch.setattr(executor_mod.QueryExecutor, "_guard_writable", _guard)
+    monkeypatch.setattr(executor_mod.QueryExecutor, "_make", _make)
+
+    from matrx_orm.query import builder as builder_mod
+
+    def _fake_build(self: Any) -> dict[str, Any]:
+        return {"table": self.model._meta.qualified_table_name, "model": self.model}
+
+    monkeypatch.setattr(builder_mod.QueryBuilder, "_build_query", _fake_build)
+
+    from matrx_orm import state as state_mod
+
+    async def _no_cache(*_a: Any, **_kw: Any) -> None:
+        return None
+
+    monkeypatch.setattr(state_mod.StateManager, "cache", _no_cache)
+
+    return calls
+
+
+@pytest.mark.asyncio
+async def test_upsert_does_not_reset_defaulted_columns_caller_omitted(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    """THE regression. A search re-discovering a known URL supplies only
+    identity + descriptive columns; the row's scrape state must survive."""
+    from matrx_orm.operations.create import upsert
+
+    await upsert(
+        _source_model(),
+        {
+            "topic_id": "t1",
+            "url": "https://example.com/a",
+            "title": "A",
+            "description": "desc",
+        },
+        ["topic_id", "url"],
+    )
+
+    sql, _ = captured_sql[0]
+    set_cols = _set_columns(sql)
+
+    leaked = set_cols & STATE_COLUMNS
+    assert not leaked, f"defaulted state columns leaked into DO UPDATE SET: {sorted(leaked)}"
+    assert set_cols == {"title", "description"}
+
+    # The INSERT half MUST still carry the defaults — a brand-new row needs them.
+    insert_cols = sql.split("(", 1)[1].split(")", 1)[0]
+    for column in STATE_COLUMNS:
+        assert column in insert_cols, f"{column} missing from the INSERT column list"
+
+
+@pytest.mark.asyncio
+async def test_upsert_still_updates_columns_the_caller_did_supply(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    """The fix must not over-correct: an explicitly supplied state column is a
+    deliberate write and must still land."""
+    from matrx_orm.operations.create import upsert
+
+    await upsert(
+        _source_model(),
+        {
+            "topic_id": "t1",
+            "url": "https://example.com/a",
+            "scrape_status": "success",
+            "server_attempts": 3,
+        },
+        ["topic_id", "url"],
+    )
+
+    set_cols = _set_columns(captured_sql[0][0])
+    assert set_cols == {"scrape_status", "server_attempts"}
+
+
+@pytest.mark.asyncio
+async def test_explicit_update_fields_still_wins(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    from matrx_orm.operations.create import upsert
+
+    await upsert(
+        _source_model(),
+        {"topic_id": "t1", "url": "https://example.com/a", "title": "A"},
+        ["topic_id", "url"],
+        update_fields=["title", "is_stale"],
+    )
+
+    assert _set_columns(captured_sql[0][0]) == {"title", "is_stale"}
+
+
+@pytest.mark.asyncio
+async def test_upsert_of_only_conflict_keys_reconciles_to_noop(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    """"Ensure this row exists" has exactly one meaning, so it reconciles to a
+    self-assignment rather than raising — and must NOT reset state columns.
+    A bare DO NOTHING would break the RETURNING * contract."""
+    from matrx_orm.operations.create import upsert
+
+    await upsert(
+        _source_model(),
+        {"topic_id": "t1", "url": "https://example.com/a"},
+        ["topic_id", "url"],
+    )
+
+    sql = captured_sql[0][0]
+    assert not _set_columns(sql) & STATE_COLUMNS
+    assert "RETURNING" in sql
+
+
+@pytest.mark.asyncio
+async def test_caller_supplied_none_does_not_write_the_column_default(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    """A None is dropped from the INSERT, so ``EXCLUDED.<col>`` would resolve to
+    the column DEFAULT — reintroducing the bug through the back door."""
+    from matrx_orm.operations.create import upsert
+
+    await upsert(
+        _source_model(),
+        {
+            "topic_id": "t1",
+            "url": "https://example.com/a",
+            "title": "A",
+            "description": None,
+        },
+        ["topic_id", "url"],
+    )
+
+    assert "description" not in _set_columns(captured_sql[0][0])
+
+
+@pytest.mark.asyncio
+async def test_bulk_upsert_uses_union_of_caller_supplied_keys(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    from matrx_orm.operations.create import bulk_upsert
+
+    await bulk_upsert(
+        _source_model(),
+        [
+            {"topic_id": "t1", "url": "https://example.com/a", "title": "A"},
+            {"topic_id": "t1", "url": "https://example.com/b", "description": "D"},
+        ],
+        ["topic_id", "url"],
+    )
+
+    set_cols = _set_columns(captured_sql[0][0])
+    assert not set_cols & STATE_COLUMNS
+    assert set_cols == {"title", "description"}
+
+
+@pytest.mark.asyncio
+async def test_expression_conflict_target_also_excludes_defaults(
+    captured_sql: list[tuple[str, tuple[Any, ...]]],
+) -> None:
+    """The ConflictTarget branch defaults to `update_fields or columns` — i.e.
+    ALL columns — so it leaked the same way."""
+    from matrx_orm.operations.create import upsert
+
+    await upsert(
+        _source_model(),
+        {"topic_id": "t1", "url": "https://example.com/a", "title": "A"},
+        ConflictTarget.partial(["topic_id", "url"], md5=("url",)),
+    )
+
+    assert not _set_columns(captured_sql[0][0]) & STATE_COLUMNS