mainsequence-metatable 0.1.28.dev50__tar.gz → 0.1.29.dev52__tar.gz

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/.agents/skills/client/metatables-meta-tables/SKILL.md

@@ -178,3 +178,7 @@ for restart recovery. On an uncertain outcome, inspect `upload_receipt()` before
 resubmitting; do not assign a fresh key to bypass uncertainty. Administrator
 reconciliation requires verified physical evidence. This workflow uses the Main
 Sequence API and has no Artifact or temporary-URL dependency.
+
+A 429 `transfer_capacity_exceeded` or 503 `database_connections_busy` means the
+API was at capacity and nothing ran. Retry after `Retry-After`, including SQL and
+writes; it is not an uncertain outcome.

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/.agents/skills/client/metatables-migrations/SKILL.md

@@ -240,6 +240,27 @@ Read `docs/operations/recovery-and-observability.md`.
 - Inspect partial DDL before retrying, especially on engines without transactional
   DDL. A finalization-only failure can be retried without reapplying committed
   revisions. Client DDL has no API executor journal.
+- Finalization sends at most 10 tables per request, the version table first, and
+  reports `Finalizing MetaTables 11-20 of 62.` for larger providers (client 0.1.28
+  and later). A run that stops partway leaves earlier batches active; rerun the
+  same upgrade to finalize the rest.
+- If the migration connection is refused with `provider_tables_misplaced`, provider
+  tables sit in the `metatables` catalog schema (MetaTables #50). Don't stamp, rerun or
+  hand-edit revisions. An administrator moves or drops those tables, then rerun.
+- If it is refused with `provider_tables_owner_unreachable`, the tables belong to a
+  retired role, named in `owners`. A database administrator runs
+  `REASSIGN OWNED BY <role> TO mt_owner;` in that database, then rerun.
+- If the client refuses because the connection "resolves unqualified names" outside the
+  default schema, it isn't the connection MetaTables issued. Use that connection, or set
+  `search_path` to the default schema.
+- If finalization reports `physical_table_missing`, a current table is absent and nothing
+  named it as removed. Its catalog binding is kept; check `other_schemas` and where your
+  revisions created it. Client 0.1.29 and later name tables removed from the models or
+  dropped by the run, such as on downgrade. Older clients fail this way after a removal or
+  downgrade, so upgrade the client.
+- `physical_table_inaccessible` means the table exists, but the MetaTables login can't
+  read all its columns or use its owner. Fix ownership or privileges; the stored contract
+  is kept.
 - Destructive catalog deletion is not migration recovery and never bypasses
   schema-management protection.
 

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/.agents/skills/client/metatables-time-index-table-updates/SKILL.md

@@ -124,3 +124,7 @@ for restart recovery. On an uncertain outcome, inspect `upload_receipt()` before
 resubmitting; do not assign a fresh key to bypass uncertainty. Administrator
 reconciliation requires verified physical evidence. This workflow uses the Main
 Sequence API and has no Artifact or temporary-URL dependency.
+
+A 429 `transfer_capacity_exceeded` or 503 `database_connections_busy` means the
+API was at capacity and nothing ran. Retry after `Retry-After`, an upload with the
+same operation UUID; it is not an uncertain outcome.

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.28.dev50
+Version: 0.1.29.dev52
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/docs/adr/api/0007-database-enforced-table-access.md

@@ -55,6 +55,16 @@
 > reuses login-role connections within a pod, only by the login role that opened them
 > and never through `SET ROLE`; no session state carries over from one use to the next.
 
+> Amendment (2026-10-09, [MetaTables #50](https://github.com/mainsequence-projects/MetaTables/issues/50)):
+> Application migrations use the shared `metatables` login ([ADR 0013](0013-application-owned-migrations.md)),
+> whose `"$user"` schema is the catalog. The invariant that they cannot accidentally create
+> tables in the catalog therefore rests on the migration connection, not on a separate login:
+>
+> - the connection is pinned to `search_path=public`, and the client verifies that path;
+> - the API refuses a provider whose tables are in `metatables`.
+>
+> This guards against accidents, not misuse: the login keeps its privileges.
+
 
 Date: 2026-09-29
 

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/docs/adr/api/0013-application-owned-migrations.md

@@ -1,5 +1,22 @@
 # ADR 0013: Application-owned migrations through the client
 
+> Amendment (2026-10-09, [MetaTables #50](https://github.com/mainsequence-projects/MetaTables/issues/50)
+> and [#51](https://github.com/mainsequence-projects/MetaTables/issues/51)): The configured login is
+> named `metatables`, so its default `"$user", public` search path starts in the catalog schema.
+>
+> - The migration connection carries `search_path=public`, the only default schema hosted
+>   PostgreSQL accepts. The client refuses a session whose effective path is anything else
+>   before it reads Alembic state. On PostgreSQL the version table is always qualified:
+>   `public` when the provider declares no schema.
+> - Before issuing the connection, the API refuses a provider whose tables are in the
+>   `metatables` schema or belong to a role the login can't use. Nothing is stamped or
+>   replayed.
+> - Lifecycle step 5 never infers removal from absence. Finalization deletes a catalog binding
+>   only for a table the client names as removed (dropped from its models, or dropped by this
+>   run, as on downgrade) and that is confirmed absent. Any other missing or unreadable table
+>   keeps its binding and fails. Older clients name nothing, so their removal and downgrade
+>   cleanup fails safe until they upgrade.
+
 Date: 2026-10-01
 
 Status: Accepted and implemented.

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/docs/api/authentication-and-errors.md

@@ -73,7 +73,7 @@ DataSource and bootstrap state and returns `Cache-Control: no-store`.
 | 404 | Resource missing or not visible to the current actor. |
 | 409 | Lifecycle, capability, protection, reference, or physical-state conflict. |
 | 422 | Request/schema validation failure. |
-| 503 | Catalog, SDK verification/source access, physical connection, or an uncertain operation outcome is unavailable. |
+| 503 | Catalog, SDK verification/source access, physical connection, or an uncertain operation outcome is unavailable; or every database connection the API may hold stayed busy (`database_connections_busy`, `Retry-After: 1`). |
 
 Most framework errors use a `detail` field. Some domain operations provide a
 structured code and field information. Raw-query validation errors can be returned
@@ -87,4 +87,4 @@ An unknown physical commit outcome requires reconciliation, not a blind retry.
 
 ## Transfer errors
 
-Data routes return 413 for request/row/response byte limits, 408 for the overall transfer deadline, and 429 with `Retry-After: 1` when per-process admission capacity is exhausted. Existing SQL/driver errors can retain their engine-specific deadline codes and status. Upload conflicts use 409 and uncertainty can use 503. A limit enforced by schema validation can return 422. See [typed client exceptions and recovery](../client/bounded-transfers.md).
+Data routes return 413 for request/row/response byte limits, 408 for the overall transfer deadline, and 429 `transfer_capacity_exceeded` with `Retry-After: 1` when no data-request slot freed up within the request deadline (each API process runs `serving.concurrency` data requests at once). A 503 `database_connections_busy` with `Retry-After: 1` means every database connection stayed busy until the deadline. In both cases no statement ran, so any request, including SQL and writes, is safe to retry. Existing SQL/driver errors can retain their engine-specific deadline codes and status. Upload conflicts use 409 and uncertainty can use 503. A limit enforced by schema validation can return 422. See [typed client exceptions and recovery](../client/bounded-transfers.md).

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/docs/api/openapi.json

@@ -679,6 +679,15 @@
               }
             ],
             "title": "Migration Provider Key"
+          },
+          "removed_meta_table_uids": {
+            "items": {
+              "format": "uuid",
+              "type": "string"
+            },
+            "maxItems": 200,
+            "title": "Removed Meta Table Uids",
+            "type": "array"
           }
         },
         "required": [

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/docs/client/bounded-transfers.md

@@ -132,6 +132,11 @@ statistics refresh (even though it uses GET), and receipt reconciliation do not
 automatically replay after uncertain failures. SDK authentication and one
 credential refresh remain; SDK and HTTP adapter write retry loops are bypassed.
 
+Capacity answers are the exception: 429 `transfer_capacity_exceeded` and 503
+`database_connections_busy` mean the API was at capacity and no statement ran.
+Retry any request after `Retry-After`, including SQL and writes; the client does
+so automatically only for safe reads.
+
 Exceptions are available in `metatables.transfer`: `TransferLimitError`,
 `TransferDeadlineError`, `TransferRetryExhausted`, `TransferConflictError`, and
 `TransferOutcomeUnknown`. Deterministic authorization/validation errors retain

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/docs/operations/hosted-runtime.md

@@ -120,6 +120,16 @@ lists what MetaTables requires.
 6. Open **Settings** and check the declaration, the resolved connection, the
    `ready` status and the applied migration revisions.
 
+The login and the catalog schema are both named `metatables`, so the login's default
+search path (`"$user", public`) starts in the catalog. Application migration connections
+are therefore pinned to `public`, and a provider with tables in `metatables` is refused
+with `provider_tables_misplaced`
+([MetaTables #50](https://github.com/mainsequence-projects/MetaTables/issues/50)). Tables
+created before 2026-09-30 may still belong to a retired `ms_ds_*_migration_owner` role. When
+a migration reports `provider_tables_owner_unreachable`, a database administrator runs
+`REASSIGN OWNED BY <role> TO mt_owner;` in that database. Granting the login that role
+doesn't fix it.
+
 ## Deployment gate
 
 In `.mainsequence/workflows/metatables-api.yaml`, the `migrate-system` Job

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mainsequence-metatable"
-version = "0.1.28.dev50"
+version = "0.1.29.dev52"
 description = "Main Sequence MetaTables Python client and standalone API"
 readme = "README.md"
 requires-python = ">=3.13"

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/mainsequence_metatable.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.28.dev50
+Version: 0.1.29.dev52
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/api/app/routes/managed_finalize.py

@@ -62,6 +62,9 @@ class FinalizeManagedRequest(BaseModel):
     migration_provider_key: str | None = Field(default=None, max_length=384)
     alembic_version_meta_table_uid: UUID | None = None
     alembic_revision: str | None = Field(default=None, max_length=255)
+    # Tables the client removed from its models or dropped in this run. Only
+    # these may lose their binding when absent; absence alone is never removal.
+    removed_meta_table_uids: list[UUID] = Field(default_factory=list, max_length=200)
     code_repository_context: dict[str, Any] | None = None
 
 
@@ -73,6 +76,18 @@ def physical_finalization_introspector() -> Callable[..., dict]:
     return introspect_physical_table
 
 
+def _missing_error(root: bool, other_schemas: list[str]) -> dict:
+    error = {
+        "code": "physical_table_missing",
+        "detail": "Alembic version table is missing." if root else (
+            "Physical table is missing and the request did not name it as removed."
+        ),
+    }
+    if other_schemas:
+        error["other_schemas"] = other_schemas
+    return error
+
+
 def _row(table: MetaTable | None, uid: UUID, *, previous: str = "unknown", exists: bool = False,
          finalized: bool = False, deleted: bool = False, error: dict | None = None,
          specialization: TimeIndexTable | None = None) -> dict:
@@ -113,6 +128,7 @@ def finalize_managed(
 ) -> dict:
     """Reconcile each requested row and return a per-table result envelope."""
     requested = list(dict.fromkeys(payload.meta_table_uids))
+    removed = set(payload.removed_meta_table_uids)
     lock_catalog(session)
     guards = {
         "migration_package": payload.migration_package,
@@ -162,16 +178,22 @@ def finalize_managed(
         except PhysicalIntrospectionError as exc:
             logger.warning("Physical table introspection failed during finalization", exc_info=True)
             if exc.code == "physical_table_not_found":
-                if uid == version_root:
-                    rows.append(_row(table, uid, previous=previous, specialization=specialization,
-                                     error={"code": "physical_table_missing", "detail": "Alembic version table is missing."}))
-                else:
+                # Absence is never evidence of removal (MetaTables #50): only a table
+                # the request names as removed loses its binding, never the root.
+                if uid != version_root and uid in removed:
                     rows.append(_row(table, uid, previous=previous, specialization=specialization,
                                      finalized=True, deleted=True))
                     session.delete(table)
+                else:
+                    rows.append(_row(table, uid, previous=previous, specialization=specialization,
+                                     error=_missing_error(uid == version_root, exc.other_schemas)))
                 continue
+            detail = (
+                "The MetaTables login can't read this table's full shape."
+                if exc.code == "physical_table_inaccessible" else "Physical introspection failed."
+            )
             rows.append(_row(table, uid, previous=previous, specialization=specialization, exists=True,
-                             error={"code": exc.code, "detail": "Physical introspection failed."}))
+                             error={"code": exc.code, "detail": detail}))
             continue
         except DataSourceAdmissionError as exc:
             session.rollback()
@@ -179,6 +201,12 @@ def finalize_managed(
         except (DataSourceIntegrationError, PhysicalConnectionUnavailable) as exc:
             session.rollback()
             raise HTTPException(status_code=503, detail="DataSource is unavailable.") from exc
+        if snapshot.get("owner_reachable") is False:
+            # Grant reconciliation must re-own platform-managed tables (ADR 0007).
+            rows.append(_row(table, uid, previous=previous, specialization=specialization, exists=True,
+                             error={"code": "physical_table_inaccessible",
+                                    "detail": "The table's owner is a role the MetaTables login can't use."}))
+            continue
         try:
             with session.begin_nested():
                 contract = contract_from_physical_snapshot(

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/api/app/routes/migration_connection.py

@@ -15,18 +15,63 @@ from metatables.api.app.auth.local_actor import catalog_session
 from metatables.api.app.routes.table_introspection import data_source_provider
 from metatables.api.app.routes.table_sql import authorized_table
 from metatables.api.backend.backends.migration_connection import migration_connection_description
+from metatables.api.backend.backends.operations import (
+    PhysicalConnectionUnavailable,
+    PhysicalIntrospectionError,
+    provider_physical_storage,
+)
 from metatables.api.backend.domain.data_source_rules import (
     DataSourceAdmissionError,
     require_data_source_access,
 )
 from metatables.api.backend.domain.permission_rules import TrustedActorFacts
 from metatables.api.backend.integrations.runtime_data_sources import DataSourceIntegrationError
+from metatables.api.backend.operations.bootstrap_migrations import PROGRESS_TABLE, VERSION_TABLE
 from metatables.api.backend.persistence.grants import GrantRepository
 from metatables.api.backend.persistence.locking import lock_catalog
-from metatables.api.backend.persistence.models import MetaTable
+from metatables.api.backend.persistence.models import Base, MetaTable
 
 router = APIRouter(prefix="/meta-tables", tags=["MetaTables"])
 
+CATALOG_TABLES = frozenset(Base.metadata.tables) | {PROGRESS_TABLE, VERSION_TABLE}
+
+
+def provider_storage_inspector():
+    return provider_physical_storage
+
+
+def provider_storage_problem(tables, storage) -> dict | None:
+    """Refuse providers whose tables a pinned connection would hide or can't use (#50)."""
+    misplaced = sorted(
+        f"metatables.{row['table_name']}"
+        for row in storage
+        if row["schema_name"] == "metatables" and row["table_name"] not in CATALOG_TABLES
+    )
+    if misplaced:
+        # A pinned connection would no longer see these and replay the provider's
+        # history next to them; nothing is stamped, replayed or chosen for it.
+        return {
+            "code": "provider_tables_misplaced",
+            "detail": "These provider tables are in the metatables catalog schema. An "
+            "administrator must move or drop them before the provider migrates.",
+            "tables": misplaced,
+        }
+    registered = {(item.physical_schema or "public", item.physical_table_name) for item in tables}
+    unreachable = [
+        row for row in storage
+        if (row["schema_name"], row["table_name"]) in registered and not row["owner_reachable"]
+    ]
+    if unreachable:
+        return {
+            "code": "provider_tables_owner_unreachable",
+            "detail": "These provider tables belong to a role the MetaTables login can't use. "
+            "An administrator must move their ownership to mt_owner (REASSIGN OWNED) "
+            "before the provider migrates.",
+            "tables": [f"{row['schema_name']}.{row['table_name']}" for row in unreachable],
+            "owners": sorted({row["owner"] for row in unreachable}),
+        }
+    return None
+
 
 class MigrationRequest(BaseModel):
     model_config = ConfigDict(extra="forbid")
@@ -47,6 +92,7 @@ def migration_connection(
     actor: TrustedActorFacts = Depends(trusted_actor_facts),
     session: Session = Depends(catalog_session),
     sources=Depends(data_source_provider),
+    inspect_storage=Depends(provider_storage_inspector),
 ):
     lock_catalog(session)
     table = authorized_table(session, actor, table_uid, edit=True, reserved_policy="reconcile")
@@ -78,11 +124,17 @@ def migration_connection(
     try:
         source = sources.get(table.data_source_uid)
         require_data_source_access(source, access="write")
+        storage = inspect_storage(
+            source=source,
+            table_names=sorted({item.physical_table_name for item in tables if item.physical_table_name}),
+        )
         result = migration_connection_description(source)
     except DataSourceAdmissionError as exc:
         raise HTTPException(exc.status_code, exc.code) from exc
-    except DataSourceIntegrationError as exc:
+    except (DataSourceIntegrationError, PhysicalConnectionUnavailable, PhysicalIntrospectionError) as exc:
         raise HTTPException(503, "DataSource is unavailable.") from exc
+    if (problem := provider_storage_problem(tables, storage)) is not None:
+        raise HTTPException(409, problem)
     # Include removed provider models so finalization reconciles dropped tables.
     result["meta_table_uids"] = [str(item.uid) for item in tables if item.uid != root_uid]
     control = getattr(request.app.state, "runtime_control", None)

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/api/backend/backends/adapters/postgresql_introspection.py

@@ -14,6 +14,33 @@ from metatables.api.backend.backends.adapters.sqlite_connection import configure
 from metatables.api.backend.backends.contracts import PhysicalIntrospectionError
 from metatables.api.backend.integrations.runtime_data_sources import RuntimeDataSource
 
+# pg_class and pg_attribute are readable whatever the login's privileges, unlike
+# information_schema, which hides what the login can't access (MetaTables #51).
+# The relation kinds match those information_schema.columns describes.
+_RELATION_SQL = """
+    SELECT (SELECT count(*) FROM pg_attribute a
+             WHERE a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped) AS column_count,
+           pg_has_role(current_user, c.relowner, 'MEMBER') AS owner_reachable
+      FROM pg_class c
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+     WHERE n.nspname = %s AND c.relname = %s AND c.relkind IN ('r', 'p', 'v', 'f')
+"""
+_OTHER_SCHEMAS_SQL = """
+    SELECT n.nspname
+      FROM pg_class c
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+     WHERE c.relname = %s AND n.nspname <> %s AND c.relkind IN ('r', 'p', 'v', 'f')
+     ORDER BY n.nspname
+"""
+_PROVIDER_STORAGE_SQL = """
+    SELECT n.nspname AS schema_name, c.relname AS table_name,
+           pg_get_userbyid(c.relowner) AS owner,
+           pg_has_role(current_user, c.relowner, 'MEMBER') AS owner_reachable
+      FROM pg_class c
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+     WHERE c.relkind IN ('r', 'p') AND c.relname = ANY(%s)
+     ORDER BY n.nspname, c.relname
+"""
 _COLUMNS_SQL = """
     SELECT column_name AS name, data_type, udt_name AS backend_type,
            is_nullable = 'YES' AS nullable, ordinal_position
@@ -105,10 +132,19 @@ def introspect_postgresql_table(
             with connection.cursor(cursor_factory=RealDictCursor) as cursor:
                 configure_cursor(cursor, readonly=True, statement_timeout_ms=15000)
                 arguments = (schema, physical_table_name)
+                cursor.execute(_RELATION_SQL, arguments)
+                relation = cursor.fetchone()
+                if relation is None:
+                    cursor.execute(_OTHER_SCHEMAS_SQL, (physical_table_name, schema))
+                    raise PhysicalIntrospectionError(
+                        "physical_table_not_found",
+                        other_schemas=[row["nspname"] for row in cursor.fetchall()],
+                    )
                 cursor.execute(_COLUMNS_SQL, arguments)
                 columns = [dict(row) for row in cursor.fetchall()]
-                if not columns:
-                    raise PhysicalIntrospectionError("physical_table_not_found")
+                # Fewer visible columns than the table has: never a contract (#51).
+                if len(columns) < relation["column_count"]:
+                    raise PhysicalIntrospectionError("physical_table_inaccessible")
                 cursor.execute(_INDEXES_SQL, arguments)
                 indexes = [dict(row) for row in cursor.fetchall()]
                 cursor.execute(_CONSTRAINTS_SQL, arguments)
@@ -127,4 +163,27 @@ def introspect_postgresql_table(
         "columns": columns,
         "indexes": indexes,
         "constraints": constraints,
+        "owner_reachable": relation["owner_reachable"],
     }
+
+
+def postgresql_provider_storage(
+    *, source: RuntimeDataSource, table_names: list[str]
+) -> list[dict[str, Any]]:
+    """Every schema in which the provider's table names exist, with their owners."""
+    with open_postgresql_connection(
+        source, access="read", operation="migration_preflight"
+    ) as connection:
+        try:
+            with connection.cursor(cursor_factory=RealDictCursor) as cursor:
+                configure_cursor(cursor, readonly=True, statement_timeout_ms=15000)
+                cursor.execute(_PROVIDER_STORAGE_SQL, (list(table_names),))
+                rows = [dict(row) for row in cursor.fetchall()]
+            connection.rollback()
+        except DATABASE_ERRORS as exc:
+            try:
+                connection.rollback()
+            except DATABASE_ERRORS:
+                pass
+            raise PhysicalIntrospectionError("physical_table_introspection_failed") from exc
+    return rows

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/api/backend/backends/contracts.py

@@ -288,8 +288,10 @@ class TimeIndexQueryError(RuntimeError):
 class PhysicalIntrospectionError(RuntimeError):
     """A secret-safe physical introspection failure."""
 
-    def __init__(self, code: str) -> None:
+    def __init__(self, code: str, *, other_schemas: Sequence[str] = ()) -> None:
         self.code = code
+        # Where a same-named relation exists instead; a hint, not proof (#50).
+        self.other_schemas = list(other_schemas)
         super().__init__(code)
 
 

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/api/backend/backends/migration_connection.py

@@ -4,9 +4,29 @@ DDL authority belongs to the configured database login. This does not create a
 role or claim that a direct database connection enforces catalog table grants.
 """
 
+import re
+
 from sqlalchemy.engine import URL
 
 from metatables.api.backend.backends.registry import engine_descriptor
+from metatables.api.backend.domain.data_source_rules import DataSourceAdmissionError
+
+_PLAIN_SCHEMA = re.compile(r"[a-z_][a-z0-9_]*")
+
+
+def search_path_option(schema):
+    """Pin unqualified application DDL to the default schema (MetaTables #50).
+
+    The login is named `metatables`, so PostgreSQL's default `"$user", public`
+    path would otherwise put unqualified tables in the catalog schema.
+    """
+    if not _PLAIN_SCHEMA.fullmatch(schema or "") or schema == "metatables":
+        raise DataSourceAdmissionError(
+            "migration_schema_invalid",
+            "Application migrations need a plain default schema other than metatables.",
+            status_code=409,
+        )
+    return f"-c search_path={schema}"
 
 
 def migration_connection_description(source):
@@ -23,7 +43,11 @@ def migration_connection_description(source):
             "mssql": "mssql+pyodbc",
         }[dialect]
         if dialect == "postgresql":
-            query = {"sslmode": settings.ssl_mode, "connect_timeout": "10"}
+            query = {
+                "sslmode": settings.ssl_mode,
+                "connect_timeout": "10",
+                "options": search_path_option(settings.default_schema),
+            }
         elif dialect == "mysql":
             query = {"charset": settings.default_charset or "utf8mb4"}
         else:

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/api/backend/backends/operations.py

@@ -227,6 +227,21 @@ def introspect_physical_table(
     )
 
 
+def provider_physical_storage(*, source: RuntimeDataSource, table_names: Sequence[str]):
+    """Where a migration provider's tables exist and who owns them (MetaTables #50).
+
+    Only PostgreSQL and TimescaleDB keep the catalog in a schema that unqualified
+    application DDL can reach; other engines report nothing.
+    """
+    if source.class_type not in {"postgresql", "timescale_db"} or not table_names:
+        return []
+    from metatables.api.backend.backends.adapters.postgresql_introspection import (
+        postgresql_provider_storage,
+    )
+
+    return postgresql_provider_storage(source=source, table_names=list(table_names))
+
+
 def drop_table(*, source: RuntimeDataSource, physical_schema: str | None, physical_table_name: str):
     return backend_for(source).tables.drop(
         physical_schema=physical_schema, physical_table_name=physical_table_name

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/migrations/__init__.py

@@ -72,6 +72,8 @@ class PreparedAlembicMetaTableMigration:
     meta_table_uids: list[str]
     reserved_tables: list[MetaTable] = field(default_factory=list)
     owner_role_name: str | None = None
+    # Tables whose absence finalization may accept: removed models and tables this run dropped.
+    removed_meta_table_uids: list[str] = field(default_factory=list)
 
 
 @dataclass(frozen=True, slots=True)
@@ -619,6 +621,7 @@ class AlembicMetaTableMigration:
         finalize_meta_table_uids = list(
             dict.fromkeys([str(registry_uid), *prepared.meta_table_uids])
         )
+        removed = set(prepared.removed_meta_table_uids)
         response = _finalize_in_batches(
             finalize_meta_table_uids,
             lambda batch: MetaTable.finalize_managed(
@@ -629,6 +632,7 @@ class AlembicMetaTableMigration:
                     migration_provider_key=self.migration_provider_key,
                     alembic_version_meta_table_uid=str(registry_uid),
                     alembic_revision=alembic_revision,
+                    removed_meta_table_uids=[uid for uid in batch if uid in removed] or None,
                 ),
                 timeout=timeout,
                 on_status=on_metatable_finalize_status,

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/migrations/connection.py

@@ -9,6 +9,8 @@ from sqlalchemy.pool import NullPool
 from . import sqlite as _sqlite  # noqa: F401  Registers SQLite DDL and batch rules.
 from .tls import migration_tls_arguments
 
+CATALOG_SCHEMA = "metatables"
+
 
 def version_schema(config, dialect):
     schema = config.attributes["mainsequence_migration_provider"].version_table_schema
@@ -16,9 +18,41 @@ def version_schema(config, dialect):
         return None
     if dialect == "mssql" and schema in (None, "public"):
         return config.attributes.get("metatables_default_schema", "dbo")
+    if dialect == "postgresql" and schema is None:
+        # Never left to search_path: the registry is the table the catalog
+        # registers, in the default schema (MetaTables #50).
+        return config.attributes.get("metatables_default_schema") or "public"
     return schema
 
 
+def require_default_schema_session(connection, migration, config):
+    """Refuse a PostgreSQL session whose unqualified names could leave the default schema.
+
+    MetaTables issues migration connections pinned to the default schema; the
+    `metatables` login's own `"$user"` schema is the catalog (MetaTables #50).
+    """
+    if connection.dialect.name != "postgresql":
+        return
+    declared = {table.schema for table in migration.target_metadata.tables.values()}
+    if CATALOG_SCHEMA in declared | {migration.version_table_schema}:
+        raise RuntimeError(
+            f"Application tables cannot use the reserved {CATALOG_SCHEMA} schema."
+        )
+    expected = config.attributes.get("metatables_default_schema") or "public"
+    began = not connection.in_transaction()
+    path = [row[0] for row in connection.exec_driver_sql(
+        "SELECT unnest(current_schemas(false))::text"
+    )]
+    if began:
+        connection.rollback()
+    if path != [expected]:
+        raise RuntimeError(
+            f"The migration connection resolves unqualified names in {path or 'no schema'}; "
+            f"MetaTables requires exactly [{expected!r}]. Use the connection MetaTables "
+            "issues for this provider or set search_path to the default schema."
+        )
+
+
 @contextmanager
 def migration_engine(config):
     url = make_url(config.get_main_option("sqlalchemy.url"))

{mainsequence_metatable-0.1.28.dev50 → mainsequence_metatable-0.1.29.dev52}/src/metatables/migrations/env.py

@@ -14,7 +14,7 @@ def run_mainsequence_alembic_env(
     from alembic import context
 
     from metatables.migrations import apply_mainsequence_migration_role
-    from metatables.migrations.connection import migration_engine
+    from metatables.migrations.connection import migration_engine, require_default_schema_session
     from metatables.migrations.sqlite import AUTO_BATCH_OPTION, check_foreign_keys
 
     def migration_provider() -> Any:
@@ -105,6 +105,7 @@ def run_mainsequence_alembic_env(
         connection = config.attributes.get("connection")
         if connection is not None:
             apply_mainsequence_migration_role(connection, config)
+            require_default_schema_session(connection, migration_provider(), config)
             context.configure(connection=connection, **configure_kwargs(connection))
             run(connection)
             return
@@ -112,6 +113,7 @@ def run_mainsequence_alembic_env(
         config.attributes.setdefault("mainsequence_migration_provider", migration_provider())
         with migration_engine(config) as engine, engine.connect() as connection:
             apply_mainsequence_migration_role(connection, config)
+            require_default_schema_session(connection, migration_provider(), config)
             context.configure(connection=connection, **configure_kwargs(connection))
             run(connection)