mainsequence-metatable 0.1.28.dev48__tar.gz → 0.1.28.dev50__tar.gz

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.28.dev48
+Version: 0.1.28.dev50
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/docs/adr/api/0007-database-enforced-table-access.md

@@ -32,6 +32,29 @@
 > Reconciliation still records each table's cascade targets, but only so the
 > [impact pre-flight](0016-impact-preflight.md) can show their effects.
 
+> Amendment (2026-10-09): On PostgreSQL and TimescaleDB, a catalog change applies
+> only what it changed. Reconciliation used to rebuild every role, re-inspect every
+> table and re-secure every routine on each change, all under the catalog lock, so
+> one registration could hold the lock for minutes and stop every other request.
+> Now a change:
+>
+> - re-inspects the policies of the tables it changed, plus the tables whose
+>   recorded cascade targets they affect;
+> - rebuilds only roles that are new or whose definition changed, and grants or
+>   revokes only the memberships and privileges that differ from the applied
+>   manifest;
+> - re-secures routines only when the owned schemas, routines or `PUBLIC` grants
+>   changed since the last sweep.
+>
+> The full sweep still runs when the runtime DataSource changes, at initialization
+> (the migration Job of every deployment) and from the admin repair action. Those
+> are when grants changed outside MetaTables are repaired, not on every catalog
+> change. MySQL and SQL Server keep the full sweep on every change.
+
+> Amendment (2026-10-09): [ADR 0022](0022-concurrent-requests-and-connection-reuse.md)
+> reuses login-role connections within a pod, only by the login role that opened them
+> and never through `SET ROLE`; no session state carries over from one use to the next.
+
 
 Date: 2026-09-29
 

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/docs/adr/api/0022-concurrent-requests-and-connection-reuse.md

@@ -1,9 +1,10 @@
 # ADR 0022: Concurrent requests per pod and reused database connections
 
-Status: Proposed.
+Status: Accepted.
 
-Implementation status: implemented on development; not deployed. Unit tests pass;
-the container suite and the load test against the development deployment are pending.
+Implementation status: implemented on development. Unit tests pass; deployment
+verification, the MySQL and SQL Server container suites and the load test against the
+development deployment are pending.
 
 Owner: this repository.
 

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/docs/adr/index.md

@@ -39,7 +39,7 @@ decisions belong in the SDK repository and are linked as dependencies.
 | [0016: Impact pre-flight for cascades and deletes](api/0016-impact-preflight.md) | Accepted; implemented; container engine verification pending | `get_impact` returns what a row delete, key update or table drop reaches, every blocker and whether the caller may write each table, from the planner the confirmed delete runs; cascade effects never block. Projects render it in their own UIs. |
 | [0017: Hosted caller identity through the SDK request identity](api/0017-hosted-request-identity.md) | Accepted; implemented for 0.1.21; hosted rollout verification pending | The hosted API installs the SDK request identity that the platform launcher requires; the SDK verifies each caller assertion once and admission reads `User.get_logged_user()`, including the caller's signed Teams and admin flag. Local keeps its loopback token admission. |
 | [0018: Table search by meaning and columns](api/0018-table-search.md) | Accepted; implemented; container engine verification pending | One route ranks visible tables by BM25 keywords and a local English embedding model over a per-table document that includes columns, merges the rankings and returns `matched_columns`; the index refreshes on every metadata write. |
-| [0022: Concurrent requests per pod and reused database connections](api/0022-concurrent-requests-and-connection-reuse.md) | Proposed; implemented on development; not deployed; container and load verification pending | One number per pod sizes its threads, data-request admission, catalog pool and login-role connections; connections are reused only by the role that opened them, and maximum pods × 3T stays within the database's connection limit. |
+| [0022: Concurrent requests per pod and reused database connections](api/0022-concurrent-requests-and-connection-reuse.md) | Accepted; implemented on development; deployment and load verification pending | One number per pod sizes its threads, data-request admission, catalog pool and login-role connections; connections are reused only by the role that opened them, and maximum pods × 3T stays within the database's connection limit. |
 
 ## Agents
 

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mainsequence-metatable"
-version = "0.1.28.dev48"
+version = "0.1.28.dev50"
 description = "Main Sequence MetaTables Python client and standalone API"
 readme = "README.md"
 requires-python = ">=3.13"

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/src/mainsequence_metatable.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.28.dev48
+Version: 0.1.28.dev50
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/src/mainsequence_metatable.egg-info/SOURCES.txt

@@ -572,6 +572,7 @@ tests/packaging/test_editable_build.py
 tests/packaging/test_installed_distribution.py
 tests/permissions/test_access_map.py
 tests/permissions/test_governed_sql_scope.py
+tests/permissions/test_incremental_security.py
 tests/permissions/test_local_actor_membership.py
 tests/permissions/test_migration_boundary.py
 tests/permissions/test_namespace_grants.py

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/src/metatables/api/backend/backends/access.py

@@ -33,9 +33,73 @@ class ExecutionIdentity:
     writable: frozenset[tuple[str, str]]
 
 
+@dataclass(frozen=True)
+class ManifestChanges:
+    """The statements that move the database from one applied manifest to the next.
+
+    ``reset`` roles are new or changed: each is rebuilt completely, so every one of
+    its memberships and privileges is granted again. ``retired`` roles are rebuilt
+    with nothing. Other roles change only by the memberships and privileges that
+    differ.
+    """
+
+    reset: frozenset
+    retired: frozenset
+    revoke_memberships: tuple
+    revoke_privileges: tuple
+    revoke_schemas: tuple
+    grant_memberships: tuple
+    grant_privileges: tuple
+
+
+def manifest_changes(previous, desired):
+    """Diff two manifests; None when ``previous`` is not a complete applied manifest."""
+    if not (
+        isinstance(previous, dict)
+        and isinstance(previous.get("roles"), dict)
+        and isinstance(previous.get("memberships"), list)
+        and isinstance(previous.get("privileges"), list)
+    ):
+        return None
+    before, roles = previous["roles"], desired["roles"]
+    reset = frozenset(name for name, definition in roles.items() if before.get(name) != definition)
+    retired = frozenset(set(before) - set(roles))
+    rebuilt = reset | retired
+    old_memberships = {tuple(row) for row in previous["memberships"]}
+    new_memberships = {tuple(row) for row in desired["memberships"]}
+    old_privileges = {(n, s, t, bool(w)) for n, s, t, w in previous["privileges"]}
+    new_privileges = {(n, s, t, bool(w)) for n, s, t, w in desired["privileges"]}
+    revoke_privileges = tuple(sorted(
+        row for row in old_privileges - new_privileges if row[0] not in rebuilt
+    ))
+    kept_schemas = {(name, schema) for name, schema, _, _ in new_privileges}
+    return ManifestChanges(
+        reset=reset,
+        retired=retired,
+        # Rebuilding a role already revokes every membership it holds.
+        revoke_memberships=tuple(sorted(
+            row for row in old_memberships - new_memberships if row[0] not in rebuilt
+        )),
+        revoke_privileges=revoke_privileges,
+        revoke_schemas=tuple(sorted(
+            {(name, schema) for name, schema, _, _ in revoke_privileges} - kept_schemas
+        )),
+        grant_memberships=tuple(sorted(
+            row for row in new_memberships if row[0] in reset or row not in old_memberships
+        )),
+        grant_privileges=tuple(sorted(
+            row for row in new_privileges if row[0] in reset or row not in old_privileges
+        )),
+    )
+
+
 class DatabaseAccess:
     """Backend facet for setup, mirroring and restricted session admission."""
 
+    # A catalog change applies only what it changed; the full sweep stays for
+    # initialization and repair (ADR 0007, amended 2026-10-09).
+    incremental = False
+
     def __init__(self, backend):
         self.backend = backend
         self.source = backend.source
@@ -87,16 +151,20 @@ class DatabaseAccess:
         deleted or their keys updated, as (schema, table, on_delete, on_update)."""
         raise NotImplementedError
 
-    def refresh_policies(self, session):
-        tables = list(
-            session.scalars(
-                select(MetaTable)
-                .where(MetaTable.data_source_uid == self.source.uid)
-                .execution_options(populate_existing=True)
-            )
+    def refresh_policies(self, session, tables=None):
+        """Inspect every table's policy, or only those of the table UIDs in ``tables``."""
+        query = (
+            select(MetaTable)
+            .where(MetaTable.data_source_uid == self.source.uid)
+            .execution_options(populate_existing=True)
         )
+        policies = select(SQLTablePolicy)
+        if tables is not None:
+            query = query.where(MetaTable.uid.in_(tables))
+            policies = policies.where(SQLTablePolicy.table_uid.in_(tables))
+        tables = list(session.scalars(query))
         current = {t.uid for t in tables}
-        for old in session.scalars(select(SQLTablePolicy)):
+        for old in session.scalars(policies):
             if old.table_uid not in current:
                 session.delete(old)
         for table in tables:
@@ -129,6 +197,59 @@ class DatabaseAccess:
             ]
         session.flush()
 
+    def policy_scope(self, session, changed):
+        """The tables whose policies a change to ``changed`` can affect.
+
+        A policy records its table's cascade targets, the tables whose foreign keys
+        act on its rows, so the scope adds the tables the changed ones reference and
+        the tables whose recorded targets name them. Returns None, meaning every
+        table, when a reference cannot be read.
+        """
+        import sqlalchemy as sa
+
+        changed = set(changed)
+        tables = {
+            t.uid: t
+            for t in session.scalars(
+                select(MetaTable).where(MetaTable.data_source_uid == self.source.uid)
+            )
+        }
+
+        def key(schema, name):
+            return self.backend.catalog.registration_schema(self.source, schema), name
+
+        by_name = {key(t.physical_schema, t.physical_table_name): uid for uid, t in tables.items()}
+        names = {key(t.physical_schema, t.physical_table_name) for uid, t in tables.items() if uid in changed}
+        names |= {
+            (policy.schema_name, policy.table_name)
+            for policy in session.scalars(
+                select(SQLTablePolicy).where(SQLTablePolicy.table_uid.in_(changed))
+            )
+        }
+        scope = set(changed)
+        inspector = sa.inspect(session.connection())
+        for uid in sorted(changed & set(tables), key=str):
+            table = tables[uid]
+            if table.provisioning_status != "active":
+                continue
+            try:
+                references = inspector.get_foreign_keys(
+                    table.physical_table_name, schema=table.physical_schema
+                )
+            except Exception:
+                return None
+            for reference in references:
+                schema = reference.get("referred_schema") or table.physical_schema
+                scope.add(by_name.get(key(schema, reference["referred_table"])))
+        for policy in session.scalars(select(SQLTablePolicy)):
+            if any(
+                key(target.get("schema"), target.get("table")) in names
+                for target in policy.cascade_targets or ()
+            ):
+                scope.add(policy.table_uid)
+        scope.discard(None)
+        return scope
+
     def plan(self, session):
         """Read only catalog facts. No submitted statement or physical inspection."""
         roles, memberships, privileges = {}, set(), set()
@@ -190,18 +311,33 @@ class DatabaseAccess:
                     memberships.add((login, role("team", team, write)))
         return dict(roles=roles, memberships=sorted(memberships), privileges=sorted(privileges))
 
-    def reconcile(self, session, *, inspect_objects=False):
+    def reconcile(self, session, *, inspect_objects=False, tables=None):
+        """Mirror the catalog into database grants.
+
+        ``inspect_objects`` is the full sweep: every policy is inspected and every
+        role rebuilt, which also repairs grants changed outside MetaTables.
+        Otherwise only the policies ``tables`` can affect are inspected, and an
+        incremental backend applies only the grants that changed.
+        """
         state = session.get(SQLSecurityState, 1)
         if state is None or not state.ready:
             raise SQLExecutionError(code="sql_security_not_initialized")
         if inspect_objects:
             self.refresh_policies(session)
+        elif tables:
+            self.refresh_policies(session, tables=self.policy_scope(session, tables))
         desired = self.plan(session)
-        self.apply(session, desired, state.manifest)
+        self.apply(session, desired, state.manifest, full=inspect_objects)
         state.manifest, state.pending = desired, False
         session.flush()
 
-    def apply(self, session, desired, previous):
+    def reconcile_change(self, session, *, tables, full=False):
+        """Reconcile one catalog unit of work just before it commits."""
+        if full or not self.incremental:
+            return self.reconcile(session, inspect_objects=True)
+        return self.reconcile(session, tables=tables)
+
+    def apply(self, session, desired, previous, *, full=True):
         raise NotImplementedError
 
     def admit(self, session, actor, *, write, reconcile=True):
@@ -318,6 +454,13 @@ def _changed(session, *_):
 
     with session.no_autoflush:
         for row in session.new | session.dirty | session.deleted:
+            if isinstance(row, MetaTable) and row.data_source_uid == source_uid:
+                # Rows, not UIDs: a new row has its UID only once it is flushed.
+                session.info.setdefault("sql_security_tables", []).append(row)
+            elif isinstance(row, DataSource) and (row.is_runtime or row.is_default):
+                session.info["sql_security_full"] = True
+            elif session.info.get("sql_security_changed"):
+                continue
             relevant = (
                 isinstance(row, MetaTable)
                 and row.data_source_uid == source_uid
@@ -347,7 +490,6 @@ def _changed(session, *_):
                     )
             if relevant:
                 session.info["sql_security_changed"] = True
-                break
 
 
 def _before_commit(session):
@@ -361,14 +503,15 @@ def _before_commit(session):
     ):
         return
     session.flush()
+    tables = {row.uid for row in session.info.pop("sql_security_tables", ())}
+    full = session.info.pop("sql_security_full", False)
     if not session.info.pop("sql_security_changed", False):
         return
     session.info["sql_security_syncing"] = True
     try:
         backend = resolver()
         backend.catalog.lock(session)
-        access = backend.access
-        access.reconcile(session, inspect_objects=True)
+        backend.access.reconcile_change(session, tables=tables, full=full)
     finally:
         session.info.pop("sql_security_syncing", None)
 
@@ -391,6 +534,8 @@ def _after_transaction_end(session, transaction):
     committed = session.info.pop("sql_commit_finished", False)
     access = session.info.pop("sql_deferred_access", None)
     session.info.pop("sql_security_changed", None)
+    session.info.pop("sql_security_tables", None)
+    session.info.pop("sql_security_full", None)
     session.info.pop("sql_admission_uncommitted", None)
     if committed and access is not None:
         try:

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/src/metatables/api/backend/backends/hosted_access.py

@@ -164,7 +164,7 @@ class MySQLAccess(HostedAccess):
                 state.pending = False
                 session.commit()
 
-    def apply(self, session, desired, previous):
+    def apply(self, session, desired, previous, *, full=True):
         q = mysql_identifier
         with ExitStack() as resources:
             engine = self.backend.catalog.open_engine(self.source, resources)
@@ -288,7 +288,7 @@ class MSSQLAccess(HostedAccess):
         )
         return {(schema, name, referential_action(delete), referential_action(update)) for schema, name, delete, update in rows}
 
-    def apply(self, session, desired, previous):
+    def apply(self, session, desired, previous, *, full=True):
         connection = session.connection()
         connection.engine.hide_parameters = True
         q = ms_identifier

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/src/metatables/api/backend/backends/postgresql_access.py

@@ -5,7 +5,7 @@ from contextlib import contextmanager
 
 from sqlalchemy import text
 
-from metatables.api.backend.backends.access import DatabaseAccess
+from metatables.api.backend.backends.access import DatabaseAccess, manifest_changes
 from metatables.api.backend.backends.contracts import SQLExecutionError
 from metatables.api.backend.persistence.models import SQLSecurityState
 
@@ -24,6 +24,8 @@ USER_SCHEMA = "{schema} !~ '^pg_' AND {schema} <> 'information_schema'"
 
 
 class PostgreSQLAccess(DatabaseAccess):
+    incremental = True
+
     def prerequisite_problems(self, connection):
         schema = getattr(self.source.connection, "default_schema", None) or "public"
         login, database, creates_roles, creates_schemas, catalog_owner, tables = connection.execute(text(
@@ -229,82 +231,84 @@ class PostgreSQLAccess(DatabaseAccess):
                     targets.add((*hypertable, delete, update))
         return targets
 
-    def reconcile(self, session, *, inspect_objects=False):
-        if inspect_objects:
-            state = session.get(SQLSecurityState, 1)
-            roles = set(self.plan(session)["roles"]) | set(
-                (state.manifest if state else {}).get("roles", {})
-            )
-            self.secure_routines(session.connection(), roles)
-        return super().reconcile(session, inspect_objects=inspect_objects)
+    def object_fingerprint(self, connection):
+        """What the routine sweep secures: owned schemas and routines, and PUBLIC grants.
+
+        A change to any of them, such as a migration that adds a routine or grants
+        to PUBLIC, makes the next reconciliation sweep them again.
+        """
+        schemas = USER_SCHEMA.format(schema="n.nspname")
+        return connection.execute(text(
+            "SELECT md5(concat_ws('|', "
+            "(SELECT string_agg(n.oid::text, ',' ORDER BY n.oid) FROM pg_namespace n WHERE "
+            + schemas + " AND " + OWNED.format(owner="n.nspowner") + "), "
+            "(SELECT string_agg(p.oid::text || ':' || p.prosecdef::text, ',' ORDER BY p.oid) "
+            "FROM pg_proc p JOIN pg_namespace n ON n.oid=p.pronamespace WHERE "
+            + schemas + " AND " + OWNED.format(owner="p.proowner") + "), "
+            "(SELECT count(*)::text FROM pg_class c JOIN pg_namespace n ON n.oid=c.relnamespace "
+            "CROSS JOIN LATERAL aclexplode(c.relacl) acl WHERE acl.grantee=0 AND "
+            + schemas + " AND " + OWNED.format(owner="c.relowner") + "), "
+            "(SELECT count(*)::text FROM pg_attribute a JOIN pg_class c ON c.oid=a.attrelid "
+            "JOIN pg_namespace n ON n.oid=c.relnamespace CROSS JOIN LATERAL aclexplode(a.attacl) acl "
+            "WHERE acl.grantee=0 AND n.nspname !~ '^pg_' AND " + OWNED.format(owner="c.relowner") + ")"
+            "))"
+        )).scalar()
+
+    def reconcile(self, session, *, inspect_objects=False, tables=None):
+        state = session.get(SQLSecurityState, 1)
+        manifest = (state.manifest if state else None) or {}
+        connection = session.connection()
+        objects = self.object_fingerprint(connection)
+        if inspect_objects or manifest.get("objects") != objects:
+            roles = set(self.plan(session)["roles"]) | set(manifest.get("roles", {}))
+            self.secure_routines(connection, roles)
+            objects = self.object_fingerprint(connection)
+        super().reconcile(session, inspect_objects=inspect_objects, tables=tables)
+        state = session.get(SQLSecurityState, 1)
+        state.manifest = {**state.manifest, "objects": objects}
+        session.flush()
 
-    def apply(self, session, desired, previous):
+    def apply(self, session, desired, previous, *, full=True):
         connection = session.connection()
         connection.engine.hide_parameters = True
-        roles = desired["roles"]
         invoker = self.role("invoker", "routines")
-        self._role(connection, invoker)
         database = connection.exec_driver_sql("SELECT current_database()").scalar()
+        changes = None if full else manifest_changes(previous, desired)
+        if changes is None:
+            return self._apply_all(connection, desired, previous, database=database, invoker=invoker)
+        roles = desired["roles"]
+        for name in sorted(changes.reset):
+            self._role(connection, name, self.password(name) if roles[name]["login"] else None)
+        for name in sorted(changes.reset | changes.retired):
+            login = name in roles and bool(roles[name]["login"])
+            self._reset_role(connection, name, login=login, database=database, invoker=invoker)
+        for member, parent in changes.revoke_memberships:
+            connection.exec_driver_sql(f"REVOKE {quote(parent)} FROM {quote(member)}")
+        for name, schema, table, _ in changes.revoke_privileges:
+            relation = f"{quote(schema)}.{quote(table)}"
+            # A dropped table took its grants with it.
+            if connection.execute(text("SELECT to_regclass(:relation) IS NOT NULL"), {"relation": relation}).scalar():
+                connection.exec_driver_sql(f"REVOKE ALL ON TABLE {relation} FROM {quote(name)}")
+        for name, schema in changes.revoke_schemas:
+            if connection.execute(text("SELECT to_regnamespace(:schema) IS NOT NULL"), {"schema": quote(schema)}).scalar():
+                connection.exec_driver_sql(f"REVOKE USAGE ON SCHEMA {quote(schema)} FROM {quote(name)}")
+        for member, parent in changes.grant_memberships:
+            connection.exec_driver_sql(f"GRANT {quote(parent)} TO {quote(member)}")
+        for schema in sorted({schema for _, schema, _, _ in changes.grant_privileges} - {"metatables"}):
+            connection.exec_driver_sql(f"GRANT USAGE ON SCHEMA {quote(schema)} TO {quote(invoker)}")
+        self._grant_privileges(connection, changes.grant_privileges)
+
+    def _apply_all(self, connection, desired, previous, *, database, invoker):
+        """The full rebuild: every role is reset and granted again, repairing drift."""
+        roles = desired["roles"]
+        self._role(connection, invoker)
         for name, definition in roles.items():
             self._role(connection, name, self.password(name) if definition["login"] else None)
         names = set(roles) | set(previous.get("roles", {})) | {invoker}
         schemas = self.owned_schemas(connection)
         for name in sorted(names):
-            owned = connection.execute(
-                text(
-                    "SELECT 1 FROM pg_class WHERE relowner=(SELECT oid FROM pg_roles WHERE rolname=:name) UNION ALL SELECT 1 FROM pg_namespace WHERE nspowner=(SELECT oid FROM pg_roles WHERE rolname=:name) UNION ALL SELECT 1 FROM pg_proc WHERE proowner=(SELECT oid FROM pg_roles WHERE rolname=:name) UNION ALL SELECT 1 FROM pg_database WHERE datdba=(SELECT oid FROM pg_roles WHERE rolname=:name)"
-                ),
-                {"name": name},
-            ).first()
-            if owned:
-                raise SQLExecutionError(code="sql_principal_permission_drift")
-            # Repair inherited drift as well as direct table privileges.
-            parents = (
-                connection.execute(
-                    text(
-                        "SELECT parent.rolname FROM pg_auth_members m JOIN pg_roles child ON child.oid=m.member JOIN pg_roles parent ON parent.oid=m.roleid WHERE child.rolname=:name"
-                    ),
-                    {"name": name},
-                )
-                .scalars()
-                .all()
-            )
-            for parent in parents:
-                connection.exec_driver_sql(f"REVOKE {quote(parent)} FROM {quote(name)}")
-            connection.exec_driver_sql(
-                f"REVOKE ALL ON DATABASE {quote(database)} FROM {quote(name)}"
-            )
-            # Revoke this role's grants on owned objects; other owners' grants are theirs.
-            granted = connection.execute(
-                text(
-                    "SELECT 'SCHEMA', quote_ident(n.nspname) FROM pg_namespace n CROSS JOIN LATERAL aclexplode(n.nspacl) acl "
-                    "JOIN pg_roles r ON r.oid=acl.grantee WHERE r.rolname=:name AND " + OWNED.format(owner="n.nspowner")
-                    + " UNION SELECT CASE WHEN c.relkind='S' THEN 'SEQUENCE' ELSE 'TABLE' END, "
-                    "quote_ident(n.nspname) || '.' || quote_ident(c.relname) FROM pg_class c "
-                    "JOIN pg_namespace n ON n.oid=c.relnamespace CROSS JOIN LATERAL aclexplode(c.relacl) acl "
-                    "JOIN pg_roles r ON r.oid=acl.grantee WHERE r.rolname=:name AND " + OWNED.format(owner="c.relowner")
-                ),
-                {"name": name},
-            ).all()
-            for kind, target in granted:
-                connection.exec_driver_sql(f"REVOKE ALL ON {kind} {target} FROM {quote(name)}")
-            # Table-level REVOKE does not remove separately granted column ACLs.
-            columns = connection.execute(
-                text(
-                    "SELECT n.nspname, c.relname, a.attname FROM pg_attribute a JOIN pg_class c ON c.oid=a.attrelid JOIN pg_namespace n ON n.oid=c.relnamespace CROSS JOIN LATERAL aclexplode(a.attacl) acl JOIN pg_roles r ON r.oid=acl.grantee WHERE r.rolname=:name AND n.nspname !~ '^pg_' AND "
-                    + OWNED.format(owner="c.relowner")
-                ),
-                {"name": name},
-            ).all()
-            for schema, table, column in columns:
-                connection.exec_driver_sql(
-                    f"REVOKE ALL ({quote(column)}) ON TABLE {quote(schema)}.{quote(table)} FROM {quote(name)}"
-                )
-            if name in roles and roles[name]["login"]:
-                connection.exec_driver_sql(
-                    f"GRANT CONNECT ON DATABASE {quote(database)} TO {quote(name)}"
-                )
-                connection.exec_driver_sql(f"GRANT {quote(invoker)} TO {quote(name)}")
+            login = name in roles and bool(roles[name]["login"])
+            self._reset_role(connection, name, login=login, database=database, invoker=invoker)
         for schema in schemas:
             if schema != "metatables":
                 connection.exec_driver_sql(
@@ -312,13 +316,74 @@ class PostgreSQLAccess(DatabaseAccess):
                 )
         for member, parent in desired["memberships"]:
             connection.exec_driver_sql(f"GRANT {quote(parent)} TO {quote(member)}")
-        for name, schema, table, write in desired["privileges"]:
+        self._grant_privileges(connection, desired["privileges"])
+
+    def _grant_privileges(self, connection, privileges):
+        for name, schema, table, write in privileges:
             connection.exec_driver_sql(f"GRANT USAGE ON SCHEMA {quote(schema)} TO {quote(name)}")
             permissions = "SELECT, INSERT, UPDATE, DELETE" if write else "SELECT"
             connection.exec_driver_sql(
                 f"GRANT {permissions} ON TABLE {quote(schema)}.{quote(table)} TO {quote(name)}"
             )
 
+    def _reset_role(self, connection, name, *, login, database, invoker):
+        """Leave ``name`` with only CONNECT and the routine invoker, if it logs in."""
+        owned = connection.execute(
+            text(
+                "SELECT 1 FROM pg_class WHERE relowner=(SELECT oid FROM pg_roles WHERE rolname=:name) UNION ALL SELECT 1 FROM pg_namespace WHERE nspowner=(SELECT oid FROM pg_roles WHERE rolname=:name) UNION ALL SELECT 1 FROM pg_proc WHERE proowner=(SELECT oid FROM pg_roles WHERE rolname=:name) UNION ALL SELECT 1 FROM pg_database WHERE datdba=(SELECT oid FROM pg_roles WHERE rolname=:name)"
+            ),
+            {"name": name},
+        ).first()
+        if owned:
+            raise SQLExecutionError(code="sql_principal_permission_drift")
+        # Repair inherited drift as well as direct table privileges.
+        parents = (
+            connection.execute(
+                text(
+                    "SELECT parent.rolname FROM pg_auth_members m JOIN pg_roles child ON child.oid=m.member JOIN pg_roles parent ON parent.oid=m.roleid WHERE child.rolname=:name"
+                ),
+                {"name": name},
+            )
+            .scalars()
+            .all()
+        )
+        for parent in parents:
+            connection.exec_driver_sql(f"REVOKE {quote(parent)} FROM {quote(name)}")
+        connection.exec_driver_sql(
+            f"REVOKE ALL ON DATABASE {quote(database)} FROM {quote(name)}"
+        )
+        # Revoke this role's grants on owned objects; other owners' grants are theirs.
+        granted = connection.execute(
+            text(
+                "SELECT 'SCHEMA', quote_ident(n.nspname) FROM pg_namespace n CROSS JOIN LATERAL aclexplode(n.nspacl) acl "
+                "JOIN pg_roles r ON r.oid=acl.grantee WHERE r.rolname=:name AND " + OWNED.format(owner="n.nspowner")
+                + " UNION SELECT CASE WHEN c.relkind='S' THEN 'SEQUENCE' ELSE 'TABLE' END, "
+                "quote_ident(n.nspname) || '.' || quote_ident(c.relname) FROM pg_class c "
+                "JOIN pg_namespace n ON n.oid=c.relnamespace CROSS JOIN LATERAL aclexplode(c.relacl) acl "
+                "JOIN pg_roles r ON r.oid=acl.grantee WHERE r.rolname=:name AND " + OWNED.format(owner="c.relowner")
+            ),
+            {"name": name},
+        ).all()
+        for kind, target in granted:
+            connection.exec_driver_sql(f"REVOKE ALL ON {kind} {target} FROM {quote(name)}")
+        # Table-level REVOKE does not remove separately granted column ACLs.
+        columns = connection.execute(
+            text(
+                "SELECT n.nspname, c.relname, a.attname FROM pg_attribute a JOIN pg_class c ON c.oid=a.attrelid JOIN pg_namespace n ON n.oid=c.relnamespace CROSS JOIN LATERAL aclexplode(a.attacl) acl JOIN pg_roles r ON r.oid=acl.grantee WHERE r.rolname=:name AND n.nspname !~ '^pg_' AND "
+                + OWNED.format(owner="c.relowner")
+            ),
+            {"name": name},
+        ).all()
+        for schema, table, column in columns:
+            connection.exec_driver_sql(
+                f"REVOKE ALL ({quote(column)}) ON TABLE {quote(schema)}.{quote(table)} FROM {quote(name)}"
+            )
+        if login:
+            connection.exec_driver_sql(
+                f"GRANT CONNECT ON DATABASE {quote(database)} TO {quote(name)}"
+            )
+            connection.exec_driver_sql(f"GRANT {quote(invoker)} TO {quote(name)}")
+
     @contextmanager
     def connection(self, identity, *, write, timeout):
         import psycopg

{mainsequence_metatable-0.1.28.dev48 → mainsequence_metatable-0.1.28.dev50}/src/metatables/api/backend/backends/sqlite_access.py

@@ -10,6 +10,8 @@ from metatables.api.backend.persistence.models import Base, SQLSecurityState
 
 
 class SQLiteAccess(DatabaseAccess):
+    incremental = True
+
     def submit(self, cursor, statement, parameters):
         cursor.execute(statement, parameters)
 
@@ -75,7 +77,7 @@ class SQLiteAccess(DatabaseAccess):
                     )
         return targets
 
-    def apply(self, session, desired, previous):
+    def apply(self, session, desired, previous, *, full=True):
         # The immutable grant snapshot is installed on each dedicated connection.
         return None