mainsequence-metatable 0.1.31.dev59__tar.gz → 0.1.31.dev60__tar.gz

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.31.dev59
+Version: 0.1.31.dev60
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/docs/adr/api/0007-database-enforced-table-access.md

@@ -51,6 +51,15 @@
 > are when grants changed outside MetaTables are repaired, not on every catalog
 > change. MySQL and SQL Server keep the full sweep on every change.
 
+> Amendment (2026-10-10): [ADR 0014](0014-main-sequence-release-jobs-and-production-migrations.md#conditional-system-bootstrap-2026-10-10-amendment)
+> replaces the unconditional deployment sweep on PostgreSQL/TimescaleDB with
+> fresh native-security and login validation under the security-state lock.
+> Every deployment still checks table policies. Unchanged security evidence uses
+> incremental grant application and one conditional write of the complete
+> manifest; drift, credential rotation, pending state or missing evidence retains
+> full repair. An explicit admin repair remains a full sweep. A routine catalog
+> update must not establish a new native baseline for unverified outside drift.
+
 > Amendment (2026-10-09): [ADR 0022](0022-concurrent-requests-and-connection-reuse.md)
 > reuses login-role connections within a pod, only by the login role that opened them
 > and never through `SET ROLE`; no session state carries over from one use to the next.

mainsequence_metatable-0.1.31.dev60/docs/adr/api/0014-main-sequence-release-jobs-and-production-migrations.md

@@ -0,0 +1,183 @@
+# ADR 0014: System migrations in the deployment workflow
+
+Date: 2026-10-02
+
+Status: Accepted. Implemented on `development`; hosted verification pending.
+Amended 2026-10-03 ([MetaTables #13](https://github.com/mainsequence-projects/MetaTables/issues/13)): the Job reads the declared runtime database itself
+and initializes it on the first deployment.
+
+Owners: MetaTables API. Main Sequence owns Job execution, images, deployment and
+platform authentication.
+
+Related: [ADR 0001: Runtime storage and bootstrap](0001-unified-api-storage-and-local-sqlite.md),
+[ADR 0008: Database workflows and recovery](0008-mysql-mssql-table-workflows.md) and
+[ADR 0013: Application-owned migrations](0013-application-owned-migrations.md).
+
+## Context
+
+Main Sequence deploys the hosted API from `.mainsequence/workflows/metatables-api.yaml`.
+Settings runs system migrations inside the running API, and that API can only
+apply the revisions it ships. A release's new revisions must be applied before
+its API serves.
+
+## Decision
+
+### The Job prepares the declared database (amended 2026-10-03)
+
+The Job no longer reads the running API. `metatables runtime upgrade` loads
+`runtime_database` from the candidate image's `configuration.yaml` and the
+Environment Secret it names ([ADR 0001](0001-unified-api-storage-and-local-sqlite.md)).
+Every run executes the same steps:
+
+1. Check that the login can secure the database (ADR 0007 prerequisites).
+2. Apply the system migrations.
+3. Register the runtime DataSource, or update its connection to the declaration.
+4. Freshly validate access setup and repair changes. Per-User role passwords
+   derive from the login password, so a rotated password reaches them on the
+   next deployment. The conditional reconciliation amendment below replaces
+   unconditional rebuilding on every deployment.
+
+The first deployment therefore initializes the database. A missing declaration
+or Secret, an unreachable database or unmet prerequisites fail the Job and block
+the rollout. The deployment workflow is the authority, so no admin caller is
+required. A new runtime DataSource records the Job's SDK user as its creator.
+The Job prints `initialized`, `upgraded` or `up_to_date` with the revisions.
+
+Settings' **Run MetaTables migrations** and `metatables runtime initialize` remain
+for Local mode only. The original decision follows. The amendment supersedes
+steps 1–2, the skip rules and the Settings initialization paragraph.
+
+### Conditional system bootstrap (2026-10-10 amendment)
+
+An unchanged deployment remains a validation gate, but does not restart completed
+work or rewrite an identical catalog/security projection. This concerns the
+MetaTables **system** Job, separate from application-provider finalization in
+[ADR 0013](0013-application-owned-migrations.md).
+
+```text
+resolve declaration and Secret
+  -> bootstrap lock
+  -> fresh history, recovery and runtime-binding checks
+  -> current heads + completed journal?
+       yes: fresh physical system-schema validation; preserve journal
+       no: existing Alembic execution and durable recovery; validate schema
+  -> reconcile runtime registration if changed
+  -> security-state lock; fresh native security and login validation
+  -> unchanged evidence? fresh table-policy checks; apply only grant differences
+       otherwise: full security repair, including credential rotation
+  -> commit; fresh registration/reference verification; release bootstrap lock
+```
+
+Hosted configuration defers database inspection to the authoritative preflight
+inside the bootstrap lock. The local path retains its pre-lock check, so an old
+store cannot be replaced or a new file created before compatibility is checked.
+Registration/reference validation is read again after commit under the lock;
+it does not repeat schema reflection or the already verified migration history.
+Startup and explicit activation continue their existing fresh checks.
+
+Matching Alembic heads alone are insufficient. Only a **completed** progress
+journal allows skipping Alembic and journal writes, after fresh schema validation.
+A missing journal is established through the existing path. An incomplete
+journal retains plan/binding checks, acknowledged-step recovery and rejection
+of unknown outcomes. A completed journal records its original execution; a
+source-code change without new revisions does not reset that history. Schema
+drift still blocks deployment and is not silently repaired by replaying Alembic.
+
+On PostgreSQL/TimescaleDB, the existing private `SQLSecurityState.manifest` stores
+versioned native-security evidence and a domain-separated credential checksum,
+alongside roles, memberships, privileges and routine evidence. No schema revision
+is needed. Initialization compares fresh database evidence: role attributes and
+memberships; schema, relation, column, routine, database and default ACLs; owners,
+routine definitions and row-security policies. Table data and statistics are
+excluded. Declared schemas and grant-target tables are included even when the
+login has grant authority without owning them. Fresh per-login authentication
+uses the configured TLS material and a new connection, never a pooled session
+that could conceal a password change.
+The security-state row stays locked until commit to serialize with catalog
+changes and caller admission.
+
+All registered table policies are freshly checked even on the unchanged path.
+Unchanged table owners and `PUBLIC` grants cause no ownership/revoke writes.
+Policies and grant differences still apply when a trigger or catalog change
+affects access. Native drift, credential rotation, pending/unready state or
+missing evidence invokes full repair. Login authentication failures likewise
+require repair. Exhausted login connection capacity conservatively falls back
+to the existing repair. libpq can omit SQLSTATE on a connection/authentication
+failure; these ambiguous driver connection errors also require repair rather
+than allowing a skip. Other probe failures fail deployment. An old installation
+therefore performs one full repair to establish evidence, then converges.
+Explicit admin repair remains a full sweep; MySQL/SQL Server keep their existing
+security initialization behavior.
+
+The complete manifest is compared and persisted once, including backend fields;
+reconciliation must not remove and then restore the routine fingerprint. Ordinary
+incremental changes cannot checkpoint unverified native drift as success. Failed
+security work rolls back its catalog checkpoint and remains repairable on retry.
+
+Named timings cover declaration/Secret resolution, bootstrap/security lock waits,
+preflight, catalog/Alembic execution, system-schema validation, registration,
+native security and login authentication validation, routine repair, table
+policies, roles/grants, credential-store setup, commit and final registration
+verification. Each timing
+reports completion/failure without connection strings, passwords, checksums or
+driver exception text. Successful security setup logs the selected path and its
+reason, including first-run evidence establishment.
+
+Focused verification uses disposable SQLite catalog transactions and recording
+PostgreSQL adapters: unchanged deployments invoke no Alembic or catalog/journal
+DML; unchanged native security emits no role/grant DDL or manifest UPDATE;
+schema drift, interrupted/ambiguous recovery, credential rotation, permission
+drift and failed repair remain checked. Live PostgreSQL/TimescaleDB validation is
+separate from these non-container checks.
+
+### Original decision (2026-10-02)
+
+The deployment workflow upgrades the runtime from the candidate image before the
+rollout:
+
+```text
+candidate image -> migrate-system Job -> API deployment
+```
+
+A failed upgrade blocks the rollout.
+
+The Job, `jobs/migrate_system.py`, runs `metatables runtime upgrade`:
+
+1. Find the Environment's MetaTables API the same way the client does.
+2. Read `GET /runtime-context/` as an organization admin. For admins,
+   `bootstrap.candidate` is the Settings DataSource's public configuration with
+   its Secret references; the Secrets are resolved through the SDK.
+3. Apply this package's system migrations through the shared bootstrap operation:
+   backend lock, progress recovery, runtime registration, access setup and
+   verification. A runtime already at the current revision is left untouched.
+
+The command acts only on an active runtime. With no API deployment or no active
+runtime yet, it succeeds without DDL. The Environment's DataSource is set
+through the deployed API, so if it is set the command migrates when necessary;
+if it is not, it skips migrations and the API deploys. An API release that the
+workflow created but has never deployed counts as no API deployment, so the
+first run deploys the API. A non-admin identity or an unreadable
+context fails before DDL. When it registers the runtime DataSource, it records
+the Job identity's user UID as the creator.
+
+Settings' **Run MetaTables migrations** and `metatables runtime initialize` keep
+initializing a newly selected DataSource: the API serving them is the deployed
+revision and runs its own migrations, so there is no version skew. API startup
+stays inspection-only.
+
+Both targets declare `automatic_redeployment` with `tag_regex: null`, so every
+eligible commit runs the graph. Package publication stays in GitHub Actions.
+
+The upgrade owns only `metatables.api.backend.migrations` and
+`metatables_catalog_version`. Application providers keep their own histories
+under ADR 0013.
+
+## Consequences
+
+- The previous API keeps serving during the upgrade, so each schema change must
+  stay compatible with the previous release. Coordination with its writes is not
+  defined yet.
+- An upgrade followed by a failed deployment leaves the database upgraded. There
+  is no automatic downgrade.
+- The workflow runs only when the platform receives the branch's commit.
+  Verification in a hosted Environment is pending.

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mainsequence-metatable"
-version = "0.1.31.dev59"
+version = "0.1.31.dev60"
 description = "Main Sequence MetaTables Python client and standalone API"
 readme = "README.md"
 requires-python = ">=3.13"

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/src/mainsequence_metatable.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.31.dev59
+Version: 0.1.31.dev60
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/src/mainsequence_metatable.egg-info/SOURCES.txt

@@ -332,6 +332,7 @@ src/metatables/api/backend/migrations/versions/0012_table_creator.py
 src/metatables/api/backend/migrations/versions/__init__.py
 src/metatables/api/backend/operations/__init__.py
 src/metatables/api/backend/operations/bootstrap_migrations.py
+src/metatables/api/backend/operations/bootstrap_timing.py
 src/metatables/api/backend/operations/catalog_schema.py
 src/metatables/api/backend/operations/create_table.py
 src/metatables/api/backend/operations/finalize_managed.py
@@ -575,6 +576,7 @@ tests/packaging/test_distribution_sources.py
 tests/packaging/test_editable_build.py
 tests/packaging/test_installed_distribution.py
 tests/permissions/test_access_map.py
+tests/permissions/test_deployment_security.py
 tests/permissions/test_governed_sql_scope.py
 tests/permissions/test_incremental_security.py
 tests/permissions/test_local_actor_membership.py

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/src/metatables/api/app/bootstrap.py

@@ -34,6 +34,7 @@ from metatables.api.backend.integrations.credential_store import (
 )
 from metatables.api.backend.integrations.local_workspace import _bind_file, _read_marker
 from metatables.api.backend.integrations.runtime_binding import RuntimeStorageBinding
+from metatables.api.backend.operations.bootstrap_timing import bootstrap_phase
 from metatables.api.backend.persistence.database import (
     create_catalog_engine,
     migration_config,
@@ -234,7 +235,7 @@ class RuntimeBootstrap:
         self.reader = None
         self.resources.close()
 
-    def configure(self, body, owner, *, validate_schema=True):
+    def configure(self, body, owner, *, validate_schema=True, inspect_database=True):
         # Validate public configuration before replacing the current candidate.
         values = RuntimeSourceInput.model_validate(body).model_dump(mode="json")
         kind = values["class_type"]
@@ -272,7 +273,8 @@ class RuntimeBootstrap:
         self.candidate, self.owner = values, owner
         self.active, self.uid = False, None
         self.status, self.error, self.current = "unconfigured", None, []
-        self.inspect(validate_schema=validate_schema)
+        if inspect_database:
+            self.inspect(validate_schema=validate_schema)
 
     def open_engine(self, *, create=False):
         from metatables.api.backend.backends.registry import backend_for_engine
@@ -332,34 +334,7 @@ class RuntimeBootstrap:
                         validate_system_schema,
                     )
                     validate_system_schema(connection, self.backend.catalog)
-            with self.factory() as session:
-                row = session.scalar(select(DataSource).where(DataSource.is_runtime.__eq__(True)))
-                if row is None:
-                    self.status = "registration_required"
-                    with engine.connect() as connection:
-                        self.require_prerequisites(connection)
-                elif not self.matches(row):
-                    if self.config.local_mode:
-                        self.status = "incompatible"
-                        self.error = "The stored runtime DataSource does not match this connection."
-                    else:
-                        # The declaration is authoritative; the deployment updates the record.
-                        self.status = "registration_required"
-                        self.error = "The runtime database declaration changed. The next deployment updates its registration."
-                else:
-                    binding = RuntimeStorageBinding(str(engine.url), self.workspace, row.uid, self.candidate["configuration"])
-                    for registered in session.scalars(select(DataSource)):
-                        binding.validate_source(registered)
-                    for model in (MetaTable, PhysicalOperation):
-                        query = select(model.uid).where(model.data_source_uid != row.uid)
-                        if model is MetaTable:
-                            query = query.where(MetaTable.management_mode != "external_registered")
-                        if session.scalar(query.limit(1)):
-                            raise BootstrapError("Managed runtime catalog references another DataSource.")
-                    if session.scalar(select(MetaTable.uid).where(~MetaTable.data_source_uid.in_(select(DataSource.uid))).limit(1)):
-                        raise BootstrapError("MetaTable references an unregistered DataSource.")
-                    self.uid = row.uid
-                    self.status = "ready"
+            self.inspect_registration()
         except (BootstrapError, CatalogConfigurationError) as exc:
             logger.warning("Runtime database is incompatible", exc_info=True)
             self.status, self.error = "incompatible", str(exc)
@@ -375,6 +350,39 @@ class RuntimeBootstrap:
             logger.info("Runtime database inspection took %.3fs (status %s, schema validated: %s)",
                         perf_counter() - started, self.status, validate_schema)
 
+    def inspect_registration(self):
+        """Fresh binding and reference checks, without repeating schema/history reads."""
+        self.error = None
+        engine = self.engine
+        with self.factory() as session:
+            row = session.scalar(select(DataSource).where(DataSource.is_runtime.__eq__(True)))
+            if row is None:
+                self.status = "registration_required"
+                with engine.connect() as connection:
+                    self.require_prerequisites(connection)
+            elif not self.matches(row):
+                if self.config.local_mode:
+                    self.status = "incompatible"
+                    self.error = "The stored runtime DataSource does not match this connection."
+                else:
+                    # The declaration is authoritative; the deployment updates the record.
+                    self.status = "registration_required"
+                    self.error = "The runtime database declaration changed. The next deployment updates its registration."
+            else:
+                binding = RuntimeStorageBinding(str(engine.url), self.workspace, row.uid, self.candidate["configuration"])
+                for registered in session.scalars(select(DataSource)):
+                    binding.validate_source(registered)
+                for model in (MetaTable, PhysicalOperation):
+                    query = select(model.uid).where(model.data_source_uid != row.uid)
+                    if model is MetaTable:
+                        query = query.where(MetaTable.management_mode != "external_registered")
+                    if session.scalar(query.limit(1)):
+                        raise BootstrapError("Managed runtime catalog references another DataSource.")
+                if session.scalar(select(MetaTable.uid).where(~MetaTable.data_source_uid.in_(select(DataSource.uid))).limit(1)):
+                    raise BootstrapError("MetaTable references an unregistered DataSource.")
+                self.uid = row.uid
+                self.status = "ready"
+
     def require_prerequisites(self, connection):
         # Before any setup DDL: a database the login cannot secure must never be
         # half-initialized. A registered runtime may hold external objects.
@@ -401,42 +409,55 @@ class RuntimeBootstrap:
         try:
             # Check before creating a file or scope marker: an earlier local
             # store must remain untouched even when this target is absent.
-            self.inspect(validate_schema=False)
-            if self.status not in {"migration_required", "registration_required", "ready"}:
-                raise BootstrapError(self.error or "The selected database cannot be migrated.")
-            with self.migration_lock():
-                self.inspect(validate_schema=False)
+            if self.config.local_mode:
+                with bootstrap_phase("local_preflight"):
+                    self.inspect(validate_schema=False)
+                if self.status not in {"migration_required", "registration_required", "ready"}:
+                    raise BootstrapError(self.error or "The selected database cannot be migrated.")
+            with ExitStack() as locks:
+                with bootstrap_phase("lock_wait"):
+                    locks.enter_context(self.migration_lock())
+                with bootstrap_phase("locked_preflight"):
+                    self.inspect(validate_schema=False)
                 if self.status not in {"migration_required", "registration_required", "ready"}:
                     raise BootstrapError(self.error or "The selected database cannot be migrated.")
+                previous = sorted(self.current)
                 self.status = "migrating"
-                upgrade_catalog(self.engine)
+                with bootstrap_phase("catalog_migrations"):
+                    upgrade_catalog(self.engine)
                 # The registry is used only after Alembic created/upgraded it.
                 with self.factory() as session:
-                    row = session.scalar(select(DataSource).where(DataSource.is_runtime.__eq__(True)))
-                    if row is None:
-                        if session.scalar(select(DataSource.uid).where(DataSource.is_default.__eq__(True))):
-                            raise BootstrapError("An existing default requires explicit reconciliation before runtime registration.")
-                        row = DataSource(uid=self.workspace.data_source.uid if self.workspace else uuid4(),
-                            created_by_user_uid=user_uid, **self.candidate, is_runtime=True,
-                            is_default=True, status="AVAILABLE", storage_access_mode="read_write")
-                        session.add(row)
-                    elif not self.matches(row):
-                        if self.config.local_mode:
-                            raise BootstrapError("The existing runtime registration cannot be replaced by this configuration or caller.")
-                        # Same database, new declared connection (host, login Secret, TLS).
-                        row.class_type, row.configuration = self.candidate["class_type"], self.candidate["configuration"]
-                        row.is_default, row.status, row.storage_access_mode = True, "AVAILABLE", "read_write"
-                    from metatables.api.backend.backends.registry import backend_for
-                    session.flush()
-                    backend_for(replace(self.backend.source, uid=row.uid)).access.initialize(session)
-                    credentials = bind_store(self.credential_store, session, user_uid)
-                    if hasattr(credentials, "initialize"):
-                        credentials.initialize()
-                    session.commit()
+                    with bootstrap_phase("runtime_registration"):
+                        row = session.scalar(select(DataSource).where(DataSource.is_runtime.__eq__(True)))
+                        if row is None:
+                            if session.scalar(select(DataSource.uid).where(DataSource.is_default.__eq__(True))):
+                                raise BootstrapError("An existing default requires explicit reconciliation before runtime registration.")
+                            row = DataSource(uid=self.workspace.data_source.uid if self.workspace else uuid4(),
+                                created_by_user_uid=user_uid, **self.candidate, is_runtime=True,
+                                is_default=True, status="AVAILABLE", storage_access_mode="read_write")
+                            session.add(row)
+                        elif not self.matches(row):
+                            if self.config.local_mode:
+                                raise BootstrapError("The existing runtime registration cannot be replaced by this configuration or caller.")
+                            # Same database, new declared connection (host, login Secret, TLS).
+                            row.class_type, row.configuration = self.candidate["class_type"], self.candidate["configuration"]
+                            row.is_default, row.status, row.storage_access_mode = True, "AVAILABLE", "read_write"
+                        from metatables.api.backend.backends.registry import backend_for
+                        session.flush()
+                    with bootstrap_phase("security_reconciliation"):
+                        backend_for(replace(self.backend.source, uid=row.uid)).access.initialize(session)
+                    with bootstrap_phase("credential_store"):
+                        credentials = bind_store(self.credential_store, session, user_uid)
+                        if hasattr(credentials, "initialize"):
+                            credentials.initialize()
+                    with bootstrap_phase("registration_commit"):
+                        session.commit()
                 # upgrade_catalog already verified the complete system schema.
                 # Recheck registration before releasing the lock, without
                 # repeating every table/column/index/constraint lookup.
-                self.inspect(validate_schema=False)
+                self.current = list(self.expected)
+                with bootstrap_phase("registration_verification"):
+                    self.inspect_registration()
                 if self.status != "ready":
                     raise BootstrapError(self.error or "Migration finished but the runtime is not ready.")
         except (BootstrapError, CatalogConfigurationError) as exc:
@@ -462,6 +483,7 @@ class RuntimeBootstrap:
             self.error = (f"MetaTables migration or registration failed ({cause}). The runtime was not activated; "
                           "inspect the database before retrying. The log has the full error.")
             raise BootstrapError(self.error) from None
+        return previous
 
     def persist_selection(self):
         from metatables.api.app.runtime_control import write_state
@@ -563,18 +585,19 @@ def deploy_runtime_database(user_uid):
 
     The workflow's migration Job runs this from the candidate image before the
     API rolls out, every time; any failure blocks the rollout. Access setup is
-    reapplied too, so a rotated login password reaches the per-User roles.
+    freshly checked too; drift and rotated credentials are repaired as needed.
     """
     from metatables.api.app.configuration import DEPLOYMENT_CONFIGURATION
     from metatables.api.app.runtime import RuntimeConfig
     target = RuntimeBootstrap(RuntimeConfig(local_mode=False, configuration_path=DEPLOYMENT_CONFIGURATION), None)
     try:
-        candidate = target.initial_candidate()
-        if candidate is None:
-            raise BootstrapError(target.error)
-        target.configure(candidate, user_uid, validate_schema=False)
-        previous = sorted(target.current)
-        target.migrate(user_uid)
+        with bootstrap_phase("candidate_resolution"):
+            candidate = target.initial_candidate()
+            if candidate is None:
+                raise BootstrapError(target.error)
+        target.configure(candidate, user_uid, validate_schema=False, inspect_database=False)
+        with bootstrap_phase("deployment_migration"):
+            previous = target.migrate(user_uid)
         revisions = sorted(target.current)
         status = "initialized" if not previous else "upgraded" if previous != revisions else "up_to_date"
         return dict(status=status, previous_revisions=previous, revisions=revisions, data_source_uid=str(target.uid))

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/src/metatables/api/backend/backends/access.py

@@ -312,11 +312,13 @@ class DatabaseAccess:
                     memberships.add((login, role("team", team, write)))
         return dict(roles=roles, memberships=sorted(memberships), privileges=sorted(privileges))
 
-    def reconcile(self, session, *, inspect_objects=False, tables=None):
+    def reconcile(self, session, *, inspect_objects=False, tables=None, full=None):
         """Mirror the catalog into database grants.
 
-        ``inspect_objects`` is the full sweep: every policy is inspected and every
-        role rebuilt, which also repairs grants changed outside MetaTables.
+        ``inspect_objects`` inspects every policy and, by default, rebuilds every
+        role, repairing grants changed outside MetaTables. A backend that has
+        freshly verified native security may explicitly choose ``full=False``
+        while retaining the fresh policy sweep.
         Otherwise only the policies ``tables`` can affect are inspected, and an
         incremental backend applies only the grants that changed.
         """
@@ -328,7 +330,12 @@ class DatabaseAccess:
         elif tables:
             self.refresh_policies(session, tables=self.policy_scope(session, tables))
         desired = self.plan(session)
-        self.apply(session, desired, state.manifest, full=inspect_objects)
+        full = inspect_objects if full is None else full
+        self.apply(session, desired, state.manifest, full=full)
+        self.save_manifest(session, state, desired)
+
+    def save_manifest(self, session, state, desired):
+        """Persist the complete backend manifest once, only if it changed."""
         # JSON loads arrays as lists, while plan() builds tuples. Compare the
         # persisted representation so equal grants do not rewrite security state.
         if json.dumps(state.manifest, sort_keys=True) != json.dumps(desired, sort_keys=True):

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/src/metatables/api/backend/backends/adapters/postgresql_connection.py

@@ -118,6 +118,19 @@ def _owned_connection(connect, params, settings):
     return connection, close
 
 
+def validate_login(settings, user, password):
+    """Authenticate afresh: a pooled connection cannot detect password drift."""
+    import psycopg
+
+    params = dict(host=settings.host, port=settings.port, dbname=settings.database_name,
+                  user=user, password=password, sslmode=settings.ssl_mode, connect_timeout=5)
+    connection, close = _owned_connection(psycopg.connect, params, settings)
+    try:
+        connection.execute("SELECT 1", prepare=False)
+    finally:
+        close()
+
+
 @contextmanager
 def login_session(settings, user, password, *, prepare):
     """A psycopg connection as ``user``, reused only under the same login (ADR 0022).

{mainsequence_metatable-0.1.31.dev59 → mainsequence_metatable-0.1.31.dev60}/src/metatables/api/backend/backends/catalog.py

@@ -73,7 +73,8 @@ class PostgreSQLCatalog:
     def initialize_namespace(self, connection):
         if connection.exec_driver_sql("SELECT to_regclass('public.metatables_catalog_version')").scalar():
             raise CatalogConfigurationError("Earlier development catalogs must be recreated through Settings.")
-        connection.exec_driver_sql("CREATE SCHEMA IF NOT EXISTS metatables")
+        if not connection.exec_driver_sql("SELECT to_regnamespace('metatables')").scalar():
+            connection.exec_driver_sql("CREATE SCHEMA IF NOT EXISTS metatables")
 
     def unique_constraints(self, inspector, table, schema=None):
         return inspector.get_unique_constraints(table, schema=schema)