mainsequence-metatable 0.1.36.dev66__tar.gz → 0.1.36.dev67__tar.gz

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/.mainsequence/workflows/metatables-api.yaml

@@ -32,6 +32,9 @@ resources:
       resources:
         - key: metatables-analyst-agent
           level: view
+        # The Admin's "Run security check now" starts the daily check (ADR 0023).
+        - key: check-security
+          level: edit
 
   - key: metatables-analyst-agent
     kind: harness_agent
@@ -76,6 +79,30 @@ resources:
         - name: METATABLES_RUNTIME_DATABASE
           level: view
 
+  # The full database security check, outside the deployment (ADR 0023). Not a
+  # workflow step: every push activates its image, and it skips while the
+  # database still awaits that image's system migrations.
+  - key: check-security
+    kind: job
+    spec:
+      name: MetaTables security check
+      execution_path: jobs/check_security.py
+      cpu_request: "1"
+      memory_request: "1"
+      max_runtime_seconds: 1800
+      automatic_redeployment:
+        enabled: true
+        tag_regex: null
+      task_schedule:
+        schedule:
+          type: crontab
+          # Daily at 03:00 UTC.
+          expression: "0 3 * * *"
+    access:
+      secrets:
+        - name: METATABLES_RUNTIME_DATABASE
+          level: view
+
 execution:
   steps:
     image:

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.36.dev66
+Version: 0.1.36.dev67
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/adr/api/0007-database-enforced-table-access.md

@@ -74,6 +74,12 @@
 > login may grant them. Login passwords are set again only on credential
 > rotation, failed login validation, pending or uninitialized state.
 
+> Amendment (2026-10-15, [ADR 0023](0023-daily-security-check.md)): The full sweep
+> no longer runs in every deployment's migration Job. It runs as a daily Job, which
+> administrators can also start from the Admin; the admin repair that paused caller
+> SQL is removed. A deployment runs it only on first setup or after system migrations,
+> so grants changed outside MetaTables are repaired within a day.
+
 > Amendment (2026-10-09): [ADR 0022](0022-concurrent-requests-and-connection-reuse.md)
 > reuses login-role connections within a pod, only by the login role that opened them
 > and never through `SET ROLE`; no session state carries over from one use to the next.

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/adr/api/0014-main-sequence-release-jobs-and-production-migrations.md

@@ -37,6 +37,11 @@ Every run executes the same steps:
    next deployment. The conditional reconciliation amendment below replaces
    unconditional rebuilding on every deployment.
 
+[ADR 0023](0023-daily-security-check.md) (2026-10-15) narrows step 4: the Job sets
+up the owner role, sets login passwords again after a rotation and applies pending
+changes. It runs the full security check only on first setup or after system
+migrations; a daily Job runs it otherwise.
+
 The first deployment therefore initializes the database. A missing declaration
 or Secret, an unreachable database or unmet prerequisites fail the Job and block
 the rollout. The deployment workflow is the authority, so no admin caller is
@@ -96,6 +101,9 @@ The security-state row stays locked until commit to serialize with catalog
 changes and caller admission.
 
 All registered table policies are freshly checked even on the unchanged path.
+(Superseded by [ADR 0023](0023-daily-security-check.md): an unchanged deployment no
+longer checks table policies, compares grants or probes logins; the daily security
+check does.)
 Unchanged table owners and `PUBLIC` grants cause no ownership/revoke writes.
 Policies and grant differences still apply when a trigger or catalog change
 affects access. Grants changed outside MetaTables are found by the comparison

mainsequence_metatable-0.1.36.dev67/docs/adr/api/0023-daily-security-check.md

@@ -0,0 +1,109 @@
+# ADR 0023: A daily security check instead of a sweep in every deployment
+
+Status: Accepted.
+
+Implementation status: implemented on development. Non-container tests pass; the
+deployment, the first scheduled run and the container suites are pending.
+
+Owner: this repository.
+
+Related decisions: [ADR 0007](0007-database-enforced-table-access.md) (database roles
+enforce table access), [ADR 0014](0014-main-sequence-release-jobs-and-production-migrations.md)
+(the migration Job of every deployment) and
+[ADR 0013](0013-application-owned-migrations.md) (application migrations).
+
+## Context
+
+Every deployment's migration Job ran the full security check. It compared every role,
+membership, grant, table policy and routine MetaTables secures with the catalog's plan,
+opened a fresh connection for every login role, and repaired what differed. It held the
+security-state lock throughout, so callers' SQL waited. In development it took 135 s of
+a 147 s migration step; in one run
+([MetaTables #66](https://github.com/mainsequence-projects/MetaTables/issues/66)) it
+took 25 minutes, close to the Job's 30-minute limit.
+
+Most of that work repairs grants changed outside MetaTables. A deployment neither causes
+them nor repairs them in time: drift made just after a deployment stays until the next
+one, which in production can be weeks away. Drift the check cannot repair blocked the
+release, even a release meant to fix it.
+
+## Decision
+
+The full check runs once a day as its own Job, and administrators can run it at any
+time. A deployment does only what the new release needs.
+
+### What a deployment does
+
+The migration Job's security step:
+
+1. sets up the owner role `mt_owner`, which normally changes nothing;
+2. runs the full check on the first setup, and when the system migration heads differ
+   from the ones the last full check ran at. System migrations change catalog rows
+   without the hooks that keep grants in step. The heads are recorded with the check,
+   so nothing depends on a developer bumping a version number;
+3. sets every login password again when the DataSource password was rotated;
+4. applies pending changes.
+
+Otherwise it sends nothing.
+
+### The daily check
+
+- The `check-security` Job of the API workflow (`jobs/check_security.py`, exactly
+  `metatables runtime check-security`) runs daily at 03:00 UTC. It is not a workflow
+  step; every push activates its image.
+- It holds the bootstrap lock, so it waits for a running migration. It skips while the
+  database still awaits its image's system migrations, so a run that lands mid-rollout
+  never checks one release's catalog with another's rules. Any other state that is not
+  ready fails the run.
+- It repairs whatever differs from the plan: the owner role, routines, table policies,
+  roles, memberships and grants, and the passwords of logins that fail a fresh login
+  probe. The probe runs before the security-state lock is taken.
+- It never pauses caller SQL. A failed check rolls back what it changed.
+- System migration `0013_security_check` records the last check on the security state:
+  what started it (`schedule` or `admin`), its outcome (`running`, `passed`, `repaired`
+  or `failed`), its times, and the statements it sent or its error code.
+  `GET /security/database-permissions/` returns it as `last_check`.
+
+### Run it now
+
+`POST /security/checks/` (administrators only) replaces `POST /security/reconcile/`,
+which paused caller SQL during the repair and had to finish within the 300 s request
+timeout. Hosted starts the Job with `--trigger admin` and answers 202; the API's
+workload may run that Job (`level: edit`), one run at a time. Local runs the check in
+the request. The Admin's Database permissions panel has **Run security check now** and
+shows the last check.
+
+### Access rule changes
+
+A release that changes how MetaTables computes access reaches the database at the next
+daily check, or right away when an administrator runs it after deploying. Access rules
+are the plan (`plan()`), the table policy checks, the grant and revoke statements,
+routine securing and the owner role. Pull requests that change them carry the
+`access-rules` label, and release notes list them under "Access rule changes: run the
+security check after deploying". A version constant in the code was rejected: when
+someone forgets to bump it, the deployment skips the check silently.
+
+## Consequences
+
+- An unchanged deployment's security step reads the security state and the owner role
+  and sends nothing. It no longer grows with tables or users.
+- Grants changed outside MetaTables are repaired within a day, whatever the release
+  cadence.
+- Drift the check cannot repair fails the daily Job, not a release.
+- The daily check still makes caller SQL wait while it holds the lock, once a day at
+  03:00 UTC. If that wait matters, the check can compute the differences before taking
+  the lock.
+- Application migrations are unchanged: finalization checks only the tables it
+  finalizes (ADR 0013). A finalization waits for the daily check only if both run at
+  the same time.
+
+## Verification
+
+- `tests/permissions/test_deployment_security.py`: an unchanged deployment reads and
+  writes nothing; first setup and new system migration heads run the full check once;
+  a rotated password sets only passwords; pending changes apply; outside drift waits
+  for the check; the check probes logins before the lock and counts its repairs.
+- `tests/permissions/test_security_check.py`: the recorded outcome, the Job's lock and
+  skip, and starting the Job or refusing to.
+- `tests/auth/test_runtime_storage_binding.py`: Local runs the check in the request; a
+  failure is recorded and never pauses SQL.

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/adr/index.md

@@ -40,6 +40,7 @@ decisions belong in the SDK repository and are linked as dependencies.
 | [0017: Hosted caller identity through the SDK request identity](api/0017-hosted-request-identity.md) | Accepted; implemented for 0.1.21; hosted rollout verification pending | The hosted API installs the SDK request identity that the platform launcher requires; the SDK verifies each caller assertion once and admission reads `User.get_logged_user()`, including the caller's signed Teams and admin flag. Local keeps its loopback token admission. |
 | [0018: Table search by meaning and columns](api/0018-table-search.md) | Accepted; implemented; container engine verification pending | One route ranks visible tables by BM25 keywords and a local English embedding model over a per-table document that includes columns, merges the rankings and returns `matched_columns`; the index refreshes on every metadata write. |
 | [0022: Concurrent requests per pod and reused database connections](api/0022-concurrent-requests-and-connection-reuse.md) | Accepted; implemented on development; deployment and load verification pending | One number per pod sizes its threads, data-request admission, catalog pool and login-role connections; connections are reused only by the role that opened them, and maximum pods × 3T stays within the database's connection limit. |
+| [0023: A daily security check instead of a sweep in every deployment](api/0023-daily-security-check.md) | Accepted; implemented on development; deployment and first scheduled run pending | A deployment sets up only what the new release needs; a daily Job, which administrators can also start, repairs whatever differs from the plan without pausing caller SQL and records its outcome. |
 
 ## Agents
 

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/api/openapi.json

@@ -7709,6 +7709,26 @@
         ]
       }
     },
+    "/security/checks/": {
+      "post": {
+        "description": "Run the full security check now; caller SQL keeps running (ADR 0023).\n\nHosted starts the daily check Job and answers 202; its outcome appears in the\npermission state when it finishes. Local runs the check in this request.",
+        "operationId": "run_security_check_now_security_checks__post",
+        "responses": {
+          "200": {
+            "content": {
+              "application/json": {
+                "schema": {}
+              }
+            },
+            "description": "Successful Response"
+          }
+        },
+        "summary": "Run Security Check Now",
+        "tags": [
+          "Security"
+        ]
+      }
+    },
     "/security/database-permissions/": {
       "get": {
         "description": "Read durable permission/admission state without resolving Secrets or repairing grants.",
@@ -7811,26 +7831,6 @@
         ]
       }
     },
-    "/security/reconcile/": {
-      "post": {
-        "description": "Repair native grants; a failed repair leaves SQL admission disabled.",
-        "operationId": "reconcile_database_permissions_security_reconcile__post",
-        "responses": {
-          "200": {
-            "content": {
-              "application/json": {
-                "schema": {}
-              }
-            },
-            "description": "Successful Response"
-          }
-        },
-        "summary": "Reconcile Database Permissions",
-        "tags": [
-          "Security"
-        ]
-      }
-    },
     "/security/resources/": {
       "get": {
         "operationId": "security_resources_security_resources__get",

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/api/reference.md

@@ -63,10 +63,10 @@ These methods and paths are mounted by the current application. See the [OpenAPI
 | GET | `/runtime-context/` | Runtime Context | 200 |
 | POST | `/runtime-migrations/{lease_uid}/release/` | Release Migration | 200 |
 | GET | `/security/access-history/` | Security History | 200 |
+| POST | `/security/checks/` | Run Security Check Now | 200 |
 | GET | `/security/database-permissions/` | Database Permission Status | 200 |
 | POST | `/security/namespaces/` | Create Namespace | 201 |
 | GET | `/security/principals/` | Principal Search | 200 |
-| POST | `/security/reconcile/` | Reconcile Database Permissions | 200 |
 | GET | `/security/resources/` | Security Resources | 200 |
 | GET | `/table-update-runs/` | List Runs | 200 |
 | POST | `/table-update-runs/` | Create Update Run | 201 |

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/contributing/releasing.md

@@ -31,6 +31,16 @@ Development releases are selected explicitly:
 python -m pip install --pre mainsequence-metatable
 ```
 
+## Access rule changes
+
+A deployment does not apply a change to how MetaTables computes access; the daily
+security check, or an administrator running it, does
+([ADR 0023](../adr/api/0023-daily-security-check.md)). Access rules are `plan()`, the
+table policy checks, the grant and revoke statements, routine securing and the owner
+role in `src/metatables/api/backend/backends/`. Label a pull request that changes them
+`access-rules`. `.github/release.yml` lists such pull requests under "Access rule
+changes: run the security check after deploying" in the generated release notes.
+
 ## Release prerequisites
 
 MetaTables and the Main Sequence SDK use the MIT License. Package metadata

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/operations/hosted-runtime.md

@@ -144,8 +144,10 @@ the platform grants it on push. It:
    database or ownership of the `metatables` schema, and `CREATE` on the default schema;
 2. applies the system migrations;
 3. registers the runtime DataSource, or updates its connection if the declaration changed;
-4. reapplies database access setup. Per-User role passwords derive from the login
-   password, so a rotated password reaches them.
+4. sets up database access as far as the new release needs: the owner role, login
+   passwords again after a rotation (per-User role passwords derive from the login
+   password), and pending changes. It runs the full security check only on first setup
+   or after system migrations; the daily security check covers the rest.
 
 Any failure blocks the rollout. The Job prints `status` (`initialized`, `upgraded`
 or `up_to_date`), `previous_revisions`, `revisions` and `data_source_uid`. A new
@@ -157,6 +159,25 @@ database server's SQLSTATE and message when the server answered. The
 log, with passwords, tokens and connection-string credentials redacted. Settings
 and API errors never carry driver text, which can include hosts and connection strings.
 
+## Daily security check
+
+The `check-security` Job (`jobs/check_security.py`, exactly
+`metatables runtime check-security`) runs daily at 03:00 UTC
+([ADR 0023](../adr/api/0023-daily-security-check.md)). It reads `configuration.yaml`
+and the Secret like the migration Job, waits for a running migration, and skips while
+the database still awaits its image's system migrations. It compares everything
+MetaTables secures with the catalog's grants, repairs what differs, and records the
+outcome, which the Admin's Database permissions panel shows. Caller SQL keeps running.
+A failed check fails the Job run, changes nothing, and is recorded with its error code.
+
+Run it from the Admin with **Run security check now**:
+
+- after deploying a release whose notes list access rule changes;
+- after changing grants, owners or roles directly in the database.
+
+The API starts the Job (its workload has `edit` on `check-security`); a second request
+while a run is pending or running is refused.
+
 ## API pods and Settings
 
 API pods read the same declaration and Secret at startup and never run DDL. They

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/operations/recovery-and-observability.md

@@ -17,6 +17,16 @@ per-table finalization result, physical schema, provider identity, and registry
 state. Reuse only a compatible registry resolved by physical identity. Preserve
 Alembic's applied history; write new revisions for further changes.
 
+## Failed security check
+
+The daily security check ([ADR 0023](../adr/api/0023-daily-security-check.md)) records
+a failure on the Admin's Database permissions panel with an error code, and its Job run
+fails. Nothing it changed is kept, and caller SQL keeps running with the grants it
+found. Read the `check-security` Job run log for the cause, fix it in the database, then
+**Run security check now**. `sql_principal_permission_drift` means a MetaTables role
+owns an object or holds an elevated attribute; reassign that object or remove the
+attribute, because the check never does it.
+
 ## Deletion
 
 Ordinary delete requires edit access and respects table and inbound

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/reference/capabilities.md

@@ -310,4 +310,5 @@ This inventory records the support status of public resource methods and CLI com
 | `runtime status` | tested | visible | Read bootstrap and effective runtime state before or after initialization. |
 | `runtime initialize` | tested | visible | Explicit Local API system migrations under current admin admission; Hosted answers 409 (the deployment migrates). |
 | `runtime upgrade` | tested | visible | Deployment gate: verify, migrate and register the runtime database declared in configuration.yaml and its Environment Secret. |
+| `runtime check-security` | tested | visible | The daily security check Job (ADR 0023): compare everything MetaTables secures with the plan, repair what differs and record the outcome; the Admin starts it with --trigger admin. |
 | `admin repair` | tested | visible | Explicitly stage pinned source/dependencies; refuse replacement while in use. |

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/reference/cli.md

@@ -26,6 +26,7 @@ The list below contains visible leaf commands. Hidden aliases and unavailable co
 | `metatables migrations revision` | --message/-m, --autogenerate, --provider, --rev-id, --head, --timeout, --ttl-seconds, --sqlalchemy-url, --json | Create a normal Alembic revision for the selected provider. |
 | `metatables migrations scaffold` | --package, --namespace, --metadata, --module, --code-repository-root, --source-root, --base, --models, --alembic-version-name, --alembic-version-identifier, --alembic-version-schema, --alembic-version-table-name, --force, --json | Create a MetaTables migration package skeleton. |
 | `metatables migrations upgrade` | target_revision, --provider, --timeout, --ttl-seconds, --json | Run the application's Alembic upgrade and finalize its catalog tables. |
+| `metatables runtime check-security` | --trigger | Run the full database security check on the runtime database this deployment declares. |
 | `metatables runtime initialize` |  | Explicitly apply a Local API's system migrations; hosted deployments migrate their declared database. |
 | `metatables runtime status` |  | Read bootstrap state and effective runtime, including before initialization. |
 | `metatables runtime upgrade` |  | Verify, migrate and register the runtime database this deployment declares, before a rollout. |

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/security/index.md

@@ -148,15 +148,25 @@ grants. MySQL records durable pending work because its permission statements com
 independently; it admits queries only after reconciliation succeeds.
 
 The Admin Security page reads `GET /security/database-permissions/` to show caller
-SQL admission, pending synchronization, and policy coverage for active tables.
+SQL admission, pending synchronization, policy coverage for active tables, and the
+last security check.
 This admin-only read does not repair permissions or resolve credential Secrets.
 It reports durable catalog state; it does not audit native grants changed outside
 MetaTables. SQLite uses a connection authorizer rather than database roles.
 
-**Repair database permissions** on the Admin Security page re-establishes native
-permissions from catalog grants and rechecks physical safeguards. The corresponding
-admin-only endpoint is `POST /security/reconcile/`. Queries pause during repair.
-If permission repair fails, caller SQL stays disabled until repair succeeds.
+A security check compares everything MetaTables secures in the database with the
+catalog's grants and repairs what differs, including grants changed directly in the
+database. It runs daily at 03:00 UTC as the `check-security` Job of a hosted API's
+workflow ([ADR 0023](../adr/api/0023-daily-security-check.md)). **Run security check
+now** on the Admin Security page runs it at any time; the admin-only endpoint is
+`POST /security/checks/`. Hosted starts the Job, Local runs the check in the request.
+Queries keep running during a check, and a failed check changes nothing. The page
+shows the last check: what started it, when, and whether it passed, repaired
+something or failed with an error code.
+
+Grants changed outside MetaTables are repaired within a day. A release that changes
+how MetaTables computes access reaches the database at the next check; its release
+notes say to run the check after deploying.
 Client migration finalization refreshes the affected catalog contracts and SQL policies. Grant editing never requires a platform authentication change.
 
 The API cancels work at its deadline and bounds fetched rows. PostgreSQL, MySQL

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/docs/security/permissions.md

@@ -16,6 +16,7 @@ Organization admin status comes from the platform through the SDK.
 | Configure, validate, disable, or remove DataSources | No | Yes |
 | Select the runtime DataSource where supported | No | Yes |
 | Initialize or upgrade MetaTables system tables in Local Settings (Hosted: the deployment's migration Job) | No | Yes |
+| Run the database security check now (Hosted: it also runs daily) | No | Yes |
 | Destroy a local runtime database through Settings | No | Yes |
 | Change application-wide Settings and security policies | No | Yes |
 | Manage grants across the application and recover orphaned tables | No | Yes |

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/mkdocs.yml

@@ -86,6 +86,7 @@ nav:
           - "0017: Hosted request identity": adr/api/0017-hosted-request-identity.md
           - "0018: Table search": adr/api/0018-table-search.md
           - "0022: Concurrent requests and connection reuse": adr/api/0022-concurrent-requests-and-connection-reuse.md
+          - "0023: Daily security check": adr/api/0023-daily-security-check.md
       - Agents:
           - "0019: MetaTables Analyst agent": adr/agent/0019-metatables-analyst-agent.md
           - "0020: The Analyst reads as its requester": adr/agent/0020-analyst-reads-as-its-requester.md

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mainsequence-metatable"
-version = "0.1.36.dev66"
+version = "0.1.36.dev67"
 description = "Main Sequence MetaTables Python client and standalone API"
 readme = "README.md"
 requires-python = ">=3.13"

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/src/mainsequence_metatable.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.36.dev66
+Version: 0.1.36.dev67
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/src/mainsequence_metatable.egg-info/SOURCES.txt

@@ -46,6 +46,7 @@ docs/adr/api/0017-hosted-request-identity.md
 docs/adr/api/0018-search-document-pipeline.svg
 docs/adr/api/0018-table-search.md
 docs/adr/api/0022-concurrent-requests-and-connection-reuse.md
+docs/adr/api/0023-daily-security-check.md
 docs/adr/client/0003-api-endpoint-resolution.md
 docs/adr/client/0011-cli-local-development-with-managed-admin.md
 docs/api/authentication-and-errors.md
@@ -312,6 +313,7 @@ src/metatables/api/backend/integrations/platform_principals.py
 src/metatables/api/backend/integrations/run_logs.py
 src/metatables/api/backend/integrations/runtime_binding.py
 src/metatables/api/backend/integrations/runtime_data_sources.py
+src/metatables/api/backend/integrations/security_check_job.py
 src/metatables/api/backend/integrations/source_validation.py
 src/metatables/api/backend/migrations/__init__.py
 src/metatables/api/backend/migrations/env.py
@@ -328,6 +330,7 @@ src/metatables/api/backend/migrations/versions/0009_timescale_policies.py
 src/metatables/api/backend/migrations/versions/0010_cascade_targets.py
 src/metatables/api/backend/migrations/versions/0011_table_search.py
 src/metatables/api/backend/migrations/versions/0012_table_creator.py
+src/metatables/api/backend/migrations/versions/0013_security_check.py
 src/metatables/api/backend/migrations/versions/__init__.py
 src/metatables/api/backend/operations/__init__.py
 src/metatables/api/backend/operations/bootstrap_migrations.py
@@ -339,6 +342,7 @@ src/metatables/api/backend/operations/import_relations.py
 src/metatables/api/backend/operations/introspect_table.py
 src/metatables/api/backend/operations/read_time_index.py
 src/metatables/api/backend/operations/register_catalog.py
+src/metatables/api/backend/operations/security_check.py
 src/metatables/api/backend/persistence/__init__.py
 src/metatables/api/backend/persistence/cascade.py
 src/metatables/api/backend/persistence/catalog.py
@@ -583,6 +587,7 @@ tests/permissions/test_local_actor_membership.py
 tests/permissions/test_migration_boundary.py
 tests/permissions/test_namespace_grants.py
 tests/permissions/test_security_api.py
+tests/permissions/test_security_check.py
 tests/permissions/test_table_body_filter.py
 tests/permissions/test_table_grants.py
 tests/permissions/test_table_queries.py

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/src/metatables/api/app/bootstrap.py

@@ -445,7 +445,8 @@ class RuntimeBootstrap:
                         from metatables.api.backend.backends.registry import backend_for
                         session.flush()
                     with bootstrap_phase("security_reconciliation"):
-                        backend_for(replace(self.backend.source, uid=row.uid)).access.initialize(session)
+                        backend_for(replace(self.backend.source, uid=row.uid)).access.initialize(
+                            session, revisions=self.expected)
                     with bootstrap_phase("credential_store"):
                         credentials = bind_store(self.credential_store, session, user_uid)
                         if hasattr(credentials, "initialize"):
@@ -584,8 +585,9 @@ def deploy_runtime_database(user_uid):
     """The deployment gate: verify, migrate and register the declared runtime database.
 
     The workflow's migration Job runs this from the candidate image before the
-    API rolls out, every time; any failure blocks the rollout. Access setup is
-    freshly checked too; drift and rotated credentials are repaired as needed.
+    API rolls out, every time; any failure blocks the rollout. It sets up access
+    only as far as the new release needs (a rotated password, pending changes, a
+    full check after system migrations); outside drift is the daily check's.
     """
     from metatables.api.app.configuration import DEPLOYMENT_CONFIGURATION
     from metatables.api.app.runtime import RuntimeConfig
@@ -605,6 +607,41 @@ def deploy_runtime_database(user_uid):
         target.close()
 
 
+def check_runtime_security(trigger="schedule"):
+    """The daily security check Job, which the Admin can also start (ADR 0023).
+
+    It holds the bootstrap lock, so it waits for a running migration, and skips
+    while the declared database still awaits this image's migrations: a run that
+    lands mid-rollout never checks one release's catalog with another's rules.
+    Any other state that is not ready fails the run.
+    """
+    from metatables.api.app.configuration import DEPLOYMENT_CONFIGURATION
+    from metatables.api.app.runtime import RuntimeConfig
+    from metatables.api.backend.backends.registry import backend_for
+    from metatables.api.backend.operations.security_check import run_security_check
+    target = RuntimeBootstrap(RuntimeConfig(local_mode=False, configuration_path=DEPLOYMENT_CONFIGURATION), None)
+    try:
+        candidate = target.initial_candidate()
+        if candidate is None:
+            raise BootstrapError(target.error)
+        target.configure(candidate, None, validate_schema=False, inspect_database=False)
+        engine = target.open_engine()
+        if engine is None:
+            return dict(status="skipped", reason="migration_required")
+        with target.backend.catalog.bootstrap_lock(engine):
+            target.inspect(validate_schema=False)
+            if target.status == "migration_required":
+                return dict(status="skipped", reason="migration_required")
+            if target.status != "ready":
+                raise BootstrapError(target.error or f"The runtime database is {target.status}.")
+            access = backend_for(replace(target.backend.source, uid=target.uid)).access
+            with bootstrap_phase("security_check"), target.factory() as session:
+                record = run_security_check(session, access, trigger=trigger, revisions=target.current)
+        return dict(status=record["outcome"], statements=record["statements"], revisions=sorted(target.current))
+    finally:
+        target.close()
+
+
 router = APIRouter(prefix="/runtime-bootstrap", tags=["Runtime bootstrap"])
 
 

{mainsequence_metatable-0.1.36.dev66 → mainsequence_metatable-0.1.36.dev67}/src/metatables/api/app/routes/security.py

@@ -2,7 +2,7 @@
 from hashlib import sha256
 from uuid import UUID
 
-from fastapi import APIRouter, Depends, HTTPException, Query
+from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response
 from pydantic import BaseModel, ConfigDict, Field
 from sqlalchemy import select
 from sqlalchemy.orm import Session
@@ -52,6 +52,7 @@ def database_permission_status(actor=Depends(require_admin_actor), session: Sess
     return dict(status=status, sql_enabled=ready, synchronization_pending=pending,
         data_source_uid=str(source.uid), data_source_name=source.display_name, engine=source.class_type,
         enforcement="sqlite_authorizer" if source.class_type == "sqlite" else "database_roles",
+        last_check=state.last_check if state else None,
         active_tables=len(tables), covered_tables=len(tables) - missing, missing_policies=missing,
         tables=[dict(uid=str(table.uid), name=table.physical_table_name,
             schema=policies[table.uid].schema_name if table.uid in policies else table.physical_schema,
@@ -59,28 +60,37 @@ def database_permission_status(actor=Depends(require_admin_actor), session: Sess
             writes_supported=bool(table.uid in policies and policies[table.uid].writable)) for table in tables])
 
 
-@router.post("/security/reconcile/")
-def reconcile_database_permissions(actor=Depends(require_admin_actor),
-        session: Session = Depends(catalog_session), sources=Depends(data_source_provider)):
-    """Repair native grants; a failed repair leaves SQL admission disabled."""
+def security_check_starter():
+    """Starts the daily security check Job (ADR 0023); tests replace it."""
+    from metatables.api.backend.integrations.security_check_job import start_security_check_job
+    return start_security_check_job
+
+
+@router.post("/security/checks/")
+def run_security_check_now(request: Request, response: Response, actor=Depends(require_admin_actor),
+        session: Session = Depends(catalog_session), sources=Depends(data_source_provider),
+        start_job=Depends(security_check_starter)):
+    """Run the full security check now; caller SQL keeps running (ADR 0023).
+
+    Hosted starts the daily check Job and answers 202; its outcome appears in the
+    permission state when it finishes. Local runs the check in this request.
+    """
+    from metatables.api.backend.integrations.security_check_job import SecurityCheckUnavailable
+    if not request.app.state.runtime.config.local_mode:
+        try:
+            started = start_job()
+        except SecurityCheckUnavailable as exc:
+            raise HTTPException(exc.status_code, exc.detail) from exc
+        response.status_code = 202
+        return {"status": "started", **started}
     from metatables.api.backend.backends.registry import backend_for
-    from metatables.api.backend.persistence.models import SQLSecurityState
+    from metatables.api.backend.operations.security_check import run_security_check
     backend = backend_for(sources.get(sources.default_uid()))
-    lock_catalog(session)
-    state = session.get(SQLSecurityState, 1)
-    if state is None:
-        state = SQLSecurityState(id=1, ready=False, pending=True, manifest={})
-        session.add(state)
-    state.ready = False
-    session.info["sql_security_maintenance"] = True
-    session.commit()
     try:
-        backend.access.initialize(session)
-        session.commit()
+        record = run_security_check(session, backend.access, trigger="admin")
     except Exception as exc:
-        session.rollback()
-        raise HTTPException(503, "Database permission reconciliation failed; caller SQL remains disabled.") from exc
-    return {"ok": True, "data_source_uid": str(backend.source.uid)}
+        raise HTTPException(503, "The security check failed; its error code is in the permission state.") from exc
+    return {"status": "finished", "last_check": record}
 
 
 class Principals(BaseModel):