mainsequence-metatable 0.1.8.dev18__tar.gz → 0.1.9.dev19__tar.gz

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/.agents/skills/client/metatables-local-development/SKILL.md

@@ -72,6 +72,8 @@ Do not retry against automatic hosted discovery when a local connection fails.
    Substitute the application's own module. Provider code and revisions are
    installed in the application process; the API requires no provider approval
    or installation. System initialization and application histories remain separate.
+   Running the provider yourself is for the local runtime only; hosted runtimes
+   receive the same revisions from the deployment workflow's migration Job.
 5. Build application contracts/queries with the
    [table skill](../metatables-meta-tables/SKILL.md), or producers/readers with the
    [updater skill](../metatables-time-index-table-updates/SKILL.md). Use small,
@@ -134,8 +136,12 @@ integration-test fixture.
 ## Finish verification and switch back
 
 Record what passed, the runtime/DataSource used, and any engine-specific work still
-unverified. Carry forward the reviewed application code and migration revisions;
+unverified. Commit the reviewed application code and its migration revisions together;
 local rows, catalog UIDs, credentials and fixture data are not promoted by a switch.
+The application's deployment workflow applies those revisions to the hosted runtime
+before the code rolls out; see the [migrations skill](../metatables-migrations/SKILL.md)
+and `docs/client/deploy-application-migrations.md`. Add the migration Job if the
+workflow lacks one.
 
 When returning to the environment is part of the requested workflow:
 
@@ -155,9 +161,11 @@ When returning to the environment is part of the requested workflow:
    report the actual mode instead of claiming the switch succeeded.
 3. Restart application/test client processes and resolve catalog bindings again.
    Do not reuse local table UIDs or already-bound updater instances against the
-   environment database. Apply reviewed revisions or perform environment writes
-   only within the user's requested scope; passing local tests is not itself a
-   request to seed fixtures, run backfills or migrate that database.
+   environment database. Do not apply application revisions from this session;
+   the deployment's migration Job does. `migrations current` can confirm the
+   deployed revision. Perform other environment writes only within the user's
+   requested scope; passing local tests is not itself a request to seed fixtures
+   or run backfills.
 
 If the user instead wants to connect to a separately deployed API, follow the
 installation guide's hosted endpoint selection. Restore prior connection settings

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/.agents/skills/client/metatables-migrations/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: metatables-migrations
-description: "Create and evolve managed application tables through Alembic migration providers with the installed MetaTables Python client. Covers provider scope and placement, scaffolding, revision authoring, client-side execution, and reservation or finalization failures. Excludes API catalog migrations, client-library implementation, API/server changes and repository tooling."
+description: "Create and evolve managed application tables through Alembic migration providers with the installed MetaTables Python client. Covers provider scope and placement, scaffolding, revision authoring, local execution, the deployment-workflow migration Job for hosted runtimes, and reservation or finalization failures. Excludes API catalog migrations, client-library implementation, API/server changes and repository tooling."
 ---
 
 # MetaTables client application migrations
@@ -39,6 +39,34 @@ hosted API's deployment applies them before it rolls out, and Local mode runs th
 from Settings; see `docs/operations/catalog-migrations.md`. They are not
 application-provider revisions.
 
+## Recommended workflow
+
+Follow this sequence, which mirrors how the MetaTables API deploys its own system
+migrations. Read `docs/client/deploy-application-migrations.md`.
+
+1. **Develop locally.** Author the revision and apply it to verified local SQLite
+   with the [local development skill](../metatables-local-development/SKILL.md).
+2. **Commit the revision with the code that needs it.**
+3. **Deploy.** The application's `.mainsequence/workflows/` file declares a
+   migration Job that calls `upgrade_application` for each provider, in
+   dependency order, from the candidate image. Every deployed resource that reads
+   or writes the tables (FastAPI, `harness_agent` and others) has a deploy step
+   with `needs: [migrate]`. A failed migration blocks the rollout.
+
+Rules:
+
+- Hosted runtimes are migrated only by that Job. Do not run `migrations upgrade`
+  or `downgrade` against a hosted API from a developer or agent session. If the
+  user explicitly requests it, do it once and recommend adding the Job.
+- When the application has no migration Job yet, add one alongside its first
+  managed table or its MetaTables upgrade.
+- Do not migrate at application startup: several pods would race, and a failure
+  would not stop the rollout.
+- Keep each revision usable by the release currently deployed, which keeps serving
+  during and after a failed rollout. Add first; drop or rename in a later release.
+- Redeploying an older image does not roll back the schema. Downgrade is never a
+  deployment step.
+
 ## Source and runtime context
 
 - The API selects its active runtime DataSource. Upgrade and downgrade use that
@@ -113,7 +141,8 @@ the wrong revisions.
 
 ## Execute with the environment connection
 
-After verifying the intended runtime, use the application's provider reference:
+During development, after verifying the local runtime, use the application's
+provider reference:
 
 ```bash
 metatables --local migrations upgrade --provider ledger.migrations:migration
@@ -132,9 +161,9 @@ API Writer checks govern connection admission and provider catalog operations.
 The client reserves catalog entries, runs Alembic, closes its physical connection,
 and finalizes contracts through the API. Treat connection material as private.
 
-Use the application's setup entry point when available. Python code can call
-`metatables.upgrade_application("ledger.migrations:migration")`;
-`src/metatables/examples/scripts/setup_metatables.py` shows the pattern. Remove the retired
+Python code can call `metatables.upgrade_application("ledger.migrations:migration")`;
+`src/metatables/examples/scripts/setup_metatables.py` shows the pattern, and the
+deployment migration Job uses the same call. Remove the retired
 `application_migration_providers` setting and use Python references instead of aliases.
 Application migration histories remain separate from API system migrations.
 
@@ -181,5 +210,6 @@ Read `docs/operations/recovery-and-observability.md`.
 ## Validation
 
 Verify provider scope and placement, revision content, the selected environment
-connection, repeat execution, and final active bindings. Report separately what was checked
-offline and what was exercised against a configured API.
+connection, repeat execution, and final active bindings. Check that the deployment
+workflow runs the migration Job before every resource that uses the tables. Report
+separately what was checked offline and what was exercised against a configured API.

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/.agents/skills/client/metatables-upgrade-legacy-app/SKILL.md

@@ -96,6 +96,9 @@ Use the [application migrations skill](../metatables-migrations/SKILL.md),
 `docs/client/define-and-migrate-tables.md`, and
 `docs/adr/api/0013-application-owned-migrations.md` for that workflow. Ordinary
 SQL continues through the governed API, independently of the migration connection.
+If the application's deployment workflow does not yet run its providers in a
+migration Job before its API and agents roll out, add one as
+`docs/client/deploy-application-migrations.md` describes.
 
 ## Verify the upgrade
 

{mainsequence_metatable-0.1.8.dev18/src/mainsequence_metatable.egg-info → mainsequence_metatable-0.1.9.dev19}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.8.dev18
+Version: 0.1.9.dev19
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/docs/client/define-and-migrate-tables.md

@@ -93,6 +93,11 @@ Python setup code can call `metatables.upgrade_application("ledger.migrations:mi
 For current and downgrade, use `metatables.migrations.runner.run_migration` with
 `operation="current"` or `operation="downgrade", revision="0001"`.
 
+Hosted runtimes are migrated by the application's deployment workflow, not from a
+developer machine: a Job applies the providers from the candidate image before
+the application rolls out. See
+[Migrate in the deployment workflow](deploy-application-migrations.md).
+
 The provider is installed only in the application process. Remove the retired
 `application_migration_providers` setting from existing deployment configuration;
 replace old aliases with application Python references. No API provider registration,

mainsequence_metatable-0.1.9.dev19/docs/client/deploy-application-migrations.md

@@ -0,0 +1,103 @@
+# Migrate in the deployment workflow
+
+An application's tables must match the code that serves them. Apply application
+migrations to a hosted runtime the same way MetaTables applies its own system
+migrations: a Job in the deployment workflow runs them from the candidate image
+before anything that uses the tables rolls out. See
+[catalog migrations](../operations/catalog-migrations.md) for the API's own
+`migrate-system` Job.
+
+## Recommended workflow
+
+1. **Develop locally.** Author the revision and apply it to verified local SQLite
+   with `metatables --local migrations upgrade`. See
+   [Managed tables and migrations](define-and-migrate-tables.md) and
+   [Local storage](local-storage.md).
+2. **Commit the revision with the code that needs it.** Applied revisions are
+   immutable; every later schema change is a new revision.
+3. **Deploy.** The application's workflow runs its migration Job from the
+   candidate image, then deploys the resources that read or write the tables.
+
+Hosted runtimes are migrated only by step 3. Do not run `metatables migrations
+upgrade` or `downgrade` against a hosted API from a developer machine or agent
+session. Do not migrate at application startup either: several pods would race,
+and a failure would not stop the rollout.
+
+## Declare the migration Job
+
+Add one Job script that applies every provider the application owns, in
+dependency order:
+
+```python
+"""Main Sequence Job: apply the ledger migrations before the application rolls out."""
+
+from metatables import upgrade_application
+
+for provider in ("ledger.migrations:migration",):
+    result = upgrade_application(provider)
+    state = "migrated" if result["migrated"] else "already current"
+    print(f"{provider}: {result['revision']} ({state}) on {result['data_source_uid']}", flush=True)
+```
+
+Gate the deployments on it in `.mainsequence/workflows/<application>.yaml`:
+
+```yaml
+api_version: "2.3.0"
+name: ledger
+resources:
+  - key: ledger-api
+    kind: fastapi
+    spec:
+      source_path: api/main.py
+      automatic_deployment: true
+      automatic_redeployment:
+        enabled: true
+  - key: migrate-ledger
+    kind: job
+    spec:
+      name: Ledger migrations
+      execution_path: jobs/migrate_ledger.py
+      automatic_redeployment:
+        enabled: true
+
+execution:
+  steps:
+    image:
+      prepare_image: ledger-api
+    migrate:
+      run_job: migrate-ledger
+      image_from: image
+      needs: [image]
+    deploy_api:
+      deploy: ledger-api
+      image_from: image
+      needs: [migrate]
+```
+
+Give every other deployed resource that reads or writes the tables, such as a
+`harness_agent`, its own deploy step with `needs: [migrate]`. Validate the file
+against the branch's workflow template before committing it.
+
+## What the Job guarantees
+
+- It runs on every deployment. A database already at head is left unchanged and
+  reports `already current`.
+- An exception fails the Job and blocks the rollout; the previous release keeps
+  serving. Fix the revision and deploy again. If DDL committed but finalization
+  failed, follow [Evolve and recover](define-and-migrate-tables.md#evolve-and-recover)
+  before retrying.
+- The Job finds the Environment's MetaTables API the same way any client does
+  ([installation and connection](installation-and-connection.md)). That API must
+  be deployed with its system migrations applied; otherwise the Job fails. Its
+  SDK user needs the same API access a developer needs to run the provider.
+
+## Keep each release compatible with the previous one
+
+The Job runs before the rollout, so the previous release serves against the new
+schema while the rollout proceeds, and keeps doing so if it fails. Write
+revisions that the deployed release can still use: add tables, and nullable or
+defaulted columns, first; drop or rename in a later release, once no deployed
+code uses the old shape.
+
+Redeploying an older image does not roll back the schema. A hosted `downgrade` is
+a separate, explicitly requested operation, never a deployment step.

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/mkdocs.yml

@@ -26,6 +26,7 @@ nav:
       - First API request: client/quickstart.md
       - Market-data tutorial: client/tutorial.md
       - Managed tables and migrations: client/define-and-migrate-tables.md
+      - Migrate in the deployment workflow: client/deploy-application-migrations.md
       - External tables: client/register-existing-tables.md
       - Queries and mutations: client/query-and-mutate.md
       - Updaters: client/build-an-updater.md

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mainsequence-metatable"
-version = "0.1.8.dev18"
+version = "0.1.9.dev19"
 description = "Main Sequence MetaTables Python client and standalone API"
 readme = "README.md"
 requires-python = ">=3.13"

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19/src/mainsequence_metatable.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence-metatable
-Version: 0.1.8.dev18
+Version: 0.1.9.dev19
 Summary: Main Sequence MetaTables Python client and standalone API
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License-Expression: MIT

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/src/mainsequence_metatable.egg-info/SOURCES.txt

@@ -44,6 +44,7 @@ docs/assets/diagrams/local-client-experience.svg
 docs/client/bounded-transfers.md
 docs/client/build-an-updater.md
 docs/client/define-and-migrate-tables.md
+docs/client/deploy-application-migrations.md
 docs/client/installation-and-connection.md
 docs/client/local-storage.md
 docs/client/query-and-mutate.md

{mainsequence_metatable-0.1.8.dev18 → mainsequence_metatable-0.1.9.dev19}/uv.lock

@@ -632,7 +632,7 @@ server = [
 
 [[package]]
 name = "mainsequence-metatable"
-version = "0.1.8"
+version = "0.1.9"
 source = { editable = "." }
 dependencies = [
     { name = "alembic" },