mainsequence 4.3.9__tar.gz → 4.3.18__tar.gz

{mainsequence-4.3.9/mainsequence.egg-info → mainsequence-4.3.18}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mainsequence
-Version: 4.3.9
+Version: 4.3.18
 Summary: Main Sequence SDK 
 Author-email: Main Sequence GmbH <dev@main-sequence.io>
 License: MainSequence GmbH SDK License Agreement

{mainsequence-4.3.9 → mainsequence-4.3.18}/agent_scaffold/skills/command_center/adapter_from_api/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: command-center-adapter-from-api
-description: Use when building or changing an API so Command Center can consume it through an Adapter from API connection. This skill defines the provider-side API contract standards required for Command Center connection discovery, health checks, operation selection, config fields, secret injection metadata, canonical `TabularFrameResponse` outputs, and tabular response mappings. It does not define general API architecture and does not imply the API may only serve these endpoints.
+description: Use when building or changing an API so Command Center can consume it through an Adapter from API connection. This skill defines the provider-side API contract standards required for Command Center connection discovery, health checks, operation selection, config fields, secret injection metadata, canonical `TabularFrameResponse` outputs, and optional response-mapping metadata for frontend/editor context or future explicit transforms. It does not define general API architecture and does not imply the API may only serve these endpoints.
 ---
 
 # Command Center Adapter From API
@@ -13,7 +13,7 @@ connection.
 This skill defines the Command Center-facing contract that the API must expose. It does not own the
 whole API architecture. An API may serve other clients and routes, but if it is consumed by Command
 Center through this adapter, it must also expose the discovery, health, operation, config, secret,
-and response-mapping metadata described here.
+and any optional response-mapping metadata described here.
 
 The runtime flow is:
 
@@ -26,8 +26,12 @@ API with Command Center contract endpoints
 ```
 
 Generic table, chart, statistic, curve, transform, and agent-facing data consumers must consume
-`core.tabular_frame@v1`. API responses that are provider-native, paginated, nested, or domain
-specific need an exact response mapping before they are consumed by generic tabular widgets.
+an actual `core.tabular_frame@v1` payload at the consumption boundary. API responses that are
+provider-native, paginated, nested, or domain specific are still provider-native even when the
+contract includes `responseMappings`. A mapping may describe how a response could be interpreted by
+frontend/editor tooling or a future explicit transform, but the current backend adapter must not
+hot-path validate, extract JSONPath rows, coerce schemas, or reshape provider responses because a
+mapping exists.
 
 ## Scope
 
@@ -39,7 +43,7 @@ This skill owns:
 - operation metadata needed by Connection Query
 - public config variable definitions
 - secret variable definitions and backend injection metadata
-- response mappings for canonical tabular consumption
+- optional response-mapping metadata for frontend/editor context and future explicit transforms
 - API-side validation and maintenance rules for this contract
 
 This skill does not own:
@@ -103,7 +107,7 @@ GET /openapi.json
 ```
 
 This is supplementary metadata. The well-known Command Center contract decides what operations,
-config fields, secrets, and response mappings Command Center may use.
+config fields, secrets, and optional response-mapping metadata Command Center may use.
 
 ### 3. Dedicated Health Endpoint
 
@@ -123,8 +127,9 @@ parameters. Do not use a parameterized data endpoint as a fake health check.
 Each queryable API operation must have a stable `operationId` in the well-known contract and a
 matching route in the API.
 
-Operation endpoints may return provider-native JSON. For generic tabular consumers, the contract
-must declare exactly how that response becomes `core.tabular_frame@v1`.
+Operation endpoints may return a full canonical `core.tabular_frame@v1` payload or provider-native
+JSON. `responseMappings` may document an intended tabular interpretation, but they are not a
+runtime guarantee that provider-native JSON becomes `core.tabular_frame@v1`.
 
 ## Well-Known Contract Shape
 
@@ -185,7 +190,7 @@ Every operation listed in `availableOperations` must define:
 - `supportsMaxRows`
 - `parameters`
 - optional `requestBody`
-- `responseMappings`
+- optional `responseMappings`
 - `cache`
 
 Rules:
@@ -264,8 +269,8 @@ from mainsequence.client.command_center.data_models import TabularFrameResponse
 Declare `response_model=TabularFrameResponse` instead of recreating the canonical frame shape
 locally.
 
-If an operation returns provider-native JSON, declare a `responseMappings` entry that exposes the
-operation as `core.tabular_frame@v1`:
+If an operation returns provider-native JSON, it may declare a `responseMappings` entry that
+describes a tabular interpretation for frontend/editor metadata or future explicit transforms:
 
 ```json
 {
@@ -289,13 +294,23 @@ operation as `core.tabular_frame@v1`:
 
 Rules:
 
-- `contract` must be `core.tabular_frame@v1` for generic tabular consumption.
-- `rowsPath` must point to the array of row objects inside the operation response.
-- `fieldTypes` must cover the fields that consumers need for formatting and inference.
-- Time-series metadata should be present when the rows are meant for charts, curves, or time-aware
+- `responseMappings` are optional metadata in the current Adapter from API flow.
+- A mapping does not make provider-native JSON a `core.tabular_frame@v1` runtime payload.
+- The backend must not hot-path validate the upstream body against `responseMappings`.
+- The backend must not extract JSONPath rows, coerce schemas, or reshape the response merely
+  because a mapping exists.
+- If a mapping describes tabular interpretation, `contract` should be `core.tabular_frame@v1`.
+- If `rowsPath` is present, it should point to the array of row objects inside the operation
+  response.
+- If `fieldTypes` is present, it should cover the fields that frontend/editor tooling or future
+  transforms need for formatting and inference.
+- Time-series metadata is useful when the mapped rows are meant for charts, curves, or time-aware
   transforms.
 - If the response shape changes, update the response mapping in the same change.
-- Do not bind provider-native JSON directly to generic tabular consumers.
+- Do not claim provider-native JSON is safe for generic tabular consumers just because a
+  `responseMappings` entry exists.
+- If generic tabular widgets need to consume the operation directly today, return
+  `TabularFrameResponse` from the API operation.
 
 ## Canonical SDK Model
 
@@ -313,8 +328,9 @@ Related SDK models:
 - `TabularTimeSeriesMetaResponse`
 
 Use these models when the API operation returns the full canonical frame. If the provider operation
-returns provider-native JSON instead, keep the provider response model explicit and declare an exact
-`responseMappings` entry that maps it into `core.tabular_frame@v1`.
+returns provider-native JSON instead, keep the provider response model explicit. Add
+`responseMappings` only as metadata; do not treat it as runtime conversion into
+`core.tabular_frame@v1`.
 
 ## Required Decisions
 
@@ -327,9 +343,10 @@ Before implementing or revising an API for Adapter from API consumption, decide:
 5. Which operation IDs must remain stable?
 6. Which public config fields does the connection need?
 7. Which secrets does the connection need, and how does the backend inject them?
-8. Which operations produce generic tabular consumption?
-9. For each tabular operation, does the API return a full `core.tabular_frame@v1` document or a
-   provider-native response with an exact `responseMappings` entry?
+8. Which operations directly return a full `core.tabular_frame@v1` payload for generic tabular
+   consumption?
+9. For provider-native responses, is optional `responseMappings` metadata useful, and what explicit
+   transform or future feature would consume it?
 10. If the operation returns a full canonical frame, does it use `TabularFrameResponse` as the
     FastAPI `response_model`?
 
@@ -343,7 +360,8 @@ For a FastAPI provider:
 - expose a zero-argument health route
 - expose query operation routes with documented request parameters
 - declare every Command Center-callable operation in `availableOperations`
-- declare `core.tabular_frame@v1` response mappings for tabular consumption
+- declare `responseMappings` only as optional metadata when useful; do not rely on them for runtime
+  normalization
 - use `TabularFrameResponse` when an operation returns the full canonical frame
 - keep the API root, well-known contract, and OpenAPI document internally consistent
 
@@ -358,10 +376,11 @@ When reviewing an Adapter from API provider change, look for:
 - operations missing from `availableOperations`
 - operations exposed to Command Center without explicit allowlist metadata
 - secret values represented as public config or query payload fields
-- tabular responses without `core.tabular_frame@v1` response mappings
-- response mappings whose `rowsPath` no longer matches the API response
-- missing `fieldTypes` for fields consumed by widgets
-- API endpoints returning provider-native JSON directly to generic tabular consumers
+- provider-native responses being treated as `core.tabular_frame@v1` because a mapping exists
+- response mappings whose `rowsPath` no longer matches the API response when the mapping is present
+- missing `fieldTypes` for fields that declared mappings or future transforms need
+- API endpoints returning provider-native JSON directly to generic tabular consumers without an
+  explicit transform path
 - full canonical frame endpoints that do not use `TabularFrameResponse`
 - docs or README examples that disagree with the actual contract endpoint
 
@@ -380,10 +399,12 @@ Do not claim the API is consumable by Adapter from API until:
 - query-capable operations are explicitly marked as query-capable
 - public config and secret fields are separated
 - secret injection rules are backend-owned and explicit
-- each generic tabular operation declares `core.tabular_frame@v1`
+- each operation that must feed generic tabular consumers directly returns `core.tabular_frame@v1`
 - each full canonical frame operation uses `TabularFrameResponse`
-- each provider-native tabular response has an exact `rowsPath`
-- field types and time-series hints are present where consumers need them
+- provider-native responses are not represented as direct generic tabular outputs solely through
+  `responseMappings`
+- optional response mappings, when present, have accurate `rowsPath`, `fieldTypes`, and time-series
+  hints for their intended metadata or transform use
 - the local README documents the Command Center contract endpoints
 - any general FastAPI/API work has also followed `application_surfaces/api_surfaces`
 
@@ -396,8 +417,10 @@ Stop and surface the missing backend task when:
 - operation IDs cannot be made stable
 - auth requirements are unknown
 - secret injection cannot be described without exposing secret values
-- no exact response mapping exists for generic tabular consumption
-- the API response shape cannot be mapped to `core.tabular_frame@v1`
+- a provider-native response is being treated as generic tabular output only because
+  `responseMappings` exists
+- the API response shape must feed generic tabular consumers directly but cannot return
+  `TabularFrameResponse`
 - a full canonical frame endpoint cannot import or use `TabularFrameResponse`
 - backend Adapter from API runtime support is required but not implemented
 

{mainsequence-4.3.9 → mainsequence-4.3.18}/agent_scaffold/skills/command_center/app_components/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: command-center-app-components
-description: Use this skill when the task is about AppComponent widgets in a Main Sequence project. This skill owns AppComponent input contracts, custom forms, form sections and field definitions, and the boundary between AppComponent input contracts and widget-facing output contracts, including requiring generic tabular consumers to receive core.tabular_frame@v1 instead of ad hoc AppComponent JSON. Before changing AppComponent payloads or contracts, verify the target widget in the Command Center registry through the CLI. Source order is strict: registry detail first, SDK client models second, local Main Sequence repository docs/models third only if the first two still leave something unresolved. Main Sequence is platform-first: if an AppComponent depends on a project API, that API must already exist as a FastAPI project resource and have a corresponding FastAPI ResourceRelease before the AppComponent is considered usable from Command Center. Resource and release creation belong to the orchestration-and-releases skill. It does not own workspace layout, generic FastAPI design, or Streamlit dashboards.
+description: Use this skill when the task is about AppComponent widgets in a Main Sequence project. This skill owns AppComponent generated request forms, response-side editable-form and notification UI contracts, dynamic request/response binding contracts, and the boundary between AppComponent responses and widget-facing output contracts, including requiring generic tabular consumers to receive core.tabular_frame@v1 instead of ad hoc AppComponent JSON. Before changing AppComponent payloads or contracts, verify the target widget in the Command Center registry through the CLI. Source order is strict: registry detail first, SDK client models second, local Main Sequence repository docs/models third only if the first two still leave something unresolved. Main Sequence is platform-first: if an AppComponent depends on a project API, that API must already exist as a FastAPI project resource and have a corresponding FastAPI ResourceRelease before the AppComponent is considered usable from Command Center. Resource and release creation belong to the orchestration-and-releases skill. It does not own workspace layout, generic FastAPI design, or Streamlit dashboards.
 ---
 
 # Command Center AppComponents
@@ -11,22 +11,23 @@ Use this skill when the task is about the backend contract behind a Command Cent
 
 This skill is for:
 
-- AppComponent input contracts
-- custom forms
+- AppComponent generated request form contracts
+- response-side editable form and notification contracts
 - field and section definitions
-- deciding when default argument resolution is enough
+- deciding when default request form generation is enough
 - deciding when a widget-facing API response must use an exact SDK contract
 - requiring API-backed AppComponents to depend on deployed FastAPI resources/releases, not local-only API code
 
 ## This Skill Can Do
 
-- decide whether an AppComponent should rely on the default generated form
-- create a custom `EditableFormDefinition`
+- decide whether an AppComponent should rely on the default generated request form
+- return an `EditableFormDefinition` response when the API should render a stateful editable response form
+- return a `NotificationDefinition` response when the API should render banner-style feedback
 - define `FormSectionDefinition`
 - define `FormFieldDefinition`
 - choose the correct `FormFieldKind`
 - define stable `token` values for fields
-- review whether an AppComponent form is too thin or too custom
+- review whether the request form or response UI contract is over- or under-specified
 - separate input contracts from output contracts
 - decide when the API behind an AppComponent must return exact widget-facing response models
 - verify the target widget type in the CLI registry before changing payload or contract logic
@@ -90,8 +91,10 @@ Before changing an AppComponent backend contract, collect or infer:
 - registry detail payload from:
   - `mainsequence cc registered_widget_type detail <WIDGET_ID> --json`
 - what the widget is trying to collect from the user
-- whether the default generated form is already sufficient
-- the fields, sections, and labels the form should expose
+- whether the default generated request form is already sufficient
+- whether the request form needs supported operation-level UI metadata such as `select2` async search
+- whether the response should render as a notification banner, an editable form session, or the generic response viewer
+- the fields, sections, labels, and tokens a response-side editable form should expose
 - whether the output is:
   - a generic API contract
   - an exact widget-facing contract
@@ -118,26 +121,27 @@ If registry detail is not sufficient, and only after checking the SDK client mod
 - `mainsequence/client/command_center/app_component.py`
 - `mainsequence/client/command_center/data_models.py`
 
-If the input contract or output contract is unclear, stop before building the form.
+If the request contract, response UI contract, or widget-facing output contract is unclear, stop before building the AppComponent contract.
 
 ## Required Decisions
 
 For every non-trivial AppComponent task, decide:
 
-1. Is the default generated form sufficient?
-2. If not, what requires a custom `EditableFormDefinition`?
-3. What are the stable field tokens?
-4. What field kinds should the frontend render?
-5. Is the output generic, or does it need an exact widget-facing response contract?
-6. If the AppComponent depends on a project API, does that API already exist as a FastAPI project resource with a FastAPI `ResourceRelease`?
+1. Is the default generated request form sufficient?
+2. If not, can the request-side need be handled by supported OpenAPI UI metadata such as `select2` async search?
+3. Should the response render as generic JSON/form output, a `NotificationDefinition`, or an `EditableFormDefinition` session?
+4. What are the stable field tokens for editable-form responses?
+5. What field kinds should the frontend render?
+6. Is the output generic, or does it need an exact widget-facing response contract?
+7. If the AppComponent depends on a project API, does that API already exist as a FastAPI project resource with a FastAPI `ResourceRelease`?
 
 ## Build Rules
 
-### 1. Default generated form first
+### 1. Default generated request form first
 
-Do not jump to a custom form by default.
+Do not jump to a response-side editable form by default.
 
-If the operation only exposes simple flat arguments with a straightforward shape, let Command Center resolve the form automatically.
+If the operation only exposes simple flat arguments with a straightforward shape, let Command Center resolve the request form automatically from OpenAPI parameters and request body schema.
 
 Typical cases where default generation is enough:
 
@@ -176,9 +180,14 @@ This skill does not create resources or releases. If the FastAPI project resourc
 
 Only return to AppComponent contract work once the backing API deployment surface is real on the platform.
 
-### 2. Use `EditableFormDefinition` only when the form needs to be specialized
+### 2. Use `EditableFormDefinition` only for response-side editable form sessions
 
-Use a custom form when the widget needs:
+`EditableFormDefinition` is a response model. Command Center renders it after the operation returns successfully when the selected operation's primary success response advertises:
+
+- `"x-ui-role": "editable-form"`
+- `"x-ui-widget": "definition-v1"`
+
+Use it when the response should become a stateful editable form session with:
 
 - grouped sections
 - domain-specific labels
@@ -186,7 +195,14 @@ Use a custom form when the widget needs:
 - more control over editable vs read-only behavior
 - stable custom tokens for bindings or draft state
 
-Do not use a custom form just to restate a trivial flat contract.
+Do not use `EditableFormDefinition` to define the pre-submit request form. Request inputs are generated from OpenAPI path, query, header, and JSON body metadata. If the request side needs richer behavior, use supported operation-level UI metadata instead of returning a form definition.
+
+The current supported request-side UI enhancement is:
+
+- `"x-ui-widget": "select2"`
+- `"x-ui-role": "async-select-search"`
+
+This enhancement is resolved from the selected OpenAPI operation metadata and currently targets query-parameter search helpers on that operation.
 
 ### 3. Treat tokens as stable identities
 
@@ -212,7 +228,10 @@ Example:
 
 Keep the boundary clear:
 
-- `EditableFormDefinition` and related form objects describe what the widget should collect
+- OpenAPI parameters and request body schemas describe what the widget should collect before execution
+- operation-level `select2` metadata can specialize supported request-side generated fields
+- `EditableFormDefinition` and related form objects describe a response-side editable form session
+- `NotificationDefinition` describes response-side banner feedback
 - `mainsequence.client.command_center.data_models.TabularFrameResponse` is the SDK canonical model for `core.tabular_frame@v1`
 - other SDK widget data models describe specialized widget-facing API responses when those exist
 
@@ -265,24 +284,37 @@ Do not use AppComponent as a shortcut source node for generic workspace data jus
 operation itself is the intended form-driven action or workflow producing the canonical tabular
 result.
 
-### 6.2 `x-ui-role` is what makes the contract render as richer UI
+### 6.2 `x-ui-role` is what makes supported contracts render as richer UI
 
-For AppComponent contracts, prefer SDK models whose OpenAPI schema carries the explicit UI role markers.
+For AppComponent response contracts, prefer SDK models whose OpenAPI schema carries the explicit UI role markers.
 
 The AppComponent should always try to implement:
 
-- `"x-ui-role": "editable-form"` for input-side contracts
-- `"x-ui-role": "notification"` for response-side contracts
+- `"x-ui-role": "editable-form"` with `"x-ui-widget": "definition-v1"` for response-side editable form sessions
+- `"x-ui-role": "notification"` with `"x-ui-widget": "banner-v1"` for response-side banner feedback
+
+For request-side generated form enhancements, the current supported operation-level contract is:
+
+- `"x-ui-role": "async-select-search"` with `"x-ui-widget": "select2"`
 
 These markers are not cosmetic. They are what tell Command Center to treat the payload as a richer UI contract instead of generic JSON.
 
 That means:
 
-- use `EditableFormDefinition` and related input models when the AppComponent needs a specialized input form
+- use OpenAPI parameters and request bodies for pre-submit inputs
+- use operation-level `select2` metadata when a request field should render as an async search input
+- use `EditableFormDefinition` and related models when the AppComponent response should render as a stateful editable form
 - use `NotificationDefinition` for response-side user feedback when the backend should return a banner-style notification
 - do not handcraft loose dictionaries for these cases when the SDK model already exists
 - keep input and response contracts separate instead of overloading one model to do both jobs
 
+Current frontend resolution details matter:
+
+- generated request forms come from OpenAPI path, query, header, and JSON body metadata
+- request-side `select2` metadata is read from the selected OpenAPI operation
+- response-side editable-form and notification metadata is read from the primary success response schema first, then from the operation metadata
+- placing these response UI markers only on the OpenAPI media-type or response object is not sufficient in the current frontend
+
 ### 6.3 AppComponent bindings are dynamic, port-to-port, and response-shape aware
 
 Treat AppComponent bindings as normal canonical widget bindings, not as a separate AppComponent-only
@@ -369,8 +401,8 @@ When reviewing an AppComponent task, look for:
 
 - inferred or guessed `widget_id` values
 - AppComponent work that skipped `registered_widget_type list/detail`
-- a custom form that should have been auto-generated
-- a flat autogenerated form that should have been specialized
+- a response-side editable form being used where the request form should simply be generated from OpenAPI
+- a flat autogenerated request form that should have used supported operation-level UI metadata
 - unstable or poorly named field tokens
 - wrong `FormFieldKind` choices
 - generic API output being used where an exact widget-facing contract should have been returned
@@ -388,14 +420,16 @@ Do not claim success until you have checked:
   - `mainsequence cc registered_widget_type list --json`
   - `mainsequence cc registered_widget_type detail <WIDGET_ID> --json`
 - registry detail was used as the first source of truth
-- the choice between autogenerated form and custom form is intentional
-- custom forms use `EditableFormDefinition`
+- the choice between autogenerated request form and supported request-side UI metadata is intentional
+- response-side editable forms use `EditableFormDefinition`
+- response-side notifications use `NotificationDefinition`
 - sections and fields are explicit where needed
 - field tokens are stable and meaningful
 - field kinds reflect business meaning
 - input and output contracts are not mixed together
-- input-side AppComponent contracts use `"x-ui-role": "editable-form"` when a specialized form is intended
-- response-side AppComponent contracts use `"x-ui-role": "notification"` when the API is returning user-facing banner feedback
+- request-side AppComponent enhancements use operation-level `"x-ui-role": "async-select-search"` with `"x-ui-widget": "select2"` when async search is intended
+- response-side AppComponent contracts use `"x-ui-role": "editable-form"` with `"x-ui-widget": "definition-v1"` when the API is returning a stateful editable form
+- response-side AppComponent contracts use `"x-ui-role": "notification"` with `"x-ui-widget": "banner-v1"` when the API is returning user-facing banner feedback
 - widget-facing outputs use exact SDK response models when applicable
 - generic tabular consumers receive `core.tabular_frame@v1`
 - AppComponent operations producing full canonical tabular frames use `TabularFrameResponse`
@@ -411,7 +445,8 @@ Do not claim success until you have checked:
 - the task depends on guessed widget behavior without registry verification
 - registry detail is insufficient and the required contract cannot be resolved from Main Sequence docs/models in this repository
 - the task is really about workspace structure instead of AppComponent contracts
-- the form contract is unclear but a custom form is being forced anyway
+- the request form contract is unclear but a response-side editable form is being forced anyway
+- the response UI contract is unclear but `EditableFormDefinition` or `NotificationDefinition` is being forced anyway
 - the widget-facing output contract is unclear and no exact SDK model is available
 - the task is really a generic API design problem rather than an AppComponent problem
 - the AppComponent depends on a project API that does not yet exist as a FastAPI resource and FastAPI `ResourceRelease`

mainsequence-4.3.18/agent_scaffold/skills/data_publishing/meta_table_migrations/SKILL.md

@@ -0,0 +1,102 @@
+---
+name: mainsequence-metatable-migrations
+description: Use this skill when the task is about Main Sequence MetaTable schema evolution, Alembic provider setup, migration scaffold generation, revision creation, migration upgrade/current commands, provider-scoped MetaTable reservation/finalization, or debugging migration lifecycle failures. This skill owns the migration-first workflow for platform-managed MetaTables and should be used instead of direct MetaTable.register() guidance.
+---
+
+# Main Sequence MetaTable Migrations
+
+## Overview
+
+Use this skill for platform-managed MetaTable schema lifecycle work. The
+standard path is migration-first: declare SQLAlchemy MetaTable models, put them
+in a provider, run `mainsequence migrations ...`, and let the SDK reserve,
+migrate, finalize, and bind provider-scoped MetaTables.
+
+Do not recommend direct `MetaTable.register()` or model `.register()` calls for
+platform-managed application tables outside the migration workflow.
+
+## Read First
+
+1. `docs/tutorial/metatable_migrations.md`
+2. `docs/knowledge/meta_tables/migrations.md`
+3. `docs/knowledge/meta_tables/api.md`
+4. `docs/adr/0020-metatable-migration-artifact-registry.md`
+5. `docs/adr/0026-sdk-owned-migration-scaffolding.md`
+
+## Required Decisions
+
+- Which provider module owns the migration stream?
+- Which package and migration namespace identify the provider?
+- Which `AlembicVersionMetaTable` binding points to Alembic's version table?
+- Which SQLAlchemy `MetaData` object is the target metadata?
+- Which provider-scoped MetaTable models belong in `metatable_models`?
+- Does a dynamic provider need `metadata_for_models(...)` instead of full
+  package metadata?
+- Does the project need an `after_register_metatables` catalog hook, and does
+  that hook use `context.metatable_models` and `context.registered_metatables`
+  instead of importing a broader registry?
+
+## Standard Workflow
+
+Use SDK-owned scaffold and helpers when creating a migration package:
+
+```bash
+mainsequence migrations scaffold \
+  --package msm \
+  --module migrations \
+  --namespace mainsequence.examples \
+  --base msm.base:MarketsBase \
+  --metadata msm.base:MarketsBase.metadata
+```
+
+The generated provider should use:
+
+- `build_alembic_version_metatable(...)`
+- `build_metatable_model_registry(...)`
+- `build_metatable_migration_provider(...)`
+- SDK-owned `run_mainsequence_alembic_env(...)`
+- SDK-owned `script.py.mako`
+
+Then use the normal lifecycle:
+
+```bash
+mainsequence migrations current --provider migrations:migration
+mainsequence migrations revision --provider migrations:migration
+mainsequence migrations upgrade --provider migrations:migration head
+```
+
+`revision` writes normal Alembic files. `upgrade` reserves provider MetaTables,
+runs Alembic DDL through the backend-issued migration credential, finalizes
+provider-scoped MetaTable catalog rows, and runs the optional provider hook.
+
+## Rules
+
+- Keep Alembic as the schema migration engine; do not build custom operation
+  lists or fake migration payload formats.
+- Keep provider scope explicit. Do not scan all imported models or installed
+  packages.
+- For one-model or configured dynamic providers, use `metadata_for_models(...)`
+  and a provider-specific `metatable_models` list.
+- Never send or thread request-side `data_source_uid` through migration status
+  or apply flows. Backend migration operations resolve the data source from the
+  registered Alembic version MetaTable UID.
+- Do not create SDK reset/reconcile commands for stale reserved state. If stale
+  reserved state exists, fail clearly and require an explicit backend/admin
+  repair path.
+- Do not write direct backend migration request bodies in examples. Use the CLI
+  and SDK provider APIs.
+- Do not call platform-managed model `.register()` in normal application code.
+  Registration is reserved for the migration workflow.
+
+## Debugging
+
+- If `current` fails before Alembic runs, inspect the provider import path and
+  Alembic version MetaTable binding.
+- If `revision` autogenerate tries to create everything again, the local
+  migration connection cannot see the provider's current physical tables.
+- If `upgrade` fails during prepare, inspect provider model identifiers,
+  physical table names, and reserved/active MetaTable rows.
+- If `upgrade` fails during finalization, inspect whether Alembic created the
+  physical tables for every provider-scoped model.
+- If an after-register hook reports the wrong model count, ensure it uses the
+  provider-scoped context, not a package-global model registry.

{mainsequence-4.3.9 → mainsequence-4.3.18}/agent_scaffold/skills/data_publishing/meta_tables/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: mainsequence-meta-tables
-description: Use this skill when the task is about defining, registering, querying, migrating, or reviewing Main Sequence MetaTables. This skill owns SQLAlchemy table contracts, backend-managed table registration, external table registration, Alembic-based MetaTable migrations, governed compiled SQL operations, foreign keys, indexes, and validation rules. It does not own DataNode producers, API route contracts, scheduling, releases, or sharing policy.
+description: Use this skill when the task is about defining, querying, or reviewing Main Sequence MetaTables. This skill owns SQLAlchemy table contracts, backend-managed table authoring, external table registration, governed compiled SQL operations, foreign keys, indexes, naming, cadence, and validation rules. For Alembic migration lifecycle work, use the mainsequence-metatable-migrations skill. It does not own DataNode producers, API route contracts, scheduling, releases, or sharing policy.
 ---
 
 # Main Sequence MetaTables
@@ -15,12 +15,11 @@ This skill is for schema-driven application tables registered through TS Manager
 
 - define SQLAlchemy/Core or ORM table models for `MetaTable` registration
 - choose `platform_managed` or `external_registered` management mode
-- register platform-managed tables through the model class API
+- declare platform-managed tables for migration-managed registration
 - build registration requests from resolved SQLAlchemy metadata when inspection is useful
 - define indexes and foreign keys in SQLAlchemy metadata for Alembic-owned DDL
 - design governed compiled SQL read and write operations
-- design provider-based Alembic contract evolution for MetaTables
-- run the documented `mainsequence migrations ...` lifecycle for Alembic-backed MetaTable changes
+- route provider-based Alembic contract evolution to the MetaTable migration skill
 - review table contracts for physical-name, namespace, and identifier issues
 - review whether a task should be a `MetaTable` or a `DataNode`
 
@@ -41,6 +40,8 @@ If the user is still in the discovery process and does not yet know what data ex
 
 - discovery-only data inventory before table implementation:
   `.agents/skills/mainsequence/data_access/exploration/SKILL.md`
+- MetaTable migrations:
+  `.agents/skills/mainsequence/data_publishing/meta_table_migrations/SKILL.md`
 - DataNodes:
   `.agents/skills/mainsequence/data_publishing/data_nodes/SKILL.md`
 - APIs and FastAPI:
@@ -55,7 +56,9 @@ If the user is still in the discovery process and does not yet know what data ex
 ## Read First
 
 1. `docs/tutorial/working_with_meta_tables.md`
-2. For migration work, `docs/tutorial/metatable_migrations.md`
+2. For migration work, use
+   `.agents/skills/mainsequence/data_publishing/meta_table_migrations/SKILL.md`
+   and `docs/tutorial/metatable_migrations.md`
 3. `docs/knowledge/meta_tables/index.md`
 4. `docs/knowledge/meta_tables/sqlalchemy.md`
 5. `docs/knowledge/meta_tables/compiled_sql.md`
@@ -111,9 +114,9 @@ creation or deletion.
 The only migration workflow to recommend is the Main Sequence CLI lifecycle:
 
 ```bash
-mainsequence migrations current --provider mainsequence_migrations:migration
-mainsequence migrations revision --provider mainsequence_migrations:migration
-mainsequence migrations upgrade --provider mainsequence_migrations:migration head
+mainsequence migrations current --provider sdk_examples.migrations:migration
+mainsequence migrations revision --provider sdk_examples.migrations:migration
+mainsequence migrations upgrade --provider sdk_examples.migrations:migration head
 ```
 
 ### 1. SQLAlchemy metadata is the authoring source
@@ -279,7 +282,7 @@ both the schema and the table's intention.
 Provider scope:
 
 ```python
-migration = AlembicMetaTableMigration(
+migration = build_metatable_migration_provider(
     ...,
     metatable_models=[Account, Asset],
 )
@@ -304,9 +307,10 @@ asset_meta_table = MetaTable.register(asset_request)
 
 ### 4. Schema changes use Alembic
 
-When doing migration work, first read
-`docs/tutorial/metatable_migrations.md`. That document is the tutorial source
-for the provider-based Alembic lifecycle.
+When doing migration work, use
+`.agents/skills/mainsequence/data_publishing/meta_table_migrations/SKILL.md`
+and read `docs/tutorial/metatable_migrations.md`. The migration skill owns the
+provider-based Alembic lifecycle.
 
 Do not apply in-place contract changes by changing a `PlatformManagedMetaTable`
 SQLAlchemy class and calling normal registration again. Shape-addressed
@@ -323,8 +327,10 @@ create a new Alembic revision on top of the current head.
 For contract evolution, define or update one selected
 `AlembicMetaTableMigration` provider:
 
-- put the provider in `mainsequence_migrations.py:migration` or pass
-  `--provider module.path:migration`
+- create or update a scaffolded package provider with
+  `mainsequence migrations scaffold`
+- pass the selected provider explicitly, for example
+  `--provider sdk_examples.migrations:migration`
 - set `package`, `migration_namespace`, `script_location`, and `target_metadata`
 - set `alembic_registry` to an `AlembicVersionMetaTable` subclass
 - list the post-apply catalog scope in `metatable_models`
@@ -343,14 +349,14 @@ a bare table name. Use `schema_table_name(app, concept, suffix=None)` for the
 physical table and Alembic version table names. Use `suffix` for a namespace or
 variant, for example `schema_table_name("msm", "positions", suffix="broker")`.
 
-Do not ask users to construct backend migration payloads, call low-level
-migration request models, or use SDK helper functions directly. The backend
-request shape is reference material in the tutorial; the user-facing path is:
+Do not ask users to construct backend migration payloads or call low-level
+migration request models. The backend request shape is reference material in
+the tutorial; the user-facing path is:
 
 ```bash
-mainsequence migrations current --provider mainsequence_migrations:migration
-mainsequence migrations revision --provider mainsequence_migrations:migration
-mainsequence migrations upgrade --provider mainsequence_migrations:migration head
+mainsequence migrations current --provider sdk_examples.migrations:migration
+mainsequence migrations revision --provider sdk_examples.migrations:migration
+mainsequence migrations upgrade --provider sdk_examples.migrations:migration head
 ```
 
 All migration commands prepare the provider, reserve provider-scoped