ms-markets 2.2.13.dev37__tar.gz → 2.2.13.dev38__tar.gz

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/CHANGELOG.md

@@ -13,8 +13,81 @@ and this project follows versioned releases.
   proposes the
   Markets API as the application boundary: provider-owned namespaces and
   sidecars, a contracts package with a remote row backend, and contract-plus-
-  instances for portfolio and signal time series catalogued by Markets. It is
-  a proposal; nothing in it is implemented.
+  instances for portfolio and signal time series catalogued by Markets. Its
+  migration path step 1 is implemented in this release as described below;
+  the packaging split (step 2) and the provider and consumer moves (steps 3
+  and 4) are not.
+- `PortfoliosTS`, `PortfolioWeightsTS` and `SignalWeightsTS` are storage
+  contracts with instances (ADR 0047, decision 5).
+  `msm_portfolios.data_nodes.contract_instances` adds
+  `configured_portfolios_storage`, `configured_portfolio_weights_storage` and
+  `configured_signal_weights_storage`, which build an instance under a
+  caller-chosen MetaTables namespace, physical-name app segment and optional
+  SQLAlchemy base, with the contract's columns, dtypes, time index, index
+  names and foreign keys and a stable identifier `<Contract>.<instance_key>`.
+  The core tables remain the Markets-owned instance.
+- `PortfolioDatasetAvailabilityTable` (`PortfolioDatasetAvailability`) catalogs,
+  per portfolio, the values, weights and signal instance uids and the
+  reconciled population state of the values instance, mirroring
+  `IndexDatasetAvailability`. `SignalMetadataTable.meta_table_uid` records the
+  signal's instance. Alembic revision `0021` creates the table and the column;
+  no existing table, column or row changes.
+- `msm_portfolios.services.portfolio_datasets` adds
+  `register_portfolio_datasets`, `reconcile_portfolio_dataset_availability`,
+  `list_portfolio_dataset_states` and `verify_contract_instance`. Registration
+  verifies each declared table against the contract; reconciliation aggregates
+  each instance once for all of its portfolios and records unreadable
+  instances as `unavailable`.
+
+- `apps.v1.identity` adds `acting_identity()`, the one place the Markets API
+  decides whom a request acts for: `User.get_requester()` on a requester-bound
+  call, `User.get_logged_user()` otherwise, never both, and `None` outside a
+  request. `registry_write` wraps every API service that creates, updates or
+  deletes a registry row (accounts, asset categories, assets, calendars,
+  indexes, portfolio groups, portfolio signals, portfolios, pricing curves and
+  pricing market data) and records the acting identity with the resource,
+  action and row uid on the `apps.v1.identity` logger after a successful
+  write. The Index routes derive their actor from the acting identity, so a
+  requester-bound call acts for the requester.
+- Markets API routes the ADR 0047 boundary needs: `POST /api/v1/account/`
+  and `PATCH /api/v1/account/{uid}/` (account create and update over
+  `msm.api.accounts`); `POST /api/v1/index/{uid}/formulas/`,
+  `.../{definition_uid}/activate/`, `.../{definition_uid}/retire/`,
+  `.../formulas/preview/` and `.../{definition_uid}/preview/` (formula save,
+  activate, retire and preview over `FormulaIndex`, previews as
+  `core.tabular_frame@v1`); `GET .../by-uids/?uid__in=` batch lookups for
+  assets, accounts, portfolios and indexes (at most 500 uids, request order,
+  unknown uids omitted, no paging or discovery); `GET /api/v1/portfolio/values/`
+  and `GET /api/v1/portfolio/weights/` multi-portfolio frames grouped by
+  catalogued table; and `POST /api/v1/portfolio/{uid}/datasets/`,
+  `GET /api/v1/portfolio/{uid}/datasets/` and
+  `POST /api/v1/portfolio/datasets/reconcile/` for the portfolio dataset
+  catalog. All are registered in the Command Center adapter contract; the
+  mutating ones record the acting identity. The by-uid lookups accept
+  `include=<sidecar MetaTable uid>` and attach provider sidecar rows under
+  `includes`, resolving the sidecar's foreign key to the core `uid` from the
+  MetaTables catalog and reading each sidecar once (ADR 0047 `include`
+  hydration; the Markets workload needs `data_viewer` on the sidecar).
+- A design note for the contracts package split and `msm.client` (ADR 0047,
+  decision 2) under `docs/planning/implementation_tasks/`; the README now
+  describes `msm.client` as planned rather than shipped.
+- The `markets-api` resource in `.mainsequence/workflows/ms-markets-api.yaml`
+  declares `acts_for_requester: true`, so the platform signs the person an
+  application works for into the caller assertion and the API acts for that
+  requester (ADR 0047, decision 3). Caller-bound calls are unchanged.
+
+### Changed
+
+- `portfolio_values`, `latest_portfolio_weights` and the new `signal_weights`
+  read in `msm_portfolios.services.portfolio_reads` resolve each portfolio's
+  table from the catalog and read it by uid through `TimeIndexTableRef`,
+  filtered by `portfolio_identifier` or `signal_uid`. Portfolios without a
+  catalog row read the core tables as before. A request for several portfolios
+  groups them by table uid, reads each table once and returns one ordered
+  list. `latest_portfolio_weights` gains `order`, and the reads accept a
+  `dataset_reader` for tests. The portfolio values, weights and signal-weight
+  API routes use these reads. The `apps/v1` runtimes attach
+  `PortfolioDatasetAvailability`.
 
 ## [2.2.12] - 2026-10-22
 

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ms-markets
-Version: 2.2.13.dev37
+Version: 2.2.13.dev38
 Summary: Financial markets ORM, engines, and application management extensions for Main Sequence.
 Project-URL: Changelog, https://github.com/mainsequence-projects/MainSequenceMarkets/blob/main/CHANGELOG.md
 Project-URL: Source, https://github.com/mainsequence-projects/MainSequenceMarkets
@@ -334,7 +334,9 @@ Main package areas:
   contracts
 - `msm.constants`: static built-in keys such as asset type constants used across
   applications and examples
-- `msm.client`: client-facing Main Sequence market models and API wrappers
+- `msm.client`: planned typed HTTP client over the Markets API with a remote
+  row backend (ADR 0047, decision 2); not shipped yet. Consumers use `msm.api`
+  today
 - `msm.data_nodes`: market TimeIndexTableUpdater contracts, including account holdings,
   execution events, asset snapshots, and asset pricing details
 - `msm.api.execution`: order managers, target quantities, orders, events,

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/README.md

@@ -56,7 +56,9 @@ Main package areas:
   contracts
 - `msm.constants`: static built-in keys such as asset type constants used across
   applications and examples
-- `msm.client`: client-facing Main Sequence market models and API wrappers
+- `msm.client`: planned typed HTTP client over the Markets API with a remote
+  row backend (ADR 0047, decision 2); not shipped yet. Consumers use `msm.api`
+  today
 - `msm.data_nodes`: market TimeIndexTableUpdater contracts, including account holdings,
   execution events, asset snapshots, and asset pricing details
 - `msm.api.execution`: order managers, target quantities, orders, events,

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/docs/ADR/0047-markets-api-application-boundary.md

@@ -4,11 +4,14 @@
 
 Proposed. Drafted 2026-10-25 from an analysis of this repository, the
 `markets-analytics` API, the `alpaca-connectors` application and the installed
-MetaTables and SDK contracts. Nothing in this ADR is implemented. The
+MetaTables and SDK contracts. The
 `development` branch has since moved to MetaTables 0.1.51, whose namespace and
-table roles (`owner`, `editor`, `data_writer`, `reader`) replace the Reader and
-Writer vocabulary used below; the grant boundaries this ADR draws are
-unchanged by that rename, and implementation must use the role names.
+table roles (`owner`, `editor`, `data_writer`, `data_viewer`, `metadata_viewer`)
+replace the Reader and Writer vocabulary used below; the grant boundaries this
+ADR draws are unchanged by that rename, and implementation uses the role names.
+
+Migration path step 1 is implemented on `development` (see "Implementation
+notes" at the end). Steps 2 to 4 are not.
 
 It builds on [ADR 0008](0008-metatable-table-and-api-model-split.md) (row
 objects separate from table declarations), [ADR 0030](0030-explicit-portfolio-price-source-dependency.md)
@@ -150,8 +153,8 @@ contract, not the table schema, is the compatibility promise.
 
 Every handler and service authorizes against one identity: the requester on a
 requester-bound call, the caller otherwise, never both. The `markets-api`
-resource declares `acts_for_requester: true`, which an Organization admin must
-push. Services enforce domain policy: ownership and Team scoping for
+resource declares `acts_for_requester: true` in its workflow. Services enforce
+domain policy: ownership and Team scoping for
 portfolios, accounts, indexes and formula definitions, and application-level
 permissions such as "this application may register accounts". Writes keep
 running as the Markets workload, whose Writer grant is the single place writes
@@ -318,3 +321,90 @@ Under this ADR:
 - Row-level ownership inside shared tables. MetaTables grants are per table
   by design; this ADR works with that rather than against it.
 - Moving signals, strategies or engines into Markets.
+
+## Implementation notes
+
+### 2026-10-25: step 1, decision 5 (contract plus instances, catalog)
+
+Implemented in `msm_portfolios.data_nodes.contract_instances`,
+`msm.models.portfolio_dataset_availability`, Alembic revision `0021`,
+`msm_portfolios.services.portfolio_datasets` and
+`msm_portfolios.services.portfolio_reads`. Deviations and refinements:
+
+- The catalog row is one per portfolio, carrying the values, weights and
+  signal instance uids together, as this ADR lists them. The reconciled
+  population state, row count and time bounds describe the values instance;
+  weights and signal instances are catalogued but not aggregated. The Index
+  catalog has one row per (Index, dataset) because every cadence table is a
+  dataset in its own right; for a portfolio the values series is the dataset
+  and the other two are its supporting facts.
+- An instance is named `<storage_app>__<contract>__<instance_key>` and
+  identified as `<Contract>.<instance_key>`, namespace-prefixed outside the
+  default Markets namespace. The ADR's `alpaca_connectors__portfoliosts` is
+  therefore `alpaca_connectors__portfoliosts__alpaca`; the key keeps two
+  instances of one provider distinct.
+- Registration verifies each declared table against the contract (columns and
+  index names) through the MetaTables catalog before writing the row, and
+  reconciles at once; `reconcile=False` leaves the row `unavailable` with error
+  code `not_reconciled` until the first run. A provider that adds columns
+  passes verification.
+- Reads use `TimeIndexTableRef` for catalogued instances and keep the compiled
+  SQL path for the core tables, so a portfolio without a catalog row behaves
+  exactly as before. Reconciliation aggregates provider instances with one
+  compiled `count`, `min`, `max` statement per instance; this needs the Markets
+  workload to hold `data_viewer` on the instance, which is the grant this ADR
+  asks providers to give.
+- Signals: `SignalMetadataTable.meta_table_uid` is set by registration when the
+  portfolio has a `signal_uid`, so a signal read without a portfolio still
+  resolves its instance.
+
+### 2026-10-25: step 1, decision 3 (acting identity)
+
+Implemented in `apps/v1/identity.py`. `acting_identity()` returns the requester
+on a requester-bound call and the caller otherwise, never both, and `None`
+outside an authenticated request so local scripts and direct service calls keep
+working. `registry_write` wraps every `apps/v1` service that creates, updates or
+deletes a registry row and records resource, action, row uid and acting identity
+after a successful write. The Index actor is derived from the acting identity.
+
+Deviations and open items:
+
+- No Markets registry table has an author or owner column, so "recording the
+  acting user" is a structured log record on the `apps.v1.identity` logger, not
+  a column write. The hook is the place a column is filled once one exists.
+- Ownership and Team scoping policies are not enforced yet; the helper gives
+  every service one identity to authorize against, and the platform's own
+  grants still apply to requester-bound reads.
+- `acts_for_requester: true` on the `markets-api` resource is required for a
+  requester to be present. It is declared in
+  `.mainsequence/workflows/ms-markets-api.yaml` and deploys with the branch.
+
+### 2026-10-25: step 1, missing routes
+
+Added to `apps/v1`: account create and update; formula save, activate, retire
+and preview over `FormulaIndex`; batch lookups by uid (`uid__in`) for assets,
+accounts, portfolios and indexes; multi-portfolio values and weights frames;
+portfolio dataset registration, read and reconcile. Each follows the existing
+router and service layout, carries a Command Center contract and is registered
+in the adapter operation registry. Deviations and refinements:
+
+- Batch lookups are `GET <resource>/by-uids/?uid__in=` with repeated or
+  comma-separated uids, at most 500, returning the found rows in request order
+  inside a `ResourceCollection` without paging, search or a discovery route.
+  They are lookups, not collections, so they carry no
+  `command-center.resource_collection@v1` contract.
+- The multi-portfolio frames are `GET /portfolio/values/` and
+  `GET /portfolio/weights/` with `portfolio_uid__in`; every uid must be a
+  portfolio. `order` and `limit` apply to the merged result.
+- `include` hydration lives on the by-uid lookups: `include=<sidecar MetaTable
+  uid>` resolves the sidecar from the MetaTables catalog, requires a
+  single-column foreign key to the core table's `uid` (the sidecar rule of
+  decision 4), reads each sidecar once for the returned uids as the Markets
+  workload and attaches rows under `includes[<uid>]`. It works for a sidecar in
+  any namespace today, including `mainsequence.markets`, so the Alpaca account
+  listing can move before step 3. Search-driven list routes do not take
+  `include`; a consumer pages on its own sidecar and hydrates by uid, which is
+  the direction the worked example needs.
+- Formula save takes the Index identity from the stored row and accepts only
+  the definition and inputs, so a route cannot change identity fields while
+  storing a formula; `PATCH /index/{uid}/` remains the identity update.

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/docs/fast_api/v1/accounts.md

@@ -12,8 +12,41 @@ direct account snapshots and virtual-fund-grouped views.
   - `items` contains the library `msm.api.accounts.Account` contract:
     `uid`, `unique_identifier`, `account_name`, `is_paper`,
     `account_is_active`, `holdings_data_node_uid`, and `metadata_json`
+- `POST /api/v1/account/`
+  - creates one `Account` identity row from the `msm.api.accounts.AccountCreate`
+    payload (`unique_identifier`, `account_name`, `is_paper`,
+    `account_is_active`, `account_group_uid`, `holdings_data_node_uid`,
+    `metadata_json`) and returns it with `201`
+  - `409` when the row cannot be stored, for example a duplicate
+    `unique_identifier`
+  - ADR 0047: providers register accounts here instead of writing the Account
+    table; the write runs as the Markets workload and records the acting
+    identity
+- `GET /api/v1/account/by-uids/?uid__in=<uid>&uid__in=<uid>`
+  - batch lookup by uid; `uid__in` is repeated or comma-separated, at most 500
+  - returns the matching `Account` rows in request order inside a
+    `ResourceCollection`; unknown uids are omitted, so `totalItems` is the
+    number found
+  - a bounded lookup for consumers that hydrate core columns from their own
+    rows, not a browsable collection: no `limit`, `offset`, search or
+    discovery route
+  - `include=<sidecar MetaTable uid>` (repeated or comma-separated, at most
+    10) attaches provider sidecar rows under `includes[<uid>]` on every item
+    (`null` when the sidecar has no row). A sidecar is a provider table with a
+    single-column foreign key to this resource's `uid`, proven from the
+    MetaTables catalog; each sidecar is read once for all returned uids. The
+    Markets workload needs `data_viewer` on the sidecar (`409` otherwise,
+    `404` for a uid that is not a visible MetaTable, `400` for a table that is
+    not a sidecar of this resource). Columns come back as stored; Markets never
+    decrypts a provider column. `includes` is `{}` when nothing is included
 - `GET /api/v1/account/{uid}/`
   - returns one canonical `Account` row by uid
+- `PATCH /api/v1/account/{uid}/`
+  - updates the mutable fields of one `Account` row from the
+    `msm.api.accounts.AccountUpdate` payload (`account_name`, `is_paper`,
+    `account_is_active`, `account_group_uid`, `holdings_data_node_uid`,
+    `metadata_json`); omitted fields are unchanged
+  - `404` when the uid is unknown
 - `GET /api/v1/account/{uid}/summary/`
   - returns the reusable `FrontEndDetailSummary` response for account detail
     pages

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/docs/fast_api/v1/assets.md

@@ -20,6 +20,21 @@ provider-specific records. The API does not flatten those separate models into `
   - exposes `GET /api/v1/asset/discovery/`
   - returns the library `msm.api.assets.Asset` contract:
     `uid`, `unique_identifier`, and `asset_type`
+- `GET /api/v1/asset/by-uids/?uid__in=<uid>&uid__in=<uid>`
+  - batch lookup by uid (ADR 0047); `uid__in` is repeated or comma-separated,
+    at most 500
+  - returns the matching `Asset` identity rows in request order inside a
+    `ResourceCollection`; unknown uids are omitted
+  - not a browsable collection: no `limit`, `offset`, search or discovery route
+  - `include=<sidecar MetaTable uid>` (repeated or comma-separated, at most
+    10) attaches provider sidecar rows under `includes[<uid>]` on every item
+    (`null` when the sidecar has no row). A sidecar is a provider table with a
+    single-column foreign key to this resource's `uid`, proven from the
+    MetaTables catalog; each sidecar is read once for all returned uids. The
+    Markets workload needs `data_viewer` on the sidecar (`409` otherwise,
+    `404` for a uid that is not a visible MetaTable, `400` for a table that is
+    not a sidecar of this resource). Columns come back as stored; Markets never
+    decrypts a provider column. `includes` is `{}` when nothing is included
 - `GET /api/v1/asset/monitor/frame/`
   - supports `search`, `limit`, `offset`, optional `asset_type`, and repeated
     `unique_identifiers`

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/docs/fast_api/v1/index.md

@@ -209,10 +209,30 @@ Bearer token. Handlers read the verified caller from `request.state.user` (a
 `User.get_logged_user()`. The platform's caller assertions carry only those two
 facts, for the caller and for a requester; an assertion that lists `team_uids` is
 refused with `401`. Membership is checked on demand with
-`RequestUserIdentity.in_teams()`, which no Markets route needs. The Index routes
-derive their actor from that identity's `uid`; the deprecated
-`IndexActor.team_uids` stays empty for HTTP callers. Route code does not parse
-authentication headers or bind SDK request-header context variables.
+`RequestUserIdentity.in_teams()`, which no Markets route needs. Route code does
+not parse authentication headers or bind SDK request-header context variables.
+
+Every handler and service acts for one identity (ADR 0047, decision 3):
+`apps.v1.identity.acting_identity()` returns `User.get_requester()` on a
+requester-bound call and `User.get_logged_user()` otherwise, never both, and
+`None` outside an authenticated request. The Index routes derive their actor from
+it; the deprecated `IndexActor.team_uids` stays empty for HTTP callers. Every
+service that creates, updates or deletes a registry row is wrapped with
+`apps.v1.identity.registry_write`, which resolves the acting identity before the
+write and, after a successful write, records resource, action, row uid and the
+acting identity's uid and kind on the `apps.v1.identity` logger. No registry
+table has an author or owner column yet; when one does, the same hook supplies
+its value. Writes keep running as the Markets workload, whose `data_writer`
+role on `mainsequence.markets` is the single place writes are allowed; the API's
+policy decides who may trigger them.
+
+A requester is only present when the `markets-api` resource declares
+`acts_for_requester: true` in `.mainsequence/workflows/ms-markets-api.yaml`.
+That declaration lets the platform sign the person an application works for
+into the caller assertion; it is an ordinary workflow declaration pushed with
+the deployment. Until it is declared, every call is caller-bound and
+`acting_identity()` returns the caller, the calling application's workload
+User. The declaration is in the repository's workflow file.
 
 ## API Discoverability
 

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/docs/fast_api/v1/indexes.md

@@ -12,12 +12,18 @@ typed-library operation through `FormulaIndex`.
 | `GET` | `/api/v1/index-type/{index_type}/` | `getIndexType` | One type |
 | `GET` | `/api/v1/index/` | `listIndexes` | Counted Index page |
 | `POST` | `/api/v1/index/` | `createIndex` | New Index identity |
+| `GET` | `/api/v1/index/by-uids/` | `getIndexesByUids` | Batch lookup by uid |
 | `GET` | `/api/v1/index/{uid}/` | `getIndex` | One Index |
 | `PATCH` | `/api/v1/index/{uid}/` | `updateIndex` | Updated identity metadata |
 | `DELETE` | `/api/v1/index/{uid}/` | `deleteIndex` | Direct row deletion |
 | `GET` | `/api/v1/index/{uid}/summary/` | `getIndexSummary` | Detail summary |
 | `GET` | `/api/v1/index/{uid}/formulas/` | `listIndexFormulas` | Formula history |
+| `POST` | `/api/v1/index/{uid}/formulas/` | `saveIndexFormula` | Stored formula version |
+| `POST` | `/api/v1/index/{uid}/formulas/preview/` | `previewIndexFormula` | Unsaved formula as a tabular frame |
 | `GET` | `/api/v1/index/{uid}/formulas/{definition_uid}/` | `getIndexFormula` | Formula and exact inputs |
+| `POST` | `/api/v1/index/{uid}/formulas/{definition_uid}/activate/` | `activateIndexFormula` | Activated version |
+| `POST` | `/api/v1/index/{uid}/formulas/{definition_uid}/retire/` | `retireIndexFormula` | Retired version |
+| `POST` | `/api/v1/index/{uid}/formulas/{definition_uid}/preview/` | `previewStoredIndexFormula` | Stored formula as a tabular frame |
 | `GET` | `/api/v1/index/{uid}/datasets/` | `listIndexDatasets` | Per-Index dataset states |
 | `GET` | `/api/v1/index/{uid}/datasets/{meta_table_uid}/` | `getIndexDatasetSummary` | Selected dataset summary |
 | `GET` | `/api/v1/index/{uid}/datasets/{meta_table_uid}/values/` | `getIndexDatasetValuesFrame` | Bounded tabular frame |
@@ -26,7 +32,15 @@ typed-library operation through `FormulaIndex`.
 | `GET` | `/api/v1/asset/{uid}/related-meta-tables/` | `listAssetRelatedMetaTables` | Asset source-table discovery |
 
 All operations are present in the Adapter from API contract. Create, update,
-and delete are mutations.
+delete, and the formula save, activate and retire routes are mutations; the
+previews are queries that persist nothing.
+
+`GET /api/v1/index/by-uids/?uid__in=<uid>&uid__in=<uid>` is a bounded batch
+lookup (ADR 0047): `uid__in` is repeated or comma-separated, at most 500, the
+matching rows return in request order inside a `ResourceCollection`, unknown
+uids are omitted, and there is no paging, search or discovery route.
+`include=<sidecar MetaTable uid>` attaches provider sidecar rows under
+`includes[<uid>]`, as described for [accounts](accounts.md).
 
 ## Identity Payload
 
@@ -80,6 +94,40 @@ and input count. Detail adds alignment parameters, metadata, and exact inputs:
 The response has no source key, resolver object, configurable identity/value
 columns, selectors, transforms, coefficients, or units.
 
+## Formula Lifecycle
+
+Formula creation, lifecycle and preview go through `msm.api.FormulaIndex`
+(ADR 0037, ADR 0038); the routes are thin wrappers that take the Index
+identity from the stored row and record the acting identity (ADR 0047).
+
+- `POST /api/v1/index/{uid}/formulas/` with `{"definition": ..., "inputs": [...]}`
+  persists one version through `FormulaIndex.upsert`. `definition` is an
+  `IndexFormulaDefinition` (`valid_from`, `formula`, `alignment_policy`,
+  `alignment_parameters_json`, `missing_data_policy`, optional `status`,
+  `version`, `valid_to`, `metadata_json`); `inputs` are the exact source
+  bindings shown above. A definition identical to a stored version returns that
+  version; otherwise the next monotonic version is stored as `draft` unless
+  `status` asks for `active` or `retired`. `400` for a formula that fails the
+  contract or an Index with `calculation_method="custom"`, `404` for an unknown
+  Index or source, `409` when the row cannot be stored. Returns the
+  `IndexFormulaDetail` with `201`.
+- `POST /api/v1/index/{uid}/formulas/{definition_uid}/activate/` activates a
+  draft and retires the open predecessor atomically.
+- `POST /api/v1/index/{uid}/formulas/{definition_uid}/retire/` retires a version
+  at `valid_to` (body, optional; now by default).
+- `POST /api/v1/index/{uid}/formulas/preview/` with
+  `{"definition": ..., "inputs": [...], "start": ..., "end": ...}` validates an
+  unsaved formula, reads the pinned source MetaTables over the window through
+  `TimeIndexTableRef` and returns the calculated observations as
+  `core.tabular_frame@v1` (`time_index`, `index_identifier`, `value`,
+  `definition_uid`, `observation_status`, `source_as_of`, `metadata_json`;
+  timestamps as epoch milliseconds). Nothing is persisted.
+- `POST /api/v1/index/{uid}/formulas/{definition_uid}/preview/` with
+  `{"start": ..., "end": ...}` does the same for a stored version.
+
+`start` and `end` must be timezone-aware and `end` later than `start`. A
+definition uid that belongs to another Index is `404`.
+
 ## Related MetaTables
 
 Asset and Index related-table routes accept:

{ms_markets-2.2.13.dev37 → ms_markets-2.2.13.dev38}/docs/fast_api/v1/portfolio.md

@@ -4,7 +4,8 @@ The `apps/v1` portfolio routes expose portfolio identity rows, detail-page
 composition, latest portfolio weights, and delete operations.
 
 The routes do not create or update portfolios. Portfolio construction remains a
-library workflow outside this API surface.
+library workflow outside this API surface. ADR 0047 adds the multi-portfolio
+frames, the batch lookup and the dataset catalog routes described below.
 
 ## Runtime Sources
 
@@ -18,6 +19,11 @@ library workflow outside this API surface.
 - Signal weights resolve from nullable `Portfolio.signal_uid`, which points to
   `SignalMetadataTable.signal_uid`. The API does not recompute signal identity
   from the TimeIndexTableUpdater build configuration or by scanning shared signal storage.
+- Values, weights and signal-weight reads resolve each portfolio's table from
+  the Markets catalog (`PortfolioDatasetAvailabilityTable`) and read it by uid
+  through `TimeIndexTableRef`; a portfolio without a catalog row reads the core
+  tables (ADR 0047, see the
+  [portfolios knowledge page](../../knowledge/msm_portfolios/portfolios/index.md)).
 
 Latest weights resolve through:
 
@@ -240,6 +246,88 @@ When present, it is the canonical `SignalMetadataTable.signal_uid` used by
 `GET /api/v1/portfolio/{uid}/signals_weights/`. The signal-weights endpoint
 does not infer the signal by scanning shared signal-weight storage.
 
+## Batch Lookup By Uid
+
+```text
+GET /api/v1/portfolio/by-uids/?uid__in=<uid>&uid__in=<uid>
+```
+
+Returns the matching `Portfolio` rows in request order inside a
+`ResourceCollection`. `uid__in` is repeated or comma-separated, at most 500;
+unknown uids are omitted. A bounded lookup, not a browsable collection: no
+`limit`, `offset`, search or discovery route. `include=<sidecar MetaTable uid>`
+attaches provider sidecar rows under `includes[<uid>]`, as described for
+[accounts](accounts.md).
+
+## Values Frame For Several Portfolios
+
+```text
+GET /api/v1/portfolio/values/?portfolio_uid__in=<uid>&portfolio_uid__in=<uid>&start_date=&end_date=&order=desc&limit=500
+```
+
+Returns one `core.tabular_frame@v1` with columns `time_index`,
+`portfolio_identifier`, `portfolio_uid`, `close`, `return`, `calculated_close`
+and `close_time` for every requested portfolio. The portfolios are grouped by
+the `PortfoliosTS` instance the catalog records for them; each instance is read
+once, so comparing an Alpaca portfolio with an Interactive Brokers portfolio is
+one call whose two sources live in two namespaces. `order` sorts by
+`time_index` across all portfolios and `limit` (1 to 5000) bounds the merged
+result. Every uid must be a portfolio (`404` otherwise); `source.context`
+names the requested uids and the window.
+
+## Weights Frame For Several Portfolios
+
+```text
+GET /api/v1/portfolio/weights/?portfolio_uid__in=<uid>&weights_date=&as_of=true
+```
+
+Returns one `core.tabular_frame@v1` with one weight snapshot per portfolio:
+the latest snapshot, the latest at or before `weights_date` with `as_of=true`,
+or the exact `weights_date` snapshot with `as_of=false`. Columns are
+`time_index`, `portfolio_identifier`, `portfolio_uid`, `asset_identifier`,
+`weight`, `weight_before`, `price_current`, `price_before`, `volume_current`
+and `volume_before`. Portfolios are grouped by their `PortfolioWeightsTS`
+instance as for values.
+
+## Dataset Catalog
+
+```text
+POST /api/v1/portfolio/{uid}/datasets/
+GET  /api/v1/portfolio/{uid}/datasets/
+POST /api/v1/portfolio/datasets/reconcile/
+```
+
+A provider that computes a portfolio in its own namespace registers, per
+portfolio, the `PortfoliosTS`, `PortfolioWeightsTS` and `SignalWeightsTS`
+instances that hold its rows, and reconciles after each run (ADR 0047).
+
+- `POST /api/v1/portfolio/{uid}/datasets/` takes `values_meta_table_uid`
+  (required), `weights_meta_table_uid`, `signal_meta_table_uid` and
+  `reconcile` (default `true`). Each table must be a `TimeIndexMetaTable`
+  visible to the Markets workload that exposes the contract columns and index
+  names; extra provider columns are allowed. The row is upserted, the signal's
+  `SignalMetadata.meta_table_uid` is bound when the portfolio has a
+  `signal_uid`, and the values instance is reconciled unless `reconcile` is
+  `false`, in which case the row is `unavailable` with error code
+  `not_reconciled`. Returns the `PortfolioDatasetState` with `201`; `404` for
+  an unknown portfolio or a table that is not visible, `400` for a table that
+  does not satisfy its contract.
+- `GET /api/v1/portfolio/{uid}/datasets/` returns the catalog row, or `null`
+  when the portfolio exists and reads the core tables; `404` for an unknown
+  portfolio.
+- `POST /api/v1/portfolio/datasets/reconcile/` takes `portfolio_uids` and
+  `portfolio_identifiers` (at least one selector, at most 500 in all) and
+  returns a `PortfolioDatasetReconciliationResult`: the reconciled states, one
+  aggregate per instance however many portfolios it holds, plus
+  `skipped_portfolio_uids` for portfolios without a catalog row. `404` names
+  selectors that are not portfolios.
+
+`PortfolioDatasetState` carries `portfolio_uid`, `portfolio_identifier`, the
+three table uids, `population_state` (`populated`, `compatible_empty` or
+`unavailable`), `row_count`, `earliest_time_index`, `latest_time_index`,
+`reconciled_at` and `error`. Registration and reconciliation record the acting
+identity.
+
 ## Latest Portfolio Weights
 
 ```text