@zhuoyuezs/ml-platform 0.1.10 → 0.1.12

package/skills/feature-management/references/contracts.md

@@ -0,0 +1,766 @@
+# Current Contract Reference
+
+Use this reference for the implemented V1 schemas. Prefer the running CLI and `src/data_platform_demo/schemas.py` when they disagree with historical design documents.
+
+## Contents
+
+1. Naming and versioning
+2. Projects and namespacing
+3. Parameter
+4. Operator
+5. Feature
+6. FeatureSet
+7. DatasetManifest
+8. Catalog
+9. Cross-object validation
+
+## Naming And Versioning
+
+- Use identifiers matching `^[A-Za-z0-9][A-Za-z0-9_-]*$` and at most 128 characters.
+- Prefer lowercase snake_case business identifiers.
+- Use explicit immutable versions such as `v1`, `v2`.
+- Create a new version when any semantic field changes.
+- Keep credentials in the configured profile or server Secret, never in an asset.
+
+## Projects And Namespacing
+
+- Every registry asset is namespaced by a `project`. Its true registry key is
+  `project/name:version`, so `name:version` alone is unique only within one
+  project; the same `name:version` may exist independently under other projects.
+- The platform ships a built-in `default` project. When no project is given,
+  assets are created and read under `default`.
+- Conceptually a project is a namespace, not an attribute of the asset. In the
+  current implementation, however, the project a Parameter/Operator/Feature/
+  FeatureSet/DatasetManifest is **created** under comes from the optional
+  `project` field in its own JSON spec (it defaults to `default` when omitted).
+  So to place an asset in a non-default project, set `"project": "<name>"` in the
+  spec before `apply`.
+- For a confirmed non-default target, include that field in every draft
+  Parameter, Operator, Feature, Feature input, FeatureSet, and DatasetManifest;
+  do not rely on Pydantic's `default` fallback during authoring.
+- Reads and filters, by contrast, take the project out-of-band: `list-* --project`
+  and the leading project argument of the artifact commands. `apply --project`
+  scopes the comparison snapshot that publication is validated against.
+- All references inside one catalog resolve within a single project. Cross-project
+  references (a Feature in project A depending on a Parameter in project B) are
+  rejected by the platform.
+- Immutability is per project: publishing `name:v1` into project A never touches
+  `name:v1` in project B.
+
+## Parameter
+
+Required shape:
+
+```json
+{
+  "schema_version": "ml_data_platform.parameter/v1",
+  "name": "hot_blast_pressure",
+  "display_name": "Hot blast pressure",
+  "version": "v1",
+  "data_type": "time_series",
+  "unit": "kPa",
+  "expected_frequency": "5s",
+  "source": {},
+  "online_source": null,
+  "time_semantics": {
+    "event_time_field": "event_time",
+    "ingested_at_field": "ingested_at",
+    "timezone": "Asia/Shanghai",
+    "availability": {
+      "strategy": "source_field",
+      "field": "ingested_at",
+      "accuracy": "exact"
+    }
+  },
+  "availability_sla": {
+    "max_delay": "PT10M"
+  },
+  "value_field": "value",
+  "quality_rules": {
+    "valid_range": [0.0, 450.0],
+    "allow_missing": true,
+    "rules": [
+      {
+        "id": "finite_source_values",
+        "stage": "normalized_source",
+        "check": {"type": "finite"},
+        "acceptance": {"max_violation_rate": 0.0},
+        "enforcement": "fail"
+      }
+    ]
+  },
+  "owner": "business_owner"
+}
+```
+
+`online_source` 是可选的实时来源覆盖。离线 Dataset build、`snapshot`、`as_of` 和历史回放
+只读取 `source`；实时因果读取在声明时读取 `online_source`，否则回退到 `source`。两者必须
+表达同一个 Parameter，产生相同语义的标准长表（`timestamp, metric_name, value, unit`），
+并保持值、单位、事件时间、频率、质量和可用时间语义一致。它们可以使用不同的受支持
+adapter；凭据不进入 JSON。Feature/Operator/FeatureSet 不需要复制一份。
+
+例如：
+
+```json
+{
+  "source": {
+    "adapter": "postgresql_direct",
+    "schema": "process_data",
+    "table": "pressure_history",
+    "time_column": "event_time",
+    "value_column": "pressure",
+    "metric_name": "hot_blast_pressure"
+  },
+  "online_source": {
+    "adapter": "influxdb_direct",
+    "measurement": "blast_furnace_live",
+    "field": "pressure",
+    "metric_name": "hot_blast_pressure"
+  }
+}
+```
+
+实时 lineage 会标出 `source_role=online_source` 并保留 `offline_source`。`online_source`
+参与 Parameter hash；因此变更它必须发布新的 Parameter 版本及受影响的下游依赖。使用在线
+覆盖时，响应中的 replay 会明确标记为不支持，因为离线 manifest 只会重新读取 `source`，
+不能伪造线上读法的回放。
+
+Supported source modes are `direct_column`, `sql`, and `derived`. Prefer `direct_column` for ordinary business parameters.
+
+For `sql`, `source.sql` must contain one statement without a trailing semicolon,
+and named placeholders must be supplied through `source.params`; `start` and
+`end` are reserved runtime window parameters. `source.parameters` is not a
+supported field.
+
+`availability.strategy` 支持 `source_field`、`fixed_delay` 和 `unsupported`。只有一个
+事件时间列时，可以使用经过数据 owner 确认的固定延迟上界，例如
+`{"strategy":"fixed_delay","delay":"PT10M","accuracy":"upper_bound"}`。
+当前严格 `as_of` 只接受 `exact` 或有合同支撑的 `upper_bound`；修改这些语义需要发布
+新的 Parameter 业务版本。
+
+An optional `rounding` section states the fixed-point precision the platform
+enforces at the Parameter boundary:
+
+```json
+"rounding": {"mode": "half_up", "decimals": 2}
+```
+
+Fixing only the decimal count leaves the tie case to whichever engine evaluates
+it: PostgreSQL `ROUND` on `numeric` rounds halves away from zero (5.625 becomes
+5.63) while Python and NumPy round halves to even (5.625 becomes 5.62). A single
+0.01 difference spreads through rolling windows into many derived Features, so
+declare both fields. `mode` is `half_up` or `half_even`.
+
+The platform applies the rule with deterministic decimal quantization after the
+adapter produces the standard metric frame and before quality checks, source
+cache publication/replay, and Operator execution. Prefer an unrounded source
+expression. A source that already applies the same rule is accepted because the
+second application is idempotent, but an upstream path that already discarded a
+tie with a different rule cannot be repaired downstream. Do not repeat the same
+Parameter-level rounding in an Operator. Changing the mode, decimals, or stage
+requires a new Parameter version and the affected reverse-dependency closure.
+
+PostgreSQL, MySQL, and SQL Server direct-column Parameters share this source
+shape. Set `adapter` to `postgresql_direct`, `mysql_direct`, or
+`sqlserver_direct`; keep database-specific DSNs in server-managed Secrets.
+
+```json
+{
+  "adapter": "postgresql_direct",
+  "mode": "direct_column",
+  "schema": "process_data",
+  "table": "sensor_readings",
+  "time_column": "event_time",
+  "value_column": "hot_blast_pressure",
+  "metric_name": "hot_blast_pressure",
+  "unit_column": "unit",
+  "filters": {"furnace_id": "BF12"}
+}
+```
+
+The same contract for MySQL or SQL Server changes only the adapter name:
+
+```json
+{"adapter": "mysql_direct", "mode": "direct_column", "schema": "process_data", "table": "sensor_readings", "time_column": "event_time", "value_column": "hot_blast_pressure", "metric_name": "hot_blast_pressure"}
+```
+
+```json
+{"adapter": "sqlserver_direct", "mode": "direct_column", "schema": "process_data", "table": "sensor_readings", "time_column": "event_time", "value_column": "hot_blast_pressure", "metric_name": "hot_blast_pressure"}
+```
+
+InfluxDB direct-column example:
+
+```json
+{
+  "adapter": "influxdb_direct",
+  "mode": "direct_column",
+  "measurement": "blast_furnace",
+  "field": "hot_blast_pressure",
+  "metric_name": "hot_blast_pressure",
+  "tags": {"furnace_id": "BF12"}
+}
+```
+
+Do not put DSNs, access keys, tokens, passwords, or connection endpoints containing credentials in `source`.
+
+`quality_rules.rules` defines immutable single-Parameter assertions. Supported
+checks are `not_null`, `finite`, `range`, and `max_gap`. `stage` is one of
+`normalized_source`, `post_preprocess`, or `aligned_grid`; execution follows
+that order. `acceptance` controls evidence thresholds, while `enforcement`
+chooses `observe`, `warn`, or `fail`. Keep cross-Parameter assertions out of a
+Parameter version and declare them in DatasetManifest `parameter_relationships`.
+
+`quality_rules.missing_detection` controls source-observation gap detection and
+is independent from value validity. It defaults to `disabled`, which is the
+safe behavior for irregular event Parameters such as tapping events or silicon
+samples. Use `fixed_frequency` only when `expected_frequency` is a real fixed
+duration. For irregular Parameters, use an explicit `max_gap` or an
+`interval_multiplier` over a `mean` or `median` observed interval:
+
+```json
+"missing_detection": {
+  "mode": "interval_multiplier",
+  "multiplier": 10,
+  "baseline": "median",
+  "min_observations": 2
+}
+```
+
+`max_gap` accepts positive short durations including second precision (for
+example `30s`, `10min`, or `2h`). Detection runs on the normalized,
+post-preprocess source frame before target-grid interpolation, and its source
+windows are recorded in the build's `missing_windows.json` artifact.
+
+## Operator
+
+Feature Operator draft shape:
+
+```json
+{
+  "schema_version": "ml_data_platform.operator/v1",
+  "name": "pressure_features",
+  "version": "v1",
+  "type": "feature",
+  "function_hash": "pressure_features.formulas.v1",
+  "entrypoint": "pressure_features:compute_features",
+  "code_hash": null,
+  "package_uri": null,
+  "code_artifact": null,
+  "input_schema": {
+    "parameters": ["hot_blast_pressure:v1"]
+  },
+  "output_schema": {
+    "columns": ["event_time", "pressure_mean_5m"]
+  },
+  "config_schema": {
+    "properties": {
+      "window": {"type": "string"}
+    },
+    "required": ["window"]
+  },
+  "runtime": {"engine": "python_entrypoint", "network": "none"},
+  "resources": {"timeout_seconds": 60},
+  "deterministic": true,
+  "supports_batch": true,
+  "supports_online": false,
+  "owner": "business_owner"
+}
+```
+
+For a catalog wheel entry, `catalog.json` supplies the package file. Publication calculates and stores `code_hash`, `package_uri`, and `code_artifact`.
+
+Treat `pythonpath://` as platform-owned example behavior. Publish formal business Operators as immutable pure-Python wheels.
+
+## Feature
+
+```json
+{
+  "schema_version": "ml_data_platform.feature/v1",
+  "name": "pressure_mean_5m",
+  "version": "v1",
+  "inputs": [
+    {"parameter": "hot_blast_pressure", "version": "v1"}
+  ],
+  "operator": "pressure_features",
+  "operator_version": "v1",
+  "config": {"window": "5min"},
+  "output_column": "pressure_mean_5m",
+  "output_dtype": "float64",
+  "offline_online_supported": false,
+  "owner": "business_owner",
+  "description": "Mean pressure over the causal five-minute window."
+}
+```
+
+Rules:
+
+- Define exactly one public output column per Feature.
+- Keep `inputs` nonempty, unique, versioned, and formula-exact.
+- Require the referenced Operator to have `type=feature`.
+- Require `output_column` to appear in the Operator output schema.
+- Use `name` as the final dataset column; use `output_column` as the Operator's physical result column.
+
+## FeatureSet
+
+```json
+{
+  "schema_version": "ml_data_platform.feature_set/v1",
+  "name": "pressure_core",
+  "version": "v1",
+  "features": [
+    {"name": "pressure_mean_5m", "version": "v1"}
+  ],
+  "owner": "business_owner",
+  "description": "Ordered pressure model inputs."
+}
+```
+
+The list must be nonempty, unique, and ordered exactly as consumers expect.
+
+## DatasetManifest
+
+```json
+{
+  "schema_version": "ml_data_platform.dataset_manifest/v1",
+  "dataset_id": "pressure_training_20260701_20260702",
+  "dataset_version": "v1",
+  "mode": "training",
+  "read_policy": "snapshot",
+  "time_range": {
+    "start": "2026-07-01T00:00:00+08:00",
+    "end": "2026-07-02T00:00:00+08:00",
+    "grid": "10min"
+  },
+  "prediction": {
+    "horizon": "10min"
+  },
+  "parameters": [],
+  "parameter_missing_policies": [
+    {
+      "parameter": "hot_blast_pressure",
+      "version": "v1",
+      "project": "default",
+      "policy": "report_only"
+    }
+  ],
+  "feature_set": {"name": "pressure_core", "version": "v1"},
+  "preprocess": [],
+  "output": {
+    "format": "parquet",
+    "include_stats": true,
+    "include_lineage": true,
+    "engine": "chronon"
+  }
+}
+```
+
+Current schema rules:
+
+- Accept exactly one `feature_set`.
+- Reject old top-level `features` and `recipe` fields.
+- Use a half-open `[start, end)` target grid. `start` must be earlier than
+  `end`, the window must span at least one grid step, both bounds must agree on
+  timezone awareness, and `grid` must be one of `1min`, `5min`, `10min`,
+  `15min`, `20min`, `30min`, `1h`, `2h`, `1d`. Violations are rejected at
+  submission with a field-level error on `time_range`; no Job is created.
+- Require `as_of` only with `read_policy=as_of`.
+- Allow `snapshot_id` only with `read_policy=snapshot`.
+- Use `parameters` only to request explicit Parameter output columns.
+- Declare one dataset-wide `prediction` contract for forecast Features. `horizon`
+  accepts non-negative whole-minute durations such as `0min`, `10min`, or `2h`.
+  The platform always derives `cutoff_times = target_times - horizon` and records
+  that derivation in resolved lineage; it is not a user-configurable policy.
+
+### Optional sections
+
+These ten sections may be omitted: `source_read`, `realtime_fetch`,
+`abnormal_windows`, `target`, `prediction`, `parameter_relationships`,
+`parameter_missing_policies`, `rowset`, `rowset_splits`, and `endpoint_policy`.
+Omitting a section keeps older manifest hashes stable.
+Sections containing only inert empty values are also treated as unset, but a
+declared `realtime_fetch` policy contains meaningful defaults and changes the
+manifest hash. Declaring `endpoint_policy` also changes the manifest hash.
+
+Each explicit `parameters` request may set `missing_policy`. A
+`parameter_missing_policies` entry is reserved for a Parameter used only through
+the selected FeatureSet's input closure; it must not also appear in
+`parameters`. Its exact V1 shape is `{"parameter": "<name>", "version":
+"<version>", "project": "<project>", "policy": "<policy>"}`. Do not use
+the `ParameterRequest` field names `name` or `missing_policy` in this list. For
+a confirmed non-default target, set `project` explicitly instead of relying on
+its `default` fallback. The policies are:
+
+- `report_only`: retain all target rows and report source gaps.
+- `fail_on_requested_range_gap`: fail when a detected gap affects the requested
+  target range.
+- `drop_target_and_lookback`: remove the target rows in the detected window and
+  the following rows whose resolved Feature input lookback still reaches into
+  that window. The same merged row mask is applied to both
+`feature_dataset.parquet` and `parameter_dataset.parquet`.
+
+The default `report_only` value is omitted from canonical manifest identity so
+manifests published before `missing_policy` existed keep their artifact hash.
+When `endpoint_policy` is declared, `drop_target_and_lookback` is rejected:
+endpoint eligibility requires the canonical datasets and
+`rowset_membership.parquet` to retain the same complete candidate rowset. Use
+`report_only` or `fail_on_requested_range_gap` for those manifests.
+
+The resolved per-Parameter windows, policy, affected rows, and dropped rows are
+written to `missing_windows.json` and summarized in `validation.json`. A
+partitioned build with a drop policy must be materialized as one full source
+window so the same global gap contract is used for every rowset.
+
+`source_read` pages the source reads. A long target window read in one request
+can exceed the driver read timeout; chunking keeps one manifest covering the
+whole window instead of splitting it into per-day manifests. Chunk windows,
+attempts, rows, and elapsed time land in
+`latency_stats.fetch_timing_summary`.
+
+```json
+"source_read": {
+  "chunk_days": 1,
+  "timeout_seconds": 60,
+  "max_retries": 3,
+  "retry_sleep_seconds": 2
+}
+```
+
+`realtime_fetch` declares the policy for one low-latency causal-cutoff read:
+
+```json
+"realtime_fetch": {
+  "max_workers": 4,
+  "min_lookback": "1h",
+  "validate_freshness": true,
+  "max_source_lag_hours": 3.0,
+  "max_source_lag_hours_by_parameter": {},
+  "allow_missing": false,
+  "allow_tail_edge_missing": true,
+  "max_tail_edge_missing": 1,
+  "tail_edge_missing_max_age_minutes": 65.0,
+  "allow_internal_gap_fill": true,
+  "internal_gap_fill_max_age_hours": 5.0
+}
+```
+
+`POST /inference-data/fetch` serves one cutoff without running a batch build,
+writing parquet, or publishing a DatasetArtifact. It derives
+`target_time = cutoff_time + prediction.horizon`, filters every source to
+`timestamp <= cutoff_time`, and derives each Parameter read window from the
+selected Operators' `input_schema.history_requirements`. The response carries
+`contract.manifest_hash`; it also carries a replayable `read_policy=as_of` manifest
+unless a Parameter's `online_source` override is active, in which case
+`replay.supported=false` is returned because an offline manifest would read the
+offline `source` instead.
+
+Keep the default strict: `allow_missing=false`. Tail-edge and bounded internal
+gap-fill tolerance may return a row, but the response must mark
+`freshness.degraded=true` and include validation evidence. A freshness or
+missing-data condition that may recover returns HTTP `503`; an invalid contract
+returns HTTP `400`. Declaring `realtime_fetch` changes the manifest hash; leaving
+it unset preserves older contract hashes and uses environment fallbacks.
+
+`abnormal_windows` excludes target windows whose data must not be trusted. The
+policy and the lookback apply to the whole window set, not to individual
+windows: a dataset is either a filtered rowset or a marked full rowset.
+
+```json
+"abnormal_windows": {
+  "policy": "drop_target_and_lookback",
+  "windows": [
+    {"start": "2026-05-25T19:30:00+08:00",
+     "end": "2026-05-26T09:50:00+08:00",
+     "reason": "shutdown"}
+  ]
+}
+```
+
+| policy | Meaning |
+|-|-|
+| `drop_target_and_lookback` | Drop rows in a window, and rows after it whose feature input lookback reaches back in. Default. |
+| `drop_target_only` | Drop only rows inside a window. |
+| `mark_only` | Drop nothing; record what is affected in lineage and validation. |
+
+How far past a window to drop is derived from the widest resolved Operator history
+requirement. Cutoff-relative history contributes `prediction.horizon + lookback`;
+target-time-relative history contributes its declared lookback directly. The
+maximum is used for source fetch and abnormal-window propagation. Do not restate
+that number unless you want it asserted: an optional `input_lookback_hours` is
+checked against the derived value and a mismatch is rejected. Note that a window
+near the end of the target range can have its lookback clipped by the range end,
+and a window close to the end can remove most remaining rows.
+
+`target` states which instant the label is aligned to, so reviewing a manifest
+shows the target definition rather than only a Parameter version string. When
+the target is a Parameter, keep its complete versioned reference **inside**
+`target.parameter`; do not flatten `version` or `project` onto `target`.
+`si_time_source` is `weight_time` or `dispatch_time`. `interpolation` is an
+object with a `method`, never a bare string.
+
+```json
+"target": {
+  "parameter": {
+    "parameter": "hot_metal_si",
+    "version": "v1",
+    "project": "replace_with_project"
+  },
+  "si_time_source": "weight_time",
+  "offset_minutes": 0,
+  "interpolation": {"method": "linear"}
+}
+```
+
+Allowed interpolation methods are `none`, `linear`, and `forward_fill`. A
+business phrase such as "allow and report target nulls" belongs in an existing
+Parameter missing policy, Feature configuration, or the semantic review; it is
+not a free-form `target` or DatasetManifest field.
+
+The declaration is checked against the selected Si Parameter, but only for
+`direct_column` reads, which name their time column in the contract. A `sql` or
+`derived` Parameter can apply an offset inside its query text, so those are
+recorded as unverified with a reason and a warning — do not read the absence of
+an error as confirmation that the label instant matches.
+
+`parameter_relationships` evaluates manifest-scoped assertions over exact
+Parameter versions after preprocessing and grid alignment:
+
+```json
+"parameter_relationships": [
+  {
+    "id": "temperature_pair_consistency",
+    "inputs": [
+      {"alias": "left", "parameter": "temperature_left", "version": "v1"},
+      {"alias": "right", "parameter": "temperature_right", "version": "v1"}
+    ],
+    "alignment": {"method": "last_before_or_at", "tolerance": "PT5M"},
+    "rules": [
+      {
+        "id": "bounded_difference",
+        "stage": "aligned_grid",
+        "check": {
+          "type": "absolute_difference",
+          "left": "left",
+          "right": "right",
+          "max_difference": 10.0
+        },
+        "acceptance": {"min_comparable_rate": 0.9},
+        "enforcement": "fail"
+      }
+    ]
+  }
+]
+```
+
+Relationship checks are `compare`, `absolute_difference`, `ratio_range`,
+`sum_equals`, and `co_presence`. `exact` compares only exact grid timestamps.
+`last_before_or_at` is causal. With a tolerance, each relationship input fetch
+starts at `manifest.start - tolerance`; without one, lookup is bounded to data
+inside the manifest target window and does not trigger an unbounded history read.
+
+`rowset` selects which instants become dataset rows:
+
+```json
+"rowset": {
+  "strategy": "fixed_grid",
+  "grid": "10min"
+}
+```
+
+Supported strategies are `event_driven`, `fixed_grid`, and
+`exact_horizon_measured`. `fixed_grid` requires `grid`;
+`exact_horizon_measured` requires `horizon` and accepts a `tolerance`. An
+optional rowset `time_range` overrides the manifest range for that strategy.
+Use `rowset_splits` instead of `rowset` when training needs named strategies
+such as `support`, `validation`, and `test`; it is mutually exclusive with the
+single `rowset`. Split ranges must not overlap and must cover every emitted
+target row. A split-level `fixed_grid.grid` is an explicit assertion and must
+equal `DatasetManifest.time_range.grid`; it cannot silently resample one split.
+
+`rowset_splits` defines labels inside one dataset contract. It does not by
+itself publish three DatasetManifests or three DatasetArtifacts. A build emits
+one canonical Feature dataset and one Parameter dataset, with `rowset_split`
+identifying each candidate row. Publish separate manifests only when separate
+artifact identities, permissions, retention, or build lifecycles are required.
+
+### Endpoint eligibility and membership
+
+Use `endpoint_policy` when the platform must retain the complete canonical
+rowset while declaring which target endpoints are safe for model training or
+evaluation. Candidate construction and eligibility evaluation are two stages:
+
+```text
+time_range + rowset_splits -> candidate target rows and rowset labels
+candidate rows + endpoint_policy -> eligibility membership and reasons
+```
+
+The policy is scoped to named rowsets, so one manifest can apply it to
+`validation` and `test` while leaving `training` as pass-through:
+
+```json
+{
+  "prediction": {"horizon": "2h"},
+  "rowset_splits": {
+    "training": {
+      "strategy": "fixed_grid",
+      "grid": "10min",
+      "time_range": {
+        "start": "2025-01-01T00:00:00+08:00",
+        "end": "2026-01-01T00:00:00+08:00",
+        "grid": "10min"
+      }
+    },
+    "validation": {
+      "strategy": "fixed_grid",
+      "grid": "10min",
+      "time_range": {
+        "start": "2026-01-01T00:00:00+08:00",
+        "end": "2026-04-01T00:00:00+08:00",
+        "grid": "10min"
+      }
+    },
+    "test": {
+      "strategy": "fixed_grid",
+      "grid": "10min",
+      "time_range": {
+        "start": "2026-04-01T00:00:00+08:00",
+        "end": "2026-06-01T00:00:00+08:00",
+        "grid": "10min"
+      }
+    }
+  },
+  "abnormal_windows": {
+    "policy": "mark_only",
+    "windows": [
+      {
+        "start": "2026-03-20T00:00:00+08:00",
+        "end": "2026-03-28T00:00:00+08:00",
+        "reason": "shutdown"
+      }
+    ]
+  },
+  "endpoint_policy": {
+    "scope": ["validation", "test"],
+    "policy": "exclude_target_or_input_overlap",
+    "anchor": "cutoff",
+    "context_lookback": "28h",
+    "window_source": "abnormal_windows",
+    "membership_column": "endpoint_eligible",
+    "emit_reason": true
+  }
+}
+```
+
+`endpoint_policy` requires `prediction.horizon`, at least one abnormal window,
+and `abnormal_windows.policy=mark_only`. Every `scope` name must exist in
+`rowset_splits`; without splits, the only valid scope is `default`. The
+membership column must not collide with reserved or materialized dataset
+columns.
+
+Do not derive `context_lookback` from an Operator automatically. Operator
+history requirements answer how much source history is needed to compute a
+Feature. `endpoint_policy.context_lookback` answers how much model input context
+must be free of abnormal observations before an endpoint is usable. They can be
+different and are versioned independently in the manifest contract.
+
+For target time `t`, prediction horizon `h`, context lookback `L`, and abnormal
+window `[a, b)`, evaluation uses:
+
+```text
+cutoff c        = t - h
+model context   = [c - L, c]
+target overlap  = a <= t < b
+context overlap = c >= a and c - L < b
+eligible        = not (target overlap or context overlap), within policy scope
+```
+
+The abnormal window is half-open and the cutoff side is closed. Its effective
+exclusion tail ends at `b + h + L`; a target exactly at that instant is
+eligible. A scoped row receives `target_in_abnormal_window`,
+`input_context_overlap`, or both in `reason_codes`. Overlapping abnormal windows
+do not duplicate a reason code, but all matching `window_id` values are retained.
+
+The build does not delete ineligible rows. It writes
+`rowset_membership.parquet` with one row per candidate target:
+
+| Column | Meaning |
+|-|-|
+| `target_time` | Candidate model target time. |
+| `cutoff_time` | `target_time - prediction.horizon`. |
+| `rowset_name` | Named split, or `default` without `rowset_splits`. |
+| `candidate` | Always true for a row emitted by the current candidate resolver. |
+| `policy_applied` | Whether `rowset_name` belongs to `endpoint_policy.scope`. |
+| `eligible` | Overlap result; pass-through rows outside scope remain true. |
+| `reason_codes` | Stable list of exclusion reasons; empty when eligible or `emit_reason=false`. |
+| `matched_window_ids` | Stable IDs of all matching abnormal windows. |
+| `effective_exclusion_end` | Latest matched window end plus horizon and context lookback. |
+| `policy_hash` | Hash of the resolved endpoint policy and normalized windows. |
+| `manifest_hash` | Dataset contract identity used for the build. |
+
+For compatibility, `feature_dataset.parquet` and `parameter_dataset.parquet`
+also receive the configured `membership_column`. Its value is:
+
+```text
+endpoint_eligible = policy_applied AND eligible
+```
+
+Consequently, a row outside `scope` is a pass-through membership row with
+`eligible=true`, but its compatibility column is false. Use the membership
+artifact when consumers need to distinguish "not evaluated" from "evaluated
+and rejected"; do not interpret the compatibility column alone as a global
+data-quality flag.
+
+The resolved manifest, `quality_report.json`, `validation.json`, and lineage
+record the resolved policy and counts. Reconcile at least
+`candidate_rows`, `policy_applied_rows`, `evaluated_eligible_rows`,
+`ineligible_rows`, `pass_through_rows`, `reason_counts`, and per-rowset counts.
+`eligible_rows` includes true pass-through rows, whereas
+`evaluated_eligible_rows` counts only scoped eligible rows. Single and
+partitioned builds of the same manifest must produce equivalent merged
+membership, counts, lineage, and `policy_hash`.
+
+For the EXP301 validation contract, `h=2h` and `L=28h`, so the exclusion tail is
+30 hours after each abnormal-window end. The fixed 10-minute validation rowset
+contains 20,853 candidates: 19,263 are evaluated eligible and 1,590 are
+ineligible. Treat those figures as an example-specific regression contract,
+not platform defaults.
+
+`endpoint_policy` selects historical dataset rows and is not applied to
+`POST /inference-data/fetch`. Realtime inference reports it as not applicable
+and omits it from the replay manifest.
+
+The current public schema still records `output.engine=chronon`, while the first-phase bridge executes Python Feature Operators and does not run Chronon compile/backfill. Report this limitation accurately.
+
+## Catalog
+
+```json
+{
+  "schema_version": "ml_data_platform.catalog/v1",
+  "name": "pressure_feature_catalog",
+  "version": "v1",
+  "parameters": ["parameters/hot_blast_pressure.v1.json"],
+  "operators": [
+    {
+      "spec": "operators/pressure_features.v1.json",
+      "package": "operator_package/dist/pressure_features-1.0.0-py3-none-any.whl"
+    }
+  ],
+  "features": ["features/pressure_mean_5m.v1.json"],
+  "feature_sets": ["feature_sets/pressure_core.v1.json"],
+  "datasets": ["datasets/pressure_training_20260701_20260702.v1.json"]
+}
+```
+
+All paths must be unique, relative, remain inside the catalog directory, exist, and have the expected `.json` or `.whl` suffix.
+
+## Cross-Object Validation
+
+Require:
+
+- every Feature input Parameter to exist in the target Registry or the same catalog;
+- every Feature Operator to exist and declare `type=feature`;
+- every Feature input to be allowed by the Operator input schema when that schema declares parameters;
+- every Feature physical output to be declared by the Operator output schema;
+- every FeatureSet reference to exist;
+- every Dataset FeatureSet and preprocess Operator to exist;
+- every Dataset relationship input Parameter to exist;
+- every new immutable version to differ by key from conflicting registered content.