@zhuoyuezs/ml-platform 0.2.8 → 0.2.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/CI-ARTIFACT.md +2 -2
  2. package/checksums.json +74 -54
  3. package/package.json +1 -1
  4. package/release.json +19 -19
  5. package/runtime/business-client/README.md +64 -4
  6. package/runtime/business-client/package-lock.json +2 -2
  7. package/runtime/business-client/package.json +1 -1
  8. package/runtime/business-client/src/catalog.js +25 -6
  9. package/runtime/business-client/src/cli.js +177 -7
  10. package/runtime/business-client/src/http.js +192 -32
  11. package/skills/data-source-exploration/SKILL.md +12 -1
  12. package/skills/data-source-exploration/agents/openai.yaml +2 -2
  13. package/skills/data-source-exploration/references/data-platform.md +83 -0
  14. package/skills/feature-management/SKILL.md +29 -11
  15. package/skills/feature-management/agents/openai.yaml +2 -2
  16. package/skills/feature-management/references/commands.md +64 -7
  17. package/skills/feature-management/references/contracts.md +124 -6
  18. package/skills/feature-management/references/data-platform-mapping.md +104 -0
  19. package/skills/feature-management/references/operator-authoring.md +116 -8
  20. package/skills/feature-management/references/platform-capability-guide.md +6 -0
  21. package/skills/feature-management/references/supervised-datasets.md +13 -0
  22. package/skills/model-deployment-management/SKILL.md +7 -0
  23. package/skills/model-deployment-management/references/contracts.md +6 -0
  24. package/skills/model-deployment-management/references/operations.md +28 -1
  25. package/skills/model-deployment-management/references/troubleshooting.md +8 -0
  26. package/skills/model-lifecycle-management/SKILL.md +27 -5
  27. package/skills/model-lifecycle-management/agents/openai.yaml +2 -2
  28. package/skills/model-lifecycle-management/references/ablation.md +111 -0
  29. package/skills/model-lifecycle-management/references/discovery.md +18 -3
  30. package/skills/model-lifecycle-management/references/evaluation.md +188 -1
  31. package/skills/model-lifecycle-management/references/experiment-runs.md +59 -0
  32. package/skills/model-lifecycle-management/references/model-comparison.md +11 -9
  33. package/skills/model-lifecycle-management/references/packaging.md +46 -0
  34. package/skills/model-lifecycle-management/references/training-contracts.md +172 -1
  35. package/skills/model-lifecycle-management/references/training.md +60 -0
  36. package/skills/prediction-report-management/SKILL.md +43 -8
@@ -7,13 +7,25 @@ Read this file whenever a workflow creates or changes executable Operator code.
7
7
  1. Package contract
8
8
  2. Feature entrypoint contract
9
9
  3. Causal calculation rules
10
- 4. Required tests
11
- 5. Build and publication
12
- 6. Security boundary
10
+ 4. Online period aggregates
11
+ 5. Required tests
12
+ 6. Build and publication
13
+ 7. Security boundary
13
14
 
14
15
  ## Package Contract
15
16
 
16
- Build a standalone pure-Python wheel. Do not import report-local, experiment-local, or unregistered business packages. Declare runtime dependencies explicitly, but keep the package small and compatible with `py3-none-any`.
17
+ Build a standalone pure-Python wheel (`py3-none-any`). The platform verifies
18
+ the wheel digest and only extracts it into an isolated import path; it does not
19
+ install `Requires-Dist` dependencies, run `pip`, or accept native code. At run
20
+ time an Operator can import only the Python standard library, its own wheel,
21
+ and packages already present in the image that executes it: dataset builds
22
+ (the `dataset-chronon` dependency group, which provides `pandas`, `numpy` via
23
+ `pandas`, `pyarrow` and `pydantic`) and API realtime reads (the `api`
24
+ dependency group). Do not import report-local, experiment-local, or
25
+ unregistered business packages, and do not depend on libraries such as `scipy`
26
+ or `scikit-learn` that those images do not provide; vendor small pure-Python
27
+ helpers into the wheel instead. `[project].dependencies` documents the expected
28
+ environment for local tests only; it does not make a dependency available.
17
29
 
18
30
  Minimal `pyproject.toml`:
19
31
 
@@ -63,9 +75,19 @@ computation_hash
63
75
 
64
76
  Each `metric_frames[parameter_key]` is a standard long frame with source timestamps and values. Resolve frames from `context.inputs`; do not reach into a database, API, filesystem secret, or environment credential.
65
77
 
66
- Use `context.cutoff_times` for causal feature windows. It is derived once from
67
- `context.target_times - context.prediction_horizon` using the DatasetManifest
68
- prediction contract; do not parse a duplicated Feature-level horizon.
78
+ Use `context.cutoff_times` for causal feature windows. `context.target_times`
79
+ are the materialized rows' `event_time` values, which are the prediction
80
+ decision times, and `context.cutoff_times` equals them: a row's feature
81
+ cutoff is its own `event_time`. The DatasetManifest `prediction.horizon` moves
82
+ label availability to `label_time = event_time + horizon` (and the realtime
83
+ response `target_time = cutoff_time + horizon`); it does not move the Operator
84
+ context's feature reads earlier. Do not subtract `context.prediction_horizon`
85
+ from either index, and do not parse a duplicated Feature-level horizon. This
86
+ rule covers Operator inputs only: a DatasetManifest `endpoint_policy` evaluates
87
+ abnormal-window context overlap with its own `cutoff = t - h` formula, described
88
+ in [contracts.md](contracts.md). Resolved lineage records this as
89
+ `prediction_time_semantics: event_time_is_causal_cutoff` and
90
+ `label_time_derivation: prediction_time_plus_horizon`.
69
91
 
70
92
  An Operator requires an explicit forecast contract whenever its formula or code
71
93
  uses `context.cutoff_times`, or any `input_schema.history_requirements` entry uses
@@ -100,7 +122,11 @@ One Operator may return multiple physical columns. Each public Feature still bin
100
122
 
101
123
  ## Causal Calculation Rules
102
124
 
103
- State every time rule in tests and descriptions:
125
+ State every time rule in tests and descriptions. A formula that needs the
126
+ complete requested window rather than a bounded causal lookback (for example
127
+ full-window interpolation or smoothing) is not partition-invariant; document
128
+ that the Dataset must be built with `--partition-duration none
129
+ --max-parallelism 1` (see [commands.md](commands.md)).
104
130
 
105
131
  - forecast horizon and cutoff derivation;
106
132
  - whether an event exactly at cutoff is included (`<=`) or excluded (`<`);
@@ -113,6 +139,72 @@ State every time rule in tests and descriptions:
113
139
 
114
140
  Never infer a rule from a Feature name alone. Require a business decision when a boundary is unspecified.
115
141
 
142
+ ## Online Period Aggregates
143
+
144
+ Use this method when one Feature row summarizes a whole period (for example a
145
+ day, shift or batch window) and the Feature must also be served online through
146
+ `fetch-inference-data` or a model's `feature_lookup`. The realtime path gives
147
+ the Operator exactly one target row, `event_time = cutoff`, and only source
148
+ rows with `timestamp <= cutoff`, so the batch and online values agree only when
149
+ the formula is causal and bounded.
150
+
151
+ Row semantics:
152
+
153
+ - Anchor the row at the period end: the row at cutoff `T` summarizes the
154
+ period that ended at `T` (for a daily Feature, `T` is local midnight and the
155
+ period is the previous day). Never label a row with the period start and
156
+ aggregate data after it.
157
+ - Decide the period boundary from the source timestamp meaning, which must be
158
+ confirmed by the data owner (see [contracts.md](contracts.md), realtime
159
+ policy questions). If an hourly record is stamped at the hour start, the
160
+ record stamped `T` belongs to the next period, so use `[T - period, T)`. If it
161
+ is stamped at the hour end, the record stamped `T` closes the period, so use
162
+ `(T - period, T]`. Test the record exactly at `T` either way.
163
+
164
+ Contract declarations:
165
+
166
+ - `supports_online: true` on the Operator and `offline_online_supported: true`
167
+ on each Feature that will be served online.
168
+ - `input_schema.history_requirements` with `anchor: cutoff` and a lookback that
169
+ covers the aggregation period, any multi-period context (for example a
170
+ 3-day rolling statistic), and alignment slack for source timestamps. For
171
+ example, a daily mean that also uses the two previous days needs at least
172
+ `3d` plus slack. This lookback is also the per-Parameter realtime read
173
+ window; the default realtime floor `min_lookback=1h` is far too short for a
174
+ period aggregate, and it is not a substitute for the declaration.
175
+ - `input_schema.prediction` with `required`, `minimum_horizon` and
176
+ `maximum_horizon` (a strict nowcast uses `true`, `0min`, `0min`).
177
+ - Do not use `anchor: target_time` with zero lookback for a period aggregate;
178
+ online that reads almost nothing.
179
+
180
+ Formula rules:
181
+
182
+ - Use only rows inside the declared lookback that are at or before the row's
183
+ cutoff. A row's value must not change when data after its cutoff is added,
184
+ or when history older than the lookback is added or removed.
185
+ - Freeze fitted constants in Feature `config`. Values fitted on training data,
186
+ such as fallback medians, clipping limits or normal ranges, must be computed
187
+ once offline, written into `config`, and versioned with the Feature. Do not
188
+ recompute them from the window at run time: the online window holds only
189
+ the lookback, so a recomputed value differs between batch and online and
190
+ between partitions. A retrain that refits these constants needs a new
191
+ Feature (and usually Operator) version.
192
+ - Fill gaps only forward in time, and bound each fill by a maximum age or a
193
+ frozen constant. Never backward-fill from an observation after the row's
194
+ cutoff. Avoid backward-filling the start of the window too: the result then
195
+ depends on where the window starts and breaks partition invariance.
196
+ - The Operator owns interior gap handling. The platform's realtime tolerances
197
+ (`allow_tail_edge_missing`, `allow_internal_gap_fill`) only judge how far the
198
+ newest source row lags the cutoff; they do not insert values. Keep one
199
+ missing-data rule, inside the Operator or a declared Parameter rule, so batch
200
+ builds and realtime reads compute the same value.
201
+ - Emit data-quality flags as their own output columns, each bound to its own
202
+ Feature: for example a valid-observation count, an "insufficient data" flag,
203
+ or a business state flag such as "plant stopped". A flag tells consumers
204
+ that a row was computed under a fallback rule; do not hide that by silently
205
+ filling the value. Include a flag in the model FeatureSet only if it is an
206
+ intended model input.
207
+
116
208
  ## Required Tests
117
209
 
118
210
  Use deterministic fixtures and cover each applicable item with a separately
@@ -131,6 +223,22 @@ requested-output, event-time-order, or dtype/rounding cases below:
131
223
  9. output dtype and rounding;
132
224
  10. every dated business-rule branch.
133
225
 
226
+ For an online-capable Operator (`supports_online: true`), also add:
227
+
228
+ 11. cutoff invariance: adding source rows after the cutoff leaves the row
229
+ unchanged;
230
+ 12. lookback invariance: adding or removing history older than the declared
231
+ lookback leaves the row unchanged;
232
+ 13. partition invariance: computing many cutoffs in one call equals computing
233
+ each cutoff alone with only its lookback window, which is what a realtime
234
+ read does;
235
+ 14. fallback rows: a period with no or too few valid observations produces the
236
+ documented fallback value and quality flag, using the frozen constants.
237
+
238
+ After publication, compare `fetch-inference-data` at a few historical cutoffs
239
+ with the same rows of a batch DatasetArtifact. Any difference means the
240
+ Operator, its fill rule or its lookback is not causal; fix it before training.
241
+
134
242
  Run package tests before building the wheel. Then inspect the wheel tag and ensure it is `py3-none-any`.
135
243
 
136
244
  ## Build And Publication
@@ -14,6 +14,12 @@ replay metadata, and data Job execution. This Skill covers that data subsystem;
14
14
  use the separate model lifecycle Skill for training, evaluation and packaging.
15
15
  Model serving, source-table creation and Kubernetes administration are outside
16
16
  this Skill's scope. Do not present a Skill boundary as a platform capability gap.
17
+ Parameters may read through the platform's direct database adapters or a pinned,
18
+ published external data-platform Mapping. Mapping discovery and source query
19
+ semantics remain owned by `$data-source-exploration`; binding its approved outputs
20
+ to ordinary Parameters is documented in
21
+ [data-platform-mapping.md](data-platform-mapping.md). Kubernetes Worker image and
22
+ Secret configuration remain platform-operator responsibilities.
17
23
 
18
24
  The following distinctions are part of the platform contract:
19
25
 
@@ -151,6 +151,19 @@ For a manifest split named `train`, use `["train"]`. Hand off the exact artifact
151
151
  identity, DataSchema hash, FeatureSet order, label/temporal contracts, observed
152
152
  split counts and binding together.
153
153
 
154
+ When the same immutable DatasetArtifact must be evaluated with different date
155
+ boundaries, use the TrainingRun level derived binding instead of rebuilding the
156
+ artifact. The supported forms are `mode: derived`, with `strategy: time_range`
157
+ or `chronological_ratio`. Time ranges use timezone aware half open
158
+ ranges for `train`, `validation` and optional `test`. The resolver freezes row identities and uses
159
+ the prediction time column as the sample start. It removes rows whose
160
+ `label_time` crosses that role's end boundary and reports the count; it never
161
+ moves or rewrites the label. `test` remains an independent evaluation role,
162
+ `chronological_ratio` keeps equal prediction timestamps together. Actual counts
163
+ can differ from requested ratios. The ordinary
164
+ `assignment_column` form remains the compatibility path for artifacts that
165
+ already contain a split column.
166
+
154
167
  ## Evidence checkpoints
155
168
 
156
169
  SQL-source `verifiable=false` means the automatic clock analysis could not
@@ -35,6 +35,9 @@ deployment does not reach `READY` or an operation is rejected.
35
35
  the model capacity and sampling grid. In `feature_lookup` mode, never send
36
36
  `future_covariates`; the platform retrieves the complete known-future Feature
37
37
  window visible at the cutoff. Inline mode still supplies those rows in inputs.
38
+ - Clustering deployments take no `inference_policy`, `horizon`, or
39
+ `output_window`; an optional `include_inputs: true` echoes the model input
40
+ row. See the clustering request in [references/operations.md](references/operations.md).
38
41
  - Run `validate-deployment` before `create-deployment`. Validation is read-only
39
42
  and does not authorize creation.
40
43
  - Obtain explicit authorization for the exact deployment and operation before
@@ -43,6 +46,10 @@ deployment does not reach `READY` or an operation is rejected.
43
46
  returned an ambiguous result.
44
47
  - Treat `READY` as valid only when `ReplicasReady`, `ArtifactsVerified`, and
45
48
  `SmokePredictionPassed` are all true for the current revision.
49
+ - A READY staging revision can use unified prediction and scheduled reporting.
50
+ Do not promote merely to enable either workflow. Promote only when the user
51
+ requests routed canary/production traffic; unified prediction selects the
52
+ deployment's current revision and does not split traffic by canary weight.
46
53
  - Use `predict-deployment` for governed deployment verification. Do not infer
47
54
  Kubernetes Service names from a deployment ID.
48
55
  - Treat every prediction response `target_time` as China Standard Time in RFC
@@ -72,6 +72,12 @@ continues to carry them in `inputs.future_covariates`.
72
72
  The first deployment starts at `staging`. A replacement for an existing
73
73
  service/environment declares the active leaf in `previous_deployment_id`.
74
74
  Production is reached only by staging to canary to production promotion.
75
+ This stage sequence governs routed rollout, not eligibility for unified prediction
76
+ or reporting: a READY staging revision is sufficient for those workflows.
77
+ New promotion requires `service_exposure.type=Ingress` or an explicit
78
+ `rollout.gateway_name`. NodePort alone does not provide weighted routing.
79
+ The server checks routing prerequisites before persisting the promoted revision;
80
+ a rejected preflight preserves the existing revision and its READY state.
75
81
 
76
82
  `service_exposure` is optional and defaults to `{"type":"ClusterIP"}`. Set
77
83
  `type=NodePort` only when direct node access is required. `node_port` is
@@ -39,7 +39,11 @@ ml-platform --profile server list-deployments \
39
39
  --model-artifact-id <artifact-id>
40
40
  ```
41
41
 
42
- The filters may be used independently and combined with `--service`,
42
+ 列表默认按 `deployment_id` 升序;需要按生命周期时间排序时使用
43
+ `--sort-by created_at|updated_at --sort-order asc|desc`。
44
+
45
+ Use `--service TEXT` for a case-insensitive service-name substring; surrounding
46
+ whitespace is ignored and an empty value does not filter. The filter may be combined with
43
47
  `--environment`, `--state`, `--q`, `--limit`, and `--offset`. These results are
44
48
  ITSMP platform deployments; external model-center delivery callbacks are a
45
49
  separate status surface.
@@ -74,6 +78,20 @@ exposure mode.
74
78
 
75
79
  ## Promotion And Approval
76
80
 
81
+ Skip this step for ordinary unified prediction or scheduled reports on a READY
82
+ staging deployment. For weighted traffic, use an environment's existing
83
+ ingress-nginx with `service_exposure.type=Ingress`, or an explicitly bound,
84
+ ready Gateway via `rollout.gateway_name`. Inspect the immutable DeploymentSpec
85
+ first; changing exposure requires a reviewed replacement deployment. Do not
86
+ install a Gateway merely because the default ClusterIP or NodePort deployment
87
+ cannot be promoted.
88
+
89
+ Promotion performs Kubernetes routing dry-run admission before saving a new
90
+ revision. This checks API permissions and admission, not live ingress-controller
91
+ health or end-to-end reachability. Continue observing reconciliation and READY
92
+ conditions after an accepted promotion. Canary weights apply to the routed
93
+ entrypoint; `predict-deployment` directly verifies the selected revision.
94
+
77
95
  ```bash
78
96
  ml-platform --profile server promote-deployment <deployment-id> \
79
97
  --stage canary --actor <actor> --reason <reason>
@@ -157,6 +175,15 @@ ml-platform --profile server predict-deployment <deployment-id> \
157
175
  /absolute/path/request.json
158
176
  ```
159
177
 
178
+ For a clustering model (`task_type: clustering` in `GET /v1/model`), send no
179
+ `horizon` or `output_window`; they are rejected. The single result is
180
+ `prediction: {kind: "clustering", cluster_id, distance_to_center |
181
+ cluster_probability}` without `target_time`. Add `"include_inputs": true` to
182
+ receive `results[0].inputs`, the validated model input row in signature column
183
+ order (before the model's own transform); it defaults to `false`, is rejected
184
+ for forecasting models, and requires a package image that supports it. Missing
185
+ or non-finite retrieved inputs fail the request.
186
+
160
187
  For a package with an explicit `availability_boundary`, inspect
161
188
  `lineage.visibility` in the prediction response. It records the package's
162
189
  boundary, `cutoff_time`, `attempt_started_at`, `knowledge_time`,
@@ -10,6 +10,14 @@ ml-platform --profile server get-deployment-events <deployment-id> \
10
10
 
11
11
  Interpret conditions before considering a reconcile:
12
12
 
13
+ - A rejected promotion mentioning `rollout.gateway_name`, Gateway HTTP status,
14
+ or routing RBAC leaves the current revision unchanged. Check it with
15
+ `get-deployment`; if staging is READY, unified prediction and reporting remain
16
+ usable. Use existing Ingress for intentional canary traffic, or correct the
17
+ explicit Gateway binding and permissions. Do not repeatedly promote to make
18
+ ordinary prediction work. Historical FAILED promotion revisions are not
19
+ automatically repaired by this guard; inspect the active replacement before
20
+ planning recovery.
13
21
  - `ReplicasReady=false`: inspect scheduler, image pull, resource quota, and Pod
14
22
  readiness through the platform/cluster diagnostic boundary.
15
23
  - `ArtifactsVerified=false`: the Serving `/model` identity does not match the
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: model-lifecycle-management
3
- description: Manage ITSMP system-model discovery, model training, evaluation, immutable model packaging, and contract-bound inference requests. Use when a user asks to inspect platform-owned SystemModels and runtime bindings, validate or submit a training run, inspect/retry/cancel training jobs, configure or run governed evaluation, inspect evaluation evidence, validate a ModelArtifact, create and inspect a ModelPackage, or invoke a deployed model. Do not use for feature catalog authoring, dataset builds, model deployment, Kubernetes administration, or image release engineering.
3
+ description: Manage ITSMP system-model discovery, model training, evaluation, controlled model comparison and feature/input ablation, immutable model packaging, and contract-bound inference requests. Use when a user asks to inspect platform-owned SystemModels and runtime bindings, validate or submit a training run, run a baseline/variant ablation experiment, inspect/retry/cancel training jobs, configure or run governed evaluation, inspect evaluation evidence, validate a ModelArtifact, create and inspect a ModelPackage, or invoke a deployed model. Do not use for feature catalog authoring, dataset builds, model deployment, Kubernetes administration, or image release engineering.
4
4
  ---
5
5
 
6
6
  # Model Lifecycle Management
@@ -31,10 +31,32 @@ resources or selecting runtime bindings. It also defines Job/package preflight
31
31
  and version-skew handling.
32
32
 
33
33
  For multiple-model experiments or iterative optimization, read
34
- [references/model-comparison.md](references/model-comparison.md). This method
35
- uses platform CLI contracts and works without an ITSMP TUI or a specific Agent
36
- Runtime. An execution host may additionally persist plans, method hashes,
37
- attempts and checkpoints; those records do not replace platform evidence.
34
+ [references/model-comparison.md](references/model-comparison.md) and
35
+ [references/experiment-runs.md](references/experiment-runs.md). Use the
36
+ platform's durable `ExperimentRun` to associate candidates and reconcile jobs.
37
+ The method still comes from this Skill; ExperimentRun does not select a winner
38
+ or deploy a model.
39
+
40
+ For feature or input ablation experiments, also read
41
+ [references/ablation.md](references/ablation.md). Ablation is a comparison
42
+ protocol represented by `ExperimentRun` and linked to ordinary TrainingRun and
43
+ EvaluationRun resources; there is no separate `AblationRun` API.
44
+
45
+ For unsupervised clustering (`task.kind=clustering`, for example
46
+ `kmeans_clusterer` or `gaussian_mixture_clusterer`), read the clustering
47
+ section of [references/training-contracts.md](references/training-contracts.md)
48
+ for the Run, parameter and model-file contract, and the clustering section of
49
+ [references/packaging.md](references/packaging.md) for packaging and
50
+ single-row inference without a horizon. The cluster count and preprocessing
51
+ have silent defaults (8 clusters for KMeans, no scaling or PCA). Ask the user
52
+ for them, or select them with the documented method in that reference, before
53
+ submitting a clustering Run.
54
+
55
+ For clustering evaluation and registration, read the clustering section of
56
+ [references/evaluation.md](references/evaluation.md). Use a `task_type:
57
+ "clustering"` EvaluationConfig with `targets: ["cluster_id"]`, and label
58
+ `all`/`train` rowset results as in-sample. Register only with one PASS,
59
+ COMPLETE EvaluationResult for that artifact.
38
60
 
39
61
  Read [references/zero-shot-inference.md](references/zero-shot-inference.md) when
40
62
  the user wants synchronous zero-shot forecasting from JSON or CSV. This path
@@ -1,4 +1,4 @@
1
1
  interface:
2
2
  display_name: "模型训练、评估与打包"
3
- short_description: "管理 ITSMP 模型训练、评估与不可变模型打包流程"
4
- default_prompt: "使用 $model-lifecycle-management 管理模型训练、评估或打包流程。"
3
+ short_description: "管理 ITSMP 模型训练、评估、比较与消融实验"
4
+ default_prompt: "使用 $model-lifecycle-management 管理模型训练、评估、比较或消融实验流程。"
@@ -0,0 +1,111 @@
1
+ # Feature and input ablation
2
+
3
+ Use this method when the question is whether a feature, feature group,
4
+ covariate, input channel or preprocessing input contributes to a model result.
5
+ It is a controlled comparison protocol represented by an `ExperimentRun`. The
6
+ run links existing `TrainingRun`, `ModelArtifact`, `EvaluationRun` and
7
+ `EvaluationResult` facts. Read [experiment-runs.md](experiment-runs.md) for
8
+ execution and [model-comparison.md](model-comparison.md) for evidence rules.
9
+
10
+ ## Freeze the baseline scope
11
+
12
+ Define one baseline candidate with the complete approved input FeatureSet and
13
+ freeze these values for the whole ablation round:
14
+
15
+ - the exact `DatasetArtifact` and `manifest_hash`;
16
+ - DataSchema, label columns, prediction/label time semantics and series keys;
17
+ - train/validation/test assignment and rowset population;
18
+ - evaluation config, metric versions, calculation space and slices;
19
+ - SystemModel, runtime release, reproducibility settings and execution policy.
20
+
21
+ The baseline is the full input set for the hypothesis being tested. A model
22
+ name, FeatureSet name or metric value is not enough to identify it. Store the
23
+ baseline and variants in one immutable ExperimentRun spec.
24
+
25
+ ## Define variants
26
+
27
+ Create one candidate per removal or replacement under test. Prefer one-factor
28
+ variants:
29
+
30
+ ```text
31
+ baseline: [load, temperature, pressure, calendar]
32
+ remove_load: [temperature, pressure, calendar]
33
+ remove_weather: [load, pressure, calendar]
34
+ remove_calendar: [load, temperature, pressure]
35
+ ```
36
+
37
+ Keep the retained columns in their declared order. Give every candidate a
38
+ stable candidate ID, a short hypothesis, the exact removed input keys and the
39
+ parent baseline candidate ID in the ExperimentRun spec. Do not change hyperparameters,
40
+ labels, splits or evaluation policy at the same time as the ablated input.
41
+ If several inputs must be removed together for a single hypothesis, name that
42
+ group explicitly and do not interpret the result as the contribution of each
43
+ member separately.
44
+
45
+ For a feature-only ablation, bind each candidate to an immutable registered
46
+ FeatureSet that is an allowed subset/order projection of the same artifact.
47
+ The wide DatasetArtifact normally does not need rebuilding. Use
48
+ `$feature-management` to create or discover those FeatureSet versions; do not
49
+ edit an existing FeatureSet in place. If the hypothesis changes source data,
50
+ operator logic, label materialization, row membership, or feature values in
51
+ the artifact, create a new DatasetArtifact and start a separate comparison
52
+ scope instead of calling it a feature ablation.
53
+
54
+ ## Execute the round
55
+
56
+ 1. Create one ExperimentRun with the immutable dataset identity, evaluation
57
+ contract, baseline candidate and variants. Use
58
+ [experiment-runs.md](experiment-runs.md) for the JSON shape.
59
+ 2. Validate all candidates before submitting any Job. Validation must show each
60
+ candidate FeatureSet, projected input columns, schema hash and runtime
61
+ identity. Validation has no training side effects.
62
+ 3. Submit only within the user's authorized training scope. Reconcile the
63
+ ExperimentRun after uncertain responses; never create a replacement run to
64
+ retry an unknown candidate submission.
65
+ 4. Observe every Job and validate its ModelArtifact. A failed or infrastructure
66
+ Job has no model-quality score and must be reported separately.
67
+ 5. Evaluate every successful artifact on the same frozen validation rowset with
68
+ the same EvaluationConfig. Inspect coverage, required decisions and result
69
+ hashes; do not compute replacement metrics locally.
70
+ 6. Inspect the ExperimentRun comparison on complete, comparable results. Use
71
+ the baseline as the reference and report absolute and relative metric
72
+ deltas. For metrics where lower is better, a positive candidate-minus-
73
+ baseline delta is degradation; follow the MetricDefinition rather than
74
+ assuming direction.
75
+
76
+ Example comparison request:
77
+
78
+ ```json
79
+ {
80
+ "schema_version": "ml_data_platform.evaluation_result_comparison_request/v1",
81
+ "comparison_type": "model",
82
+ "result_ids": ["baseline_result", "remove_load_result"],
83
+ "metrics": ["regression.mae", "regression.rmse"],
84
+ "sort_direction": "ascending"
85
+ }
86
+ ```
87
+
88
+ Do not rank candidates with different rowsets, labels, horizons, metric
89
+ definitions or incomplete coverage. A candidate that fails because of a
90
+ runtime, image or contract problem is inconclusive about the ablated input.
91
+
92
+ ## Interpret and close the round
93
+
94
+ Report, for every candidate, the baseline/variant FeatureSet identities, exact
95
+ removed inputs, TrainingRun/Job/ModelArtifact IDs, EvaluationResult IDs,
96
+ coverage, metrics, deltas and failure evidence. A small metric difference is
97
+ not proof that an input is useless; state the observed tolerance and whether
98
+ the experiment had repeated seeds or uncertainty estimates. If the result is
99
+ inconclusive, keep the input undecided rather than deleting it.
100
+
101
+ Only after validation comparison selects a candidate may the normal held-out
102
+ test procedure in `model-comparison.md` be used. The test result validates the
103
+ locked candidate and must not be used to choose another ablation. Packaging,
104
+ registration and deployment remain separate authorized gates.
105
+
106
+ The round is complete only when every submitted candidate is accounted for as
107
+ complete, failed, pending, unknown or skipped with a reason. Stop when the
108
+ declared ablation set is covered, the authorized training scope is exhausted,
109
+ or a shared data/runtime defect blocks the round. Do not add an automatic
110
+ feature deletion threshold; business owners decide the practical impact and
111
+ tradeoff after reviewing comparable evidence.
@@ -65,6 +65,20 @@ Jobs in lists are immutable specs; use the exact get-job command for refreshed
65
65
  status. Lists do not reconcile or schedule workloads. `get-training-run` returns
66
66
  `training_run` plus `spec_hash`; artifact/package detail returns the object.
67
67
 
68
+ ## Download artifact files
69
+
70
+ ```bash
71
+ ml-platform --profile server download-model-artifact ARTIFACT_ID --out-dir /tmp/model-ARTIFACT_ID [--file runtime.json]
72
+ ml-platform --profile server download-evaluation-result-artifact RESULT_ID --out-dir /tmp/eval-RESULT_ID [--file predictions.parquet]
73
+ ```
74
+
75
+ Both commands download only files indexed by the registry record (all of them
76
+ unless `--file` is repeated), verify each sha256 against that record, and remove
77
+ the output directory on any mismatch. `--project` fails closed when the
78
+ resource belongs to another project. Report the returned `files[].sha256` as
79
+ content evidence; metadata reads alone do not prove file contents. Write
80
+ downloads outside the repository.
81
+
68
82
  ## Read-only preflight
69
83
 
70
84
  ```bash
@@ -97,10 +111,11 @@ Training Run `task`, `data`, `data.artifact`, `features.feature_set`, `splits`,
97
111
  config references, members and execution, plus ModelPackage requests, are also
98
112
  structured. Unknown keys return 422 with `detail[].loc` identifying the field.
99
113
  Keep algorithm-specific `trainer.parameters` governed by the SystemModel
100
- parameter schema. A typed nested contract such as Chronos-2 `lora_config`
114
+ parameter schema. A typed contract such as Chronos-2 `lora_config` or the
115
+ KMeans/Gaussian Mixture clustering parameters (including `preprocessing`)
101
116
  rejects unknown fields; a SystemModel that intentionally advertises
102
- `additionalProperties` remains extensible. Do not infer one policy from the
103
- other.
117
+ `additionalProperties` remains extensible. Read `additionalProperties` from
118
+ the exact SystemModel instead of inferring one model's policy from another.
104
119
 
105
120
  These HTTP checks preserve accepted values and do not rewrite historical stored
106
121
  contracts or hashes. Legacy records remain readable. A previously ignored typo