@zhuoyuezs/ml-platform 0.2.9 → 0.2.11

package/skills/model-lifecycle-management/references/experiment-runs.md

@@ -0,0 +1,59 @@
+# ExperimentRun
+
+`ExperimentRun` is the platform control-plane record for a finite comparison
+over one frozen `DatasetArtifact`. It supports model comparison, feature/input
+ablation and a small candidate matrix while reusing existing TrainingRun,
+TrainingJob, ModelArtifact, EvaluationRun, EvaluationResult and
+EvaluationComparison records.
+
+It does not implement AutoML, a general DAG engine, automatic model selection,
+packaging or deployment. The Skill owns the modeling method and stopping rules;
+the platform owns durable candidate state and reconciliation.
+
+## Lifecycle
+
+Use the installed `ml-platform` CLI:
+
+```text
+ml-platform create-experiment-run SPEC.json
+ml-platform validate-experiment-run RUN_ID
+ml-platform submit-experiment-run RUN_ID
+ml-platform get-experiment-run RUN_ID
+ml-platform list-experiment-candidates RUN_ID --limit 50 --offset 0
+ml-platform reconcile-experiment-run RUN_ID
+ml-platform cancel-experiment-run RUN_ID
+```
+
+Create is idempotent for the same `experiment_run_id` and immutable
+`spec_hash`. Always validate before submit. Validation resolves every candidate's
+TrainingJob snapshot and has no training side effects. Submit is the mutation
+that may create TrainingJob records, subject to `max_parallelism`.
+
+## Candidate contract
+
+The spec must contain at least two candidates, a baseline candidate, one
+DatasetArtifact identity and one evaluation contract. Each candidate carries a
+complete TrainingRun v4 JSON object, a stable `candidate_id`, an optional
+hypothesis and parent candidate, and an execution policy. Candidate FeatureSets
+may be different exact-version subset/order projections of the same wide
+DatasetArtifact. Labels, split membership, temporal semantics, evaluation
+configuration and comparison population remain shared.
+
+## Recovery rules
+
+`UNKNOWN` means the platform cannot prove whether a remote submission happened.
+Reconcile first queries the durable TrainingJob/EvaluationRun records. It may
+resume only when the lifecycle record proves no remote submission was reached;
+it never blindly retries an unknown Kubernetes operation. An unknown candidate
+also stops new candidate submissions until it is reconciled. A known terminal
+failure is preserved and does not consume a parallel slot for independent
+candidates.
+
+## Completion
+
+Every candidate must be `RESULT_READY`, `FAILED`, `SKIPPED` or otherwise
+accounted for. The service calls the existing EvaluationComparison implementation
+only after complete results exist. `COMPLETE` with `ranking_allowed=true` requires
+complete evidence, comparable population and a release-mode EvaluationConfig;
+otherwise the run is `INCONCLUSIVE`. A result never authorizes registration,
+packaging or deployment by itself.

package/skills/model-lifecycle-management/references/model-comparison.md

@@ -1,10 +1,16 @@
 # Comparing models within one experiment
 
+Feature or input ablation is the controlled one-factor variant of this
+workflow. Read [ablation.md](ablation.md) for the baseline, removal-set and
+round-closeout rules; the comparison gates below still apply.
+
 Use one documented goal and user-authorized scope: metric, target, candidate
 constraints and any maximum number of training submissions. Preserve that goal
 when a follow-up changes a constraint or asks for verification. Reuse existing
 authorization when it covers the next action; ask only for missing decisions.
 No TUI, session service, or Runtime-specific tool is required for this method.
+Persist the candidate set and execution links in an `ExperimentRun`; see
+[experiment-runs.md](experiment-runs.md) for its CLI lifecycle.
 
 ## Shared scope and candidate execution
 
@@ -45,9 +51,8 @@ forecast horizon, and do not let trajectory construction cross split
 boundaries. Compare complete platform-produced EvaluationResults only after
 each candidate passes its own TrainingRun and evaluation preflight.
 
-For each candidate, record its hypothesis, exact changed inputs, parent attempt
-if any, request JSON and returned immutable IDs in the agent's working
-directory. Submit within the user's authorization, observe the TrainingJob,
+For each candidate, record its hypothesis, exact changed inputs and parent
+candidate in the ExperimentRun spec. Submit within the user's authorization, observe the TrainingJob,
 validate the resulting ModelArtifact, then submit and inspect governed
 evaluation. Explain the observed result before choosing another candidate.
 Do not resubmit after an unknown outcome: reconcile the existing Job or retry
@@ -57,12 +62,9 @@ the authorization described in the training reference.
 ## Rounds, failure evidence and stopping
 
 A round groups candidates that test a related hypothesis under the frozen
-comparison scope. Keep a small round record in the agent's working directory
-(outside a platform source checkout), using normal file tools. Include the
-scope/request paths, candidate identities and changes, submission IDs, observed
-status, result IDs, per-candidate reflection and the round's next decision.
-This is an agent work record, not a new platform resource or required tool.
-Update it after submissions and observations so another coding agent can resume.
+comparison scope. ExperimentRun and its candidates are the durable execution
+record. Keep any additional method reflection in the agent's working directory
+without duplicating platform status or IDs.
 
 When a candidate fails, inspect the actual Job status, attempts and diagnostic
 evidence. Distinguish a model execution failure from a runtime/infrastructure

package/skills/model-lifecycle-management/references/training.md

@@ -54,6 +54,40 @@ it does not reconstruct it from manifest `rowset_splits`. Save the submitted
 TrainingRun JSON, validation output, Job ID and ModelArtifact ID in the user's
 working directory so later diagnosis needs no source repository.
 
+For a second experiment over the same immutable DatasetArtifact, a TrainingRun
+may freeze a time based split without adding a split column to the artifact:
+
+```json
+{
+  "splits": {
+    "mode": "derived",
+    "strategy": "time_range",
+    "source_rowsets": ["all"],
+    "ranges": {
+      "train": {"start": "2026-01-01T00:00:00+08:00", "end": "2026-07-01T00:00:00+08:00"},
+      "validation": {"start": "2026-07-01T00:00:00+08:00", "end": "2026-08-01T00:00:00+08:00"},
+      "test": {"start": "2026-08-01T00:00:00+08:00", "end": "2026-09-01T00:00:00+08:00"}
+    }
+  }
+}
+```
+
+`time_range` uses timezone aware, left closed/right open prediction time
+intervals and requires unique row keys. The platform freezes each role's row
+identity in the resolved TrainingJob and rechecks it in the Worker. Samples
+whose `label_time` reaches or passes their role's end are excluded and counted
+in the validation snapshot; if that leaves no train samples, validation fails.
+Training time evaluation must explicitly use `train` or `validation`; `test`
+is reserved for an independent EvaluationRun. For `chronological_ratio`, provide
+`ratios` with `train`, `validation` and `test` values summing to `1`; equal
+prediction timestamps stay together. Actual counts may differ from requested
+ratios, and the algorithm version and row identities are frozen in the Job
+snapshot. Derived splits currently use all artifact rows as candidates; use
+`assignment_column` for artifacts containing auxiliary history or truth rows.
+Only `time_range` and `chronological_ratio` are supported. Confirm the selected
+training and evaluation Worker images support derived splits before submission.
+Existing `assignment_column` contracts keep their original behavior.
+
 ## Commands
 
 Read [training-contracts.md](training-contracts.md) when authoring a Run or Job

package/skills/model-lifecycle-management/references/zero-shot-inference.md

@@ -29,6 +29,74 @@ ml-platform --profile server predict-foundation \
 
 The endpoint is `POST /foundation-models/chronos2_forecaster/predict`.
 
+### Complete request shape and preflight
+
+Build the whole request from the versioned schema before calling the CLI. This
+minimal example uses three hourly historical observations and forecasts six
+steps. Replace the timestamps and values with the user's data, keeping the
+timestamp grid aligned to `frequency` and ending exactly at `cutoff_time`.
+
+```json
+{
+  "input": {"transport": "json_records"},
+  "schema": {
+    "schema_version": "ml_data_platform.prediction_input_schema/v1",
+    "layout": "long",
+    "timestamp_column": "timestamp",
+    "series_keys": [],
+    "targets": ["temperature"],
+    "past_covariates": [],
+    "known_future_covariates": [],
+    "frequency": "1h",
+    "timezone": "UTC",
+    "cutoff_time": "2026-01-01T02:00:00Z"
+  },
+  "records": [
+    {"timestamp": "2026-01-01T00:00:00Z", "temperature": 20.1},
+    {"timestamp": "2026-01-01T01:00:00Z", "temperature": 20.3},
+    {"timestamp": "2026-01-01T02:00:00Z", "temperature": 20.2}
+  ],
+  "forecast": {
+    "horizon_steps": 6,
+    "statistics": ["median"]
+  }
+}
+```
+
+Before the one prediction call, verify all of the following:
+
+- `schema.schema_version` is exactly `ml_data_platform.prediction_input_schema/v1`;
+- `input.transport` is `json_records`, and the top-level keys are `input`,
+  `schema`, `records` and `forecast`;
+- every record has exactly the timestamp, series-key, target, past-covariate
+  and known-future-covariate columns declared in `schema`; role names are
+  disjoint;
+- `schema.cutoff_time` is timezone-aware, appears as the final historical
+  timestamp in every series, and all historical timestamps form a regular grid
+  at `schema.frequency` with no duplicates or gaps;
+- historical target values are numeric and non-null; do not include rows after
+  cutoff unless known-future covariates are declared;
+- `forecast.horizon_steps` is a positive integer within the discovered active
+  release's `max_prediction_length`; it counts frequency intervals, not a
+  duration string. There is no `output_window` field in this API contract;
+- request rows and serialized bytes are within the discovered release limits.
+
+When known-future covariates are used, declare them in
+`schema.known_future_covariates`, include their historical values as well, and
+provide exactly one future row for every forecast step. Future target values
+must be null. Do not add future rows when no known-future covariates are
+declared.
+
+If the CLI rejects the request, inspect the complete structured error once and
+compare its named field/value with this contract and the discovered release
+capabilities. Correct only a confirmed mismatch, then make at most one
+corrected retry. Do not guess a schema version, rename fields speculatively, or
+loop through 422s. If the error conflicts with this versioned contract or does
+not identify a correctable input issue, stop and report the response for
+server-side contract investigation. Report success only when the CLI returns a
+successful prediction response; include its request/prediction ID and returned
+release identity when present.
+
 ## CSV prediction
 
 CSV uses the same schema and forecast JSON as the JSON path. The CLI sends one

package/skills/prediction-report-management/SKILL.md

@@ -28,6 +28,9 @@ schedule request or deployment policy. A clustering deployment (its model
 artifact has `output_bindings`) has no horizon: omit `horizon` and
 `output_window`, which are rejected. A report run may be `SKIPPED` when the
 selected revision is not READY.
+A READY staging revision is eligible; reporting does not require canary or
+production promotion, an Ingress, or a Gateway. Do not promote a deployment
+just to enable its report schedule.
 
 ## Configure a destination