@zhuoyuezs/ml-platform 0.1.9 → 0.1.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -3
- package/checksums.json +20 -20
- package/package.json +3 -2
- package/release.json +10 -10
- package/runtime/business-client/README.md +32 -0
- package/runtime/business-client/package-lock.json +2 -2
- package/runtime/business-client/package.json +1 -1
- package/runtime/business-client/src/catalog.js +2 -1
- package/runtime/business-client/src/cli.js +6 -1
- package/scripts/verify-release-package.js +28 -0
- package/skills/feature-management/SKILL.md +14 -3
- package/skills/feature-management/references/contracts.md +37 -2
- package/skills/model-lifecycle-management/SKILL.md +2 -2
- package/skills/model-lifecycle-management/references/packaging.md +4 -2
- package/skills/model-lifecycle-management/references/training-contracts.md +95 -0
- package/skills/model-lifecycle-management/references/training.md +28 -6
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# ML Platform
|
|
2
2
|
|
|
3
|
-
ML Platform 的 JavaScript CLI
|
|
3
|
+
ML Platform 的 JavaScript CLI、`feature-management` 与 `model-lifecycle-management` Agent Skill。CLI 与 Skill 由同一个 npm
|
|
4
4
|
release 安装和升级,运行时不需要 Python 或 `uv`。
|
|
5
5
|
|
|
6
6
|
## 环境要求
|
|
@@ -17,8 +17,8 @@ release 安装和升级,运行时不需要 Python 或 `uv`。
|
|
|
17
17
|
npx @zhuoyuezs/ml-platform@latest install
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
这条命令不需要额外参数。默认安装持久化的 `ml-platform`
|
|
21
|
-
`~/.codex/skills
|
|
20
|
+
这条命令不需要额外参数。默认安装持久化的 `ml-platform` 命令,并将两个 Skill 安装到
|
|
21
|
+
`~/.codex/skills/`,供当前用户的 Codex 使用。安装后重启终端和 Agent 会话。
|
|
22
22
|
|
|
23
23
|
只在当前项目使用 Skill 时,在项目根目录运行:
|
|
24
24
|
|
|
@@ -103,6 +103,10 @@ ml-platform --help
|
|
|
103
103
|
受控构建,不要求本地存在 Catalog。`apply`、`resolve-manifest` 和带本地 JSON 路径的
|
|
104
104
|
`build-dataset` 用于新契约的编写和发布流程。
|
|
105
105
|
|
|
106
|
+
Parameter JSON 可同时声明离线 `source` 和可选的实时 `online_source`。通过
|
|
107
|
+
`ml-platform add-parameter <spec.json>` 或 Catalog `apply` 正常发布即可;客户端原样提交
|
|
108
|
+
字段,离线构建使用 `source`,`fetch-inference-data` 使用 `online_source`。
|
|
109
|
+
|
|
106
110
|
## 迁移旧 Skill
|
|
107
111
|
|
|
108
112
|
目标目录存在旧 Python Skill、实验 symlink 或其他未托管 Skill 时,确认后显式备份:
|
package/checksums.json
CHANGED
|
@@ -2,28 +2,28 @@
|
|
|
2
2
|
"files": [
|
|
3
3
|
{
|
|
4
4
|
"path": "runtime/business-client/README.md",
|
|
5
|
-
"sha256": "sha256:
|
|
6
|
-
"size_bytes":
|
|
5
|
+
"sha256": "sha256:b823c0cafc47478fa1297c7c2e61e15b537db155586dd012b40d678d788c3ce3",
|
|
6
|
+
"size_bytes": 5158
|
|
7
7
|
},
|
|
8
8
|
{
|
|
9
9
|
"path": "runtime/business-client/package-lock.json",
|
|
10
|
-
"sha256": "sha256:
|
|
10
|
+
"sha256": "sha256:b460e297ced5aa90210bc73c7ee2e8b73940674b4e1f62ffe3915fe4024eb089",
|
|
11
11
|
"size_bytes": 381
|
|
12
12
|
},
|
|
13
13
|
{
|
|
14
14
|
"path": "runtime/business-client/package.json",
|
|
15
|
-
"sha256": "sha256:
|
|
15
|
+
"sha256": "sha256:a60a2018c8714b09fcf386601cee82baa9384648526a36f95fbd7480cabcfef5",
|
|
16
16
|
"size_bytes": 501
|
|
17
17
|
},
|
|
18
18
|
{
|
|
19
19
|
"path": "runtime/business-client/src/catalog.js",
|
|
20
|
-
"sha256": "sha256:
|
|
21
|
-
"size_bytes":
|
|
20
|
+
"sha256": "sha256:717df5f630e85da89fe8b315d6dabc6549af7984f93a6167b494e52ee3697803",
|
|
21
|
+
"size_bytes": 53795
|
|
22
22
|
},
|
|
23
23
|
{
|
|
24
24
|
"path": "runtime/business-client/src/cli.js",
|
|
25
|
-
"sha256": "sha256:
|
|
26
|
-
"size_bytes":
|
|
25
|
+
"sha256": "sha256:5a31e407dba79205e56d2ed1e74260213d0b89c6bb5a665ec0f52e92927ebcaf",
|
|
26
|
+
"size_bytes": 44897
|
|
27
27
|
},
|
|
28
28
|
{
|
|
29
29
|
"path": "runtime/business-client/src/config.js",
|
|
@@ -37,8 +37,8 @@
|
|
|
37
37
|
},
|
|
38
38
|
{
|
|
39
39
|
"path": "skills/feature-management/SKILL.md",
|
|
40
|
-
"sha256": "sha256:
|
|
41
|
-
"size_bytes":
|
|
40
|
+
"sha256": "sha256:e5b927611cd401bc5e6160a2d358369efd0c4f65d3fe5cbf21bbab9adb6bd978",
|
|
41
|
+
"size_bytes": 31991
|
|
42
42
|
},
|
|
43
43
|
{
|
|
44
44
|
"path": "skills/feature-management/agents/openai.yaml",
|
|
@@ -97,8 +97,8 @@
|
|
|
97
97
|
},
|
|
98
98
|
{
|
|
99
99
|
"path": "skills/feature-management/references/contracts.md",
|
|
100
|
-
"sha256": "sha256:
|
|
101
|
-
"size_bytes":
|
|
100
|
+
"sha256": "sha256:4e10ba13399de17561a10b1dead4e2fcc57e0962cf81eba5131d12d66c4ce013",
|
|
101
|
+
"size_bytes": 29965
|
|
102
102
|
},
|
|
103
103
|
{
|
|
104
104
|
"path": "skills/feature-management/references/operator-authoring.md",
|
|
@@ -117,8 +117,8 @@
|
|
|
117
117
|
},
|
|
118
118
|
{
|
|
119
119
|
"path": "skills/model-lifecycle-management/SKILL.md",
|
|
120
|
-
"sha256": "sha256:
|
|
121
|
-
"size_bytes":
|
|
120
|
+
"sha256": "sha256:faa4f8c0dbae7d1c1e2ca94499ce9b13cc8b52e8229154c05ac6282ea5e47788",
|
|
121
|
+
"size_bytes": 3610
|
|
122
122
|
},
|
|
123
123
|
{
|
|
124
124
|
"path": "skills/model-lifecycle-management/agents/openai.yaml",
|
|
@@ -137,18 +137,18 @@
|
|
|
137
137
|
},
|
|
138
138
|
{
|
|
139
139
|
"path": "skills/model-lifecycle-management/references/packaging.md",
|
|
140
|
-
"sha256": "sha256:
|
|
141
|
-
"size_bytes":
|
|
140
|
+
"sha256": "sha256:0992f2f9104cb0d65a6325756108211d3f94856e05eca12640e9792c07e1108f",
|
|
141
|
+
"size_bytes": 3250
|
|
142
142
|
},
|
|
143
143
|
{
|
|
144
144
|
"path": "skills/model-lifecycle-management/references/training-contracts.md",
|
|
145
|
-
"sha256": "sha256:
|
|
146
|
-
"size_bytes":
|
|
145
|
+
"sha256": "sha256:b6d72f37adf40c7d3832dbe46645cd416485411dcd48ea7b93f6072d458922c9",
|
|
146
|
+
"size_bytes": 11825
|
|
147
147
|
},
|
|
148
148
|
{
|
|
149
149
|
"path": "skills/model-lifecycle-management/references/training.md",
|
|
150
|
-
"sha256": "sha256:
|
|
151
|
-
"size_bytes":
|
|
150
|
+
"sha256": "sha256:f7bb40919da78b99cad13d4f1f05fef136552409df5ba6e2838e1e6b4a2435a5",
|
|
151
|
+
"size_bytes": 5508
|
|
152
152
|
}
|
|
153
153
|
],
|
|
154
154
|
"schema_version": "data_platform.ml_platform_checksums/v1"
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zhuoyuezs/ml-platform",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.11",
|
|
4
4
|
"description": "ML 数据平台 business CLI 与 Agent Skills 管理工具",
|
|
5
5
|
"bin": {
|
|
6
6
|
"ml-platform": "scripts/main.js"
|
|
@@ -19,7 +19,8 @@
|
|
|
19
19
|
"scripts": {
|
|
20
20
|
"test": "node --test test/*.test.js",
|
|
21
21
|
"api:check": "node ../../scripts/check_api_contract.js",
|
|
22
|
-
"pack": "npm pack"
|
|
22
|
+
"pack": "npm pack",
|
|
23
|
+
"prepublishOnly": "node scripts/verify-release-package.js"
|
|
23
24
|
},
|
|
24
25
|
"engines": {
|
|
25
26
|
"node": ">=18"
|
package/release.json
CHANGED
|
@@ -13,8 +13,8 @@
|
|
|
13
13
|
"entrypoint": "src/cli.js",
|
|
14
14
|
"name": "ml-platform",
|
|
15
15
|
"path": "runtime/business-client",
|
|
16
|
-
"sha256": "sha256:
|
|
17
|
-
"version": "0.7.
|
|
16
|
+
"sha256": "sha256:95558e7845b8a177c6090d4a960640469db658e88d9147ccc9170eadf7ca3410",
|
|
17
|
+
"version": "0.7.1"
|
|
18
18
|
},
|
|
19
19
|
"policy_sha256": "sha256:02fbde0134696b78c0c365d6e0b76c9814227208da5b2b3d4e5dbd82dc4a3606",
|
|
20
20
|
"public_schema_versions": [
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"ml_data_platform.feature_set/v1",
|
|
26
26
|
"ml_data_platform.dataset_manifest/v1"
|
|
27
27
|
],
|
|
28
|
-
"release_version": "0.1.
|
|
28
|
+
"release_version": "0.1.11",
|
|
29
29
|
"runtime_requirements": {
|
|
30
30
|
"node": ">=18",
|
|
31
31
|
"os": [
|
|
@@ -37,16 +37,16 @@
|
|
|
37
37
|
"skills": {
|
|
38
38
|
"feature-management": {
|
|
39
39
|
"path": "skills/feature-management",
|
|
40
|
-
"requires_cli": ">=0.7.
|
|
41
|
-
"revision": "0.1.
|
|
42
|
-
"sha256": "sha256:
|
|
40
|
+
"requires_cli": ">=0.7.1 <0.8.0",
|
|
41
|
+
"revision": "0.1.11",
|
|
42
|
+
"sha256": "sha256:182b6f2df4fe58c946e482dfaf416c7d5b78c6fde9c8c8d7293103246618cb4e"
|
|
43
43
|
},
|
|
44
44
|
"model-lifecycle-management": {
|
|
45
45
|
"path": "skills/model-lifecycle-management",
|
|
46
|
-
"requires_cli": ">=0.7.
|
|
47
|
-
"revision": "0.1.
|
|
48
|
-
"sha256": "sha256:
|
|
46
|
+
"requires_cli": ">=0.7.1 <0.8.0",
|
|
47
|
+
"revision": "0.1.11",
|
|
48
|
+
"sha256": "sha256:61ba17f955b36d1c2e8329a10454d040ee7eca5afdb90ff0914d3f85b6e2d203"
|
|
49
49
|
}
|
|
50
50
|
},
|
|
51
|
-
"source_commit": "
|
|
51
|
+
"source_commit": "374d9935e9a7267ae6485be140a8d99b2af57b44"
|
|
52
52
|
}
|
|
@@ -27,6 +27,10 @@ ml-platform build-registered-dataset <dataset_id> <dataset_version> \
|
|
|
27
27
|
`resolve-manifest` 和 `build-dataset <manifest.json>` 仍用于尚未注册的本地契约。Catalog
|
|
28
28
|
normalizer 保留并校验非默认 `project`;`apply --project` 与 Catalog 内项目不一致时拒绝执行。
|
|
29
29
|
|
|
30
|
+
同一 Parameter 需要从历史库训练、从实时库取数时,在 Parameter JSON 同时填写 `source` 与
|
|
31
|
+
可选的 `online_source`,然后照常使用 `add-parameter` 或 Catalog `apply`;CLI 原样透传该
|
|
32
|
+
字段,不需要额外参数。离线构建读取 `source`,实时因果读取自动选择 `online_source`。
|
|
33
|
+
|
|
30
34
|
Registry 资产详情和版本发现使用与 `dev` OpenAPI 一致的命令:
|
|
31
35
|
|
|
32
36
|
```bash
|
|
@@ -44,6 +48,34 @@ ml-platform get-parameter-lifecycle <name> <version> --project <project>
|
|
|
44
48
|
ml-platform fetch-inference-context /absolute/path/to/request.json
|
|
45
49
|
```
|
|
46
50
|
|
|
51
|
+
已部署模型的标准推理使用 Serving URL,不使用控制面 `--api-url`。Chronos-2 v2 的
|
|
52
|
+
`known_future` 协变量放在 inline 请求的 `inputs.future_covariates` 中;每个预测步一行,
|
|
53
|
+
行数必须等于模型的 `prediction_length`。下例为 15 分钟频率、预测 30 分钟;预测明天时
|
|
54
|
+
同样提供整天的所有未来步:
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"input_mode": "inline",
|
|
59
|
+
"cutoff_time": "2026-09-09T00:00:00+08:00",
|
|
60
|
+
"horizon": [{"duration": "PT30M"}],
|
|
61
|
+
"inputs": {
|
|
62
|
+
"timestamps": ["...", "2026-09-09T00:00:00+08:00"],
|
|
63
|
+
"values": [{"storage_soc": 51.2, "storage_power": 4.0}, {"storage_soc": 52.0, "storage_power": 3.8}],
|
|
64
|
+
"future_covariates": [
|
|
65
|
+
{"weather_temperature": 28.0, "peak_valley": 1},
|
|
66
|
+
{"weather_temperature": 27.5, "peak_valley": 1}
|
|
67
|
+
]
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
ml-platform predict-model /absolute/path/to/request.json \
|
|
74
|
+
--serving-url https://storage-forecast.example
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
训练历史必须使用各个历史 cutoff 当时可取得的天气预报,不得以事后实测天气替代。
|
|
78
|
+
|
|
47
79
|
Parameter、Feature、FeatureSet 的删除命令执行软删除,并支持带 actor、reason 和幂等键的
|
|
48
80
|
恢复;调用方应在任何变更前查询直接与传递引用。
|
|
49
81
|
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zhuoyuezs/ml-platform-business-client",
|
|
3
|
-
"version": "0.7.
|
|
3
|
+
"version": "0.7.1",
|
|
4
4
|
"lockfileVersion": 3,
|
|
5
5
|
"requires": true,
|
|
6
6
|
"packages": {
|
|
7
7
|
"": {
|
|
8
8
|
"name": "@zhuoyuezs/ml-platform-business-client",
|
|
9
|
-
"version": "0.7.
|
|
9
|
+
"version": "0.7.1",
|
|
10
10
|
"license": "UNLICENSED",
|
|
11
11
|
"bin": {
|
|
12
12
|
"ml-platform": "src/cli.js"
|
|
@@ -103,7 +103,7 @@ function normalizeRelationshipRule(value, relationshipId) { const label = `param
|
|
|
103
103
|
function normalizeParameterRelationship(value, parentProject) { exactKeys(value, ["id", "inputs", "alignment", "rules", "description"], ["id", "inputs", "rules"], "parameter relationship"); const id = identifier(value.id, "parameter relationship id"); const inputs = objectList(value.inputs, "parameter relationship inputs").map((item) => { exactKeys(item, ["alias", "parameter", "version", "project"], ["alias", "parameter", "version"], "parameter relationship input"); validateReferenceProject(item, parentProject, "parameter relationship input"); return { alias: identifier(item.alias, "parameter relationship input alias"), parameter: identifier(item.parameter, "parameter relationship input parameter"), version: identifier(item.version, "parameter relationship input version"), project: parentProject }; }); if (inputs.length < 2) throw new Error("parameter relationship requires at least two inputs"); unique(inputs.map((item) => item.alias), "parameter relationship input aliases"); unique(inputs.map(parameterReferenceKey), "parameter relationship inputs"); const alignmentValue = object(defaultIfAbsent(value, "alignment", {}), "parameter relationship alignment"); exactKeys(alignmentValue, ["method", "tolerance"], [], "parameter relationship alignment"); const tolerance = optionalDuration(alignmentValue.tolerance, "parameter relationship alignment tolerance"); if (tolerance != null && durationSeconds(tolerance) === 0) throw new Error("parameter relationship alignment tolerance must be positive"); const alignment = { method: choice(defaultIfAbsent(alignmentValue, "method", "last_before_or_at"), ["exact", "last_before_or_at"], "parameter relationship alignment method"), tolerance }; const rules = objectList(value.rules, "parameter relationship rules").map((item) => normalizeRelationshipRule(item, id)); if (!rules.length) throw new Error("parameter relationship rules must not be empty"); unique(rules.map((item) => item.id), `parameter relationship ${id} rule ids`); const aliases = new Set(inputs.map((item) => item.alias)); for (const rule of rules) { if (rule.stage !== "aligned_grid") throw new Error(`parameter relationship ${id} rule ${rule.id} must use stage=aligned_grid`); const missing = relationshipAliases(rule.check).filter((item) => !aliases.has(item)); if (missing.length) throw new Error(`parameter relationship ${id} rule ${rule.id} references undeclared aliases: ${missing.join(", ")}`); } return { id, inputs, alignment, rules, description: pythonString(defaultIfAbsent(value, "description", "")) }; }
|
|
104
104
|
function normalizeDataset(payload) {
|
|
105
105
|
object(payload, "dataset"); payload = Object.fromEntries(Object.entries(payload).filter(([name]) => name !== "deleted_at"));
|
|
106
|
-
exactKeys(payload, ["schema_version", "project", "dataset_id", "dataset_version", "mode", "read_policy", "snapshot_id", "as_of", "time_range", "parameters", "feature_set", "preprocess", "output", "source_read", "realtime_fetch", "abnormal_windows", "prediction", "target", "parameter_relationships", "parameter_missing_policies", "rowset", "rowset_splits", "endpoint_policy"], ["dataset_id", "time_range", "feature_set"], "dataset");
|
|
106
|
+
exactKeys(payload, ["schema_version", "project", "dataset_id", "dataset_version", "mode", "read_policy", "snapshot_id", "as_of", "time_range", "parameters", "feature_set", "preprocess", "output", "source_read", "realtime_fetch", "abnormal_windows", "prediction", "target", "label_materializations", "parameter_relationships", "parameter_missing_policies", "rowset", "rowset_splits", "endpoint_policy"], ["dataset_id", "time_range", "feature_set"], "dataset");
|
|
107
107
|
const parentProject = identifier(defaultIfAbsent(payload, "project", "default"), "dataset project");
|
|
108
108
|
const readPolicy = choice(defaultIfAbsent(payload, "read_policy", "snapshot"), ["snapshot", "as_of", "latest"], "dataset read_policy");
|
|
109
109
|
const snapshotId = payload.snapshot_id == null ? null : identifier(payload.snapshot_id, "dataset snapshot_id");
|
|
@@ -122,6 +122,7 @@ function normalizeDataset(payload) {
|
|
|
122
122
|
if (missingPolicies.length) output.parameter_missing_policies = missingPolicies;
|
|
123
123
|
for (const name of ["source_read", "realtime_fetch", "abnormal_windows", "prediction", "rowset", "rowset_splits", "endpoint_policy"]) if (payload[name] != null) output[name] = payload[name];
|
|
124
124
|
if (payload.target != null) output.target = normalizeTarget(payload.target, parentProject);
|
|
125
|
+
if (payload.label_materializations != null) output.label_materializations = payload.label_materializations;
|
|
125
126
|
const relationships = objectList(defaultIfAbsent(payload, "parameter_relationships", []), "dataset parameter_relationships").map((item) => normalizeParameterRelationship(item, parentProject)); unique(relationships.map((item) => item.id), "dataset parameter_relationship ids"); if (relationships.length) output.parameter_relationships = relationships;
|
|
126
127
|
return output;
|
|
127
128
|
}
|
|
@@ -28,7 +28,7 @@ const BUSINESS_COMMANDS = new Set([
|
|
|
28
28
|
"add-parameter", "add-feature", "add-feature-set", "add-operator", "publish-operator",
|
|
29
29
|
"add-dataset", "update-dataset", "delete-dataset", "list-dataset-artifacts",
|
|
30
30
|
"register-operator", "show-operator-specs", "delete-operator", "run-operator",
|
|
31
|
-
"get-job", "wait-job", "retry-job", "cancel-job", "get-dataset-artifact", "download-dataset-artifact", "fetch-inference-data", "fetch-inference-context", "apply",
|
|
31
|
+
"get-job", "wait-job", "retry-job", "cancel-job", "get-dataset-artifact", "download-dataset-artifact", "fetch-inference-data", "fetch-inference-context", "predict-model", "apply",
|
|
32
32
|
"register-trainer", "list-trainer-definitions", "get-trainer-definition", "register-training-run", "validate-training-run", "submit-training-job", "get-training-job", "retry-training-job", "cancel-training-job",
|
|
33
33
|
"register-evaluation-config", "get-evaluation-config", "register-metric-definition", "get-metric-definition",
|
|
34
34
|
"register-executable-package", "verify-executable-package", "publish-executable-package",
|
|
@@ -105,6 +105,7 @@ const COMMAND_USAGE = {
|
|
|
105
105
|
"download-dataset-artifact": "download-dataset-artifact DATASET_ID MANIFEST_HASH [--project PROJECT] --out-dir PATH [--file PATH] [--force]",
|
|
106
106
|
"fetch-inference-data": "fetch-inference-data MANIFEST --cutoff-time TIMESTAMP [--max-workers N] [--max-source-lag-hours HOURS] [--no-validate-freshness] [--allow-missing]",
|
|
107
107
|
"fetch-inference-context": "fetch-inference-context REQUEST_JSON",
|
|
108
|
+
"predict-model": "predict-model REQUEST_JSON --serving-url URL",
|
|
108
109
|
"register-trainer": "register-trainer DEFINITION_JSON",
|
|
109
110
|
"list-trainer-definitions": "list-trainer-definitions",
|
|
110
111
|
"get-trainer-definition": "get-trainer-definition NAME VERSION [--project PROJECT]",
|
|
@@ -451,6 +452,10 @@ async function runBusinessCli(argv) {
|
|
|
451
452
|
const request = { manifest: readDataset(manifest), cutoff_time: cutoffTime }; if (Object.keys(realtime).length) request.realtime_fetch = realtime; result = await client(options).post("/inference-data/fetch", request);
|
|
452
453
|
} else if (options.command === "fetch-inference-context") {
|
|
453
454
|
result = await client(options).post("/inference-data/context", readSpec(positional(rest, "request JSON")));
|
|
455
|
+
} else if (options.command === "predict-model") {
|
|
456
|
+
const request = readSpec(positional(rest, "request JSON")); const servingUrl = take(rest, "--serving-url");
|
|
457
|
+
if (!servingUrl) throw new Error("--serving-url is required");
|
|
458
|
+
result = await new PlatformApiClient(servingUrl, options.timeout).post("/v1/inference", request);
|
|
454
459
|
} else if (options.command === "get-model-runtime-release") {
|
|
455
460
|
result = await client(options).get("/model-runtime-release");
|
|
456
461
|
} else if (options.command.startsWith("list-") && LIFECYCLE_COLLECTIONS.has(options.command.slice(5))) {
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
"use strict";
|
|
3
|
+
|
|
4
|
+
const path = require("node:path");
|
|
5
|
+
|
|
6
|
+
const { verifyPackage } = require("./lib");
|
|
7
|
+
|
|
8
|
+
function verifyReleasePackage(packageRoot = path.resolve(__dirname, "..")) {
|
|
9
|
+
try {
|
|
10
|
+
return verifyPackage(packageRoot);
|
|
11
|
+
} catch (error) {
|
|
12
|
+
throw new Error(
|
|
13
|
+
`拒绝发布未验证的 ML Platform 包:${error.message}。`
|
|
14
|
+
+ " 请只发布 build_ml_platform_release.py 生成的 .tgz 归档。",
|
|
15
|
+
);
|
|
16
|
+
}
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
if (require.main === module) {
|
|
20
|
+
try {
|
|
21
|
+
verifyReleasePackage();
|
|
22
|
+
} catch (error) {
|
|
23
|
+
process.stderr.write(`${error.message}\n`);
|
|
24
|
+
process.exitCode = 1;
|
|
25
|
+
}
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
module.exports = { verifyReleasePackage };
|
|
@@ -258,6 +258,15 @@ Parameter name.
|
|
|
258
258
|
|
|
259
259
|
Create one stable, versioned source contract for each independently readable value. Keep credentials out of JSON. Use only supported direct adapters (`postgresql_direct`, `mysql_direct`, `sqlserver_direct`, and `influxdb_direct`) and explicit source identity. The server supplies database DSNs from managed Secrets; never add them to CLI arguments or Catalog JSON. Put source data validity in `quality_rules`; put dataset-specific transformations in preprocess Operators.
|
|
260
260
|
|
|
261
|
+
When historical and live systems expose the same value, keep one Parameter and add an optional
|
|
262
|
+
`online_source`. Batch/as-of reads always use `source`; realtime causal reads use `online_source`
|
|
263
|
+
when present and otherwise fall back to `source`. Both sources must produce the same standard long
|
|
264
|
+
table and preserve value, unit, event-time, frequency, quality, availability, and missingness
|
|
265
|
+
semantics. Do not duplicate the Feature or Operator. Realtime lineage identifies the selected source
|
|
266
|
+
and includes the offline source; the response is not replayable through the offline manifest when an
|
|
267
|
+
override is active. The executable JSON shape is in
|
|
268
|
+
[references/contracts.md](references/contracts.md).
|
|
269
|
+
|
|
261
270
|
When the approved source contract supplies SQL, copy its query text and named
|
|
262
271
|
parameters exactly into `source.sql`/`source.params`. Do not retype, beautify,
|
|
263
272
|
rename an identifier, change a join key, or "simplify" an approved query while
|
|
@@ -404,9 +413,11 @@ Confirm the cutoff and prediction horizon. The command must call
|
|
|
404
413
|
publish a DatasetArtifact.
|
|
405
414
|
|
|
406
415
|
Verify that the response reports the requested cutoff and derived target time,
|
|
407
|
-
`uses_post_cutoff_data=false`, `contract.manifest_hash`,
|
|
408
|
-
|
|
409
|
-
|
|
416
|
+
`uses_post_cutoff_data=false`, `contract.manifest_hash`, and per-Parameter
|
|
417
|
+
freshness evidence. A response without `online_source` should include a
|
|
418
|
+
replayable `as_of` manifest; when an online source override is active, `replay.supported`
|
|
419
|
+
must be false because the offline manifest cannot reproduce that source. Report
|
|
420
|
+
tolerated missing or gap-filled inputs as degraded freshness, not as a normal read.
|
|
410
421
|
|
|
411
422
|
## Verify The Artifact
|
|
412
423
|
|
|
@@ -61,6 +61,7 @@ Required shape:
|
|
|
61
61
|
"unit": "kPa",
|
|
62
62
|
"expected_frequency": "5s",
|
|
63
63
|
"source": {},
|
|
64
|
+
"online_source": null,
|
|
64
65
|
"time_semantics": {
|
|
65
66
|
"event_time_field": "event_time",
|
|
66
67
|
"ingested_at_field": "ingested_at",
|
|
@@ -92,6 +93,38 @@ Required shape:
|
|
|
92
93
|
}
|
|
93
94
|
```
|
|
94
95
|
|
|
96
|
+
`online_source` 是可选的实时来源覆盖。离线 Dataset build、`snapshot`、`as_of` 和历史回放
|
|
97
|
+
只读取 `source`;实时因果读取在声明时读取 `online_source`,否则回退到 `source`。两者必须
|
|
98
|
+
表达同一个 Parameter,产生相同语义的标准长表(`timestamp, metric_name, value, unit`),
|
|
99
|
+
并保持值、单位、事件时间、频率、质量和可用时间语义一致。它们可以使用不同的受支持
|
|
100
|
+
adapter;凭据不进入 JSON。Feature/Operator/FeatureSet 不需要复制一份。
|
|
101
|
+
|
|
102
|
+
例如:
|
|
103
|
+
|
|
104
|
+
```json
|
|
105
|
+
{
|
|
106
|
+
"source": {
|
|
107
|
+
"adapter": "postgresql_direct",
|
|
108
|
+
"schema": "process_data",
|
|
109
|
+
"table": "pressure_history",
|
|
110
|
+
"time_column": "event_time",
|
|
111
|
+
"value_column": "pressure",
|
|
112
|
+
"metric_name": "hot_blast_pressure"
|
|
113
|
+
},
|
|
114
|
+
"online_source": {
|
|
115
|
+
"adapter": "influxdb_direct",
|
|
116
|
+
"measurement": "blast_furnace_live",
|
|
117
|
+
"field": "pressure",
|
|
118
|
+
"metric_name": "hot_blast_pressure"
|
|
119
|
+
}
|
|
120
|
+
}
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
实时 lineage 会标出 `source_role=online_source` 并保留 `offline_source`。`online_source`
|
|
124
|
+
参与 Parameter hash;因此变更它必须发布新的 Parameter 版本及受影响的下游依赖。使用在线
|
|
125
|
+
覆盖时,响应中的 replay 会明确标记为不支持,因为离线 manifest 只会重新读取 `source`,
|
|
126
|
+
不能伪造线上读法的回放。
|
|
127
|
+
|
|
95
128
|
Supported source modes are `direct_column`, `sql`, and `derived`. Prefer `direct_column` for ordinary business parameters.
|
|
96
129
|
|
|
97
130
|
For `sql`, `source.sql` must contain one statement without a trailing semicolon,
|
|
@@ -415,8 +448,10 @@ writing parquet, or publishing a DatasetArtifact. It derives
|
|
|
415
448
|
`target_time = cutoff_time + prediction.horizon`, filters every source to
|
|
416
449
|
`timestamp <= cutoff_time`, and derives each Parameter read window from the
|
|
417
450
|
selected Operators' `input_schema.history_requirements`. The response carries
|
|
418
|
-
`contract.manifest_hash
|
|
419
|
-
|
|
451
|
+
`contract.manifest_hash`; it also carries a replayable `read_policy=as_of` manifest
|
|
452
|
+
unless a Parameter's `online_source` override is active, in which case
|
|
453
|
+
`replay.supported=false` is returned because an offline manifest would read the
|
|
454
|
+
offline `source` instead.
|
|
420
455
|
|
|
421
456
|
Keep the default strict: `allow_missing=false`. Tail-edge and bounded internal
|
|
422
457
|
gap-fill tolerance may return a row, but the response must mark
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: model-lifecycle-management
|
|
3
|
-
description: Manage ITSMP model training, evaluation,
|
|
3
|
+
description: Manage ITSMP model training, evaluation, immutable model packaging, and contract-bound inference requests. Use when a user asks to list or inspect registered TrainerDefinitions, register a trainer, validate or submit a training run, inspect/retry/cancel training jobs, configure or run governed evaluation, inspect evaluation evidence, validate a ModelArtifact, create and inspect a ModelPackage, or invoke a deployed model. Do not use for feature catalog authoring, dataset builds, model deployment, Kubernetes administration, or image release engineering.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Model Lifecycle Management
|
|
7
7
|
|
|
8
|
-
Use the `ml-platform` executable installed with the same npm release as this Skill. Verify `ml-platform version`, `ml-platform show-config`, and server health before API operations. Never use a platform source checkout or a second client runtime.
|
|
8
|
+
Use the `ml-platform` executable installed with the same npm release as this Skill. Verify `ml-platform version`, `ml-platform show-config`, and server health before control-plane API operations. Use `predict-model` with the deployed model's Serving URL; it is distinct from the control-plane API URL. Never use a platform source checkout or a second client runtime.
|
|
9
9
|
|
|
10
10
|
```text
|
|
11
11
|
DatasetArtifact -> TrainingRun -> TrainingJob -> ModelArtifact
|
|
@@ -25,8 +25,10 @@ runtime contract, not repeated package versions. Unknown request keys return 422
|
|
|
25
25
|
fields. Inspect the returned resolved package.
|
|
26
26
|
|
|
27
27
|
`input_modes` must be unique, include `inline`, and may additionally include
|
|
28
|
-
`feature_lookup` only when supported by the selected
|
|
29
|
-
|
|
28
|
+
`feature_lookup` or `tabular_forecast` only when supported by the selected
|
|
29
|
+
runtime. `feature_lookup` still requires artifact lineage
|
|
30
|
+
`online_eligible=true`; `tabular_forecast` requires a direct tabular model
|
|
31
|
+
signature. Do not infer either capability from training success.
|
|
30
32
|
Omit rather than use an empty list to request default inline behavior.
|
|
31
33
|
|
|
32
34
|
Always query the exact package ID and version; states are `PACKAGING`, `READY`,
|
|
@@ -68,6 +68,66 @@ Non-temporal regression can omit it. Do not change task kind just to evade check
|
|
|
68
68
|
`series_keys` must identify entity-key columns when multiple series coexist;
|
|
69
69
|
an empty list treats the entire artifact as one series.
|
|
70
70
|
|
|
71
|
+
## Direct tabular forecasting with known-future features
|
|
72
|
+
|
|
73
|
+
Tree and other matrix-based models can forecast a variable target grid without
|
|
74
|
+
pretending that future observations are available. Set the tabular adapter to
|
|
75
|
+
`forecast_mode="direct"`. The dataset's `TemporalBinding.prediction_time_column`
|
|
76
|
+
is the historical `cutoff_time`; `label_time_column` is the row's future
|
|
77
|
+
`target_time`. Each row is one `(cutoff_time, target_time)` pair. Values such as
|
|
78
|
+
weather forecasts, schedules, calendar fields and `lead_hours` are ordinary
|
|
79
|
+
feature columns; list the forecast columns in `known_future_columns`.
|
|
80
|
+
|
|
81
|
+
```json
|
|
82
|
+
{
|
|
83
|
+
"input_adapter": {
|
|
84
|
+
"kind": "tabular",
|
|
85
|
+
"forecast_mode": "direct",
|
|
86
|
+
"known_future_columns": ["weather_temperature", "weather_wind"]
|
|
87
|
+
},
|
|
88
|
+
"data": {
|
|
89
|
+
"temporal": {
|
|
90
|
+
"prediction_time_column": "cutoff_time",
|
|
91
|
+
"label_time_column": "target_time",
|
|
92
|
+
"frequency": "1h",
|
|
93
|
+
"horizon": "240h",
|
|
94
|
+
"series_keys": ["market"]
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
At serving time use `input_mode="tabular_forecast"` and send one row per
|
|
101
|
+
target. `target_time` must be later than the cutoff and aligned to the model
|
|
102
|
+
frequency. When known-future columns are declared, every row must include
|
|
103
|
+
`available_at <= cutoff_time`; this prevents using a forecast revision that did
|
|
104
|
+
not exist when the prediction was made.
|
|
105
|
+
|
|
106
|
+
```json
|
|
107
|
+
{
|
|
108
|
+
"input_mode": "tabular_forecast",
|
|
109
|
+
"cutoff_time": "2026-09-09T10:30:00+08:00",
|
|
110
|
+
"targets": [
|
|
111
|
+
{
|
|
112
|
+
"target_time": "2026-09-10T00:00:00+08:00",
|
|
113
|
+
"available_at": "2026-09-09T08:00:00+08:00",
|
|
114
|
+
"values": {
|
|
115
|
+
"weather_temperature": 21.3,
|
|
116
|
+
"weather_wind": 2.8,
|
|
117
|
+
"lead_hours": 13.5
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
]
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
The serving runtime calls the same plugin `predict()` with a two-dimensional
|
|
125
|
+
batch and returns one result carrying each `target_time`. XGBoost and LightGBM
|
|
126
|
+
already implement this batch matrix operation; the platform adapter owns the
|
|
127
|
+
cutoff, target-grid and availability checks. Training must use historical
|
|
128
|
+
forecast snapshots selected by `available_at <= cutoff_time`, never later actual
|
|
129
|
+
weather observations.
|
|
130
|
+
|
|
71
131
|
Inspect `data_schema.json`, `resolved_manifest.json` and the feature Parquet via
|
|
72
132
|
`download-dataset-artifact DATASET_ID MANIFEST_HASH --project PROJECT --out-dir DIR`
|
|
73
133
|
(repeat `--file` for selective downloads). The FeatureSet reference must match
|
|
@@ -79,6 +139,41 @@ in `required_inputs` and at least one `online_inputs` entry, which must be a
|
|
|
79
139
|
subset; it must have an inverse. Do not use target transforms to fix incorrect
|
|
80
140
|
dataset label clocks or to invent unsupported transforms.
|
|
81
141
|
|
|
142
|
+
## Chronos-2 v2 targets and covariates
|
|
143
|
+
|
|
144
|
+
Use the v2 adapter when Chronos-2 should jointly forecast multiple storage
|
|
145
|
+
channels or consume covariates. The target and output-label arrays are ordered,
|
|
146
|
+
and `data.labels` must use that same output-label order:
|
|
147
|
+
|
|
148
|
+
```json
|
|
149
|
+
{
|
|
150
|
+
"labels": ["future_storage_soc", "future_storage_power"],
|
|
151
|
+
"forecast_target_adapter": {
|
|
152
|
+
"schema_version": "ml_data_platform.forecast_target_adapter/v2",
|
|
153
|
+
"kind": "continuous_target_series_with_covariates",
|
|
154
|
+
"targets": [
|
|
155
|
+
{"input_column": "storage_soc", "output_label_column": "future_storage_soc"},
|
|
156
|
+
{"input_column": "storage_power", "output_label_column": "future_storage_power"}
|
|
157
|
+
],
|
|
158
|
+
"covariates": [
|
|
159
|
+
{"column": "recent_load", "role": "past_only"},
|
|
160
|
+
{"column": "weather_temperature", "role": "known_future"},
|
|
161
|
+
{"column": "peak_valley", "role": "known_future"}
|
|
162
|
+
],
|
|
163
|
+
"target_gap_policy": "reject",
|
|
164
|
+
"point_output": {"statistic": "median", "step": "horizon"}
|
|
165
|
+
}
|
|
166
|
+
}
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Every target and covariate must be a numeric Feature. `past_only` is available
|
|
170
|
+
only through the cutoff. `known_future` must be represented in training by the
|
|
171
|
+
forecast or schedule available at that historic cutoff, never by a later actual
|
|
172
|
+
observation. At runtime callers provide one finite numeric value for every
|
|
173
|
+
`known_future` column and every prediction step. The current contract validates
|
|
174
|
+
the names, values, and row count; retain forecast issue/availability lineage in
|
|
175
|
+
the upstream Feature data when revisions must be audited.
|
|
176
|
+
|
|
82
177
|
## TrainingJobRequest and identity propagation
|
|
83
178
|
|
|
84
179
|
After validation and authorized registration, use the returned `spec_hash`
|
|
@@ -13,16 +13,17 @@ FeatureSet order and evaluation runtime before the first Job. See
|
|
|
13
13
|
for device`, preserve the rejected identity, device and release ID. Use `get-model-runtime-release` to find the
|
|
14
14
|
approved binding; if the deployed release lacks discovery, request its contract;
|
|
15
15
|
do not try similar names or re-register trainers to bypass approval.
|
|
16
|
-
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
business inputs:
|
|
16
|
+
- Chronos-2 supports the legacy v1 single-target adapter and the v2
|
|
17
|
+
covariate-aware adapter. v2 jointly forecasts ordered target channels and
|
|
18
|
+
accepts numeric `past_only` and `known_future` covariates. Do not silently
|
|
19
|
+
drop business inputs: declare every target and covariate in the adapter.
|
|
20
20
|
- Keep labels outside FeatureSet order, verify temporal horizon against
|
|
21
21
|
`label_time - prediction_time`, and validate the complete TrainingRun before
|
|
22
22
|
registration/submission. Validation may not catch undeclared plugin limits or
|
|
23
23
|
missing dependencies in the eventual Worker.
|
|
24
|
-
- For a continuous future trajectory, use TrainingRun v3
|
|
25
|
-
`
|
|
24
|
+
- For a continuous future trajectory, use TrainingRun v3. Use
|
|
25
|
+
`continuous_target_series` for legacy v1 or
|
|
26
|
+
`continuous_target_series_with_covariates` for v2. Its fixed maximum
|
|
26
27
|
prediction length comes from temporal horizon divided by frequency; do not
|
|
27
28
|
put `prediction_length` in trainer-specific parameters.
|
|
28
29
|
|
|
@@ -83,3 +84,24 @@ ml-platform --profile server cancel-training-job <job-id>
|
|
|
83
84
|
```
|
|
84
85
|
|
|
85
86
|
Retry and cancellation require exact-Job authorization. Never change a run contract while retrying; create a new immutable run when semantic inputs change.
|
|
87
|
+
|
|
88
|
+
## Chronos-2 future-known inputs
|
|
89
|
+
|
|
90
|
+
For storage forecasting, tomorrow's weather forecast and peak/valley schedule
|
|
91
|
+
are `known_future` covariates. The v2 adapter binds their Feature columns and
|
|
92
|
+
the ordered target/output-label pairs; see [training-contracts.md](training-contracts.md)
|
|
93
|
+
for the complete JSON fragment.
|
|
94
|
+
|
|
95
|
+
Training records must contain the forecast that was available at each historic
|
|
96
|
+
cutoff, not actual weather observed later. At online inference, pass one numeric
|
|
97
|
+
row per forecast step using the Serving runtime, not `fetch-inference-data`:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
ml-platform predict-model /absolute/path/to/inference-request.json \
|
|
101
|
+
--serving-url https://storage-forecast.example
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
For `input_mode=inline`, put those rows in `inputs.future_covariates`. For
|
|
105
|
+
`input_mode=feature_lookup`, put them in the top-level `future_covariates`.
|
|
106
|
+
Their column names must exactly match the adapter's `known_future` covariates,
|
|
107
|
+
and their count must equal the trained `prediction_length`.
|