@zhuoyuezs/ml-platform 0.2.2-alpha.2 → 0.2.2-alpha.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DEVELOPMENT.md +2 -1
- package/README.md +7 -5
- package/checksums.json +24 -14
- package/package.json +1 -1
- package/release.json +17 -11
- package/runtime/business-client/README.md +3 -0
- package/runtime/business-client/package-lock.json +2 -2
- package/runtime/business-client/package.json +1 -1
- package/runtime/business-client/src/cli.js +42 -1
- package/skills/data-source-exploration/SKILL.md +90 -0
- package/skills/data-source-exploration/agents/openai.yaml +4 -0
- package/skills/model-lifecycle-management/SKILL.md +4 -2
- package/skills/model-lifecycle-management/references/discovery.md +9 -3
- package/skills/model-lifecycle-management/references/training-contracts.md +53 -0
- package/skills/model-lifecycle-management/references/training.md +64 -0
package/DEVELOPMENT.md
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
- Skill 直接调用正式命令 `ml-platform`,不嵌入 CLI runtime。
|
|
11
11
|
- 正式 npm 包只有 `@zhuoyuezs/ml-platform`,正式 bin 只有 `ml-platform`。
|
|
12
12
|
- `business-client-js/src-ts/` 是 business CLI 的唯一 TypeScript 源码,`src/` 是供 Node.js 和
|
|
13
|
-
release payload 使用的编译输出;仓库 `skills/`
|
|
13
|
+
release payload 使用的编译输出;仓库 `skills/` 下四个受支持 Skill 是各自唯一源码。
|
|
14
14
|
- `npx @zhuoyuezs/ml-platform@<version> install` 是面向用户的统一安装入口;默认目标是 Codex
|
|
15
15
|
用户级 Skill,不要求额外安装参数。
|
|
16
16
|
- 管理命令默认输出简洁结果;Agent 和 CI 显式使用 `--json` 获取稳定结构化输出。
|
|
@@ -31,6 +31,7 @@ business-client-js/
|
|
|
31
31
|
└── src/ generated Node.js runtime
|
|
32
32
|
npm/ml-platform/src/ TypeScript installer and dispatcher source
|
|
33
33
|
skills/feature-management/
|
|
34
|
+
skills/data-source-exploration/
|
|
34
35
|
skills/model-lifecycle-management/
|
|
35
36
|
skills/model-deployment-management/
|
|
36
37
|
npm/ml-platform/
|
package/README.md
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
# ML Platform
|
|
2
2
|
|
|
3
|
-
ML Platform 的 TypeScript CLI(编译为 Node.js JavaScript runtime)、`feature-management
|
|
4
|
-
`model-
|
|
3
|
+
ML Platform 的 TypeScript CLI(编译为 Node.js JavaScript runtime)、`feature-management`、
|
|
4
|
+
`data-source-exploration`、`model-lifecycle-management` 与 `model-deployment-management`
|
|
5
|
+
Agent Skill。CLI 与 Skill 由同一个 npm
|
|
5
6
|
release 安装和升级,运行时不需要 Python 或 `uv`。
|
|
6
7
|
|
|
7
8
|
## 环境要求
|
|
@@ -18,7 +19,7 @@ release 安装和升级,运行时不需要 Python 或 `uv`。
|
|
|
18
19
|
npx @zhuoyuezs/ml-platform@latest install
|
|
19
20
|
```
|
|
20
21
|
|
|
21
|
-
这条命令不需要额外参数。默认安装持久化的 `ml-platform`
|
|
22
|
+
这条命令不需要额外参数。默认安装持久化的 `ml-platform` 命令,并将四个 Skill 安装到
|
|
22
23
|
`~/.codex/skills/`,供当前用户的 Codex 使用。安装后重启终端和 Agent 会话。
|
|
23
24
|
|
|
24
25
|
只在当前项目使用 Skill 时,在项目根目录运行:
|
|
@@ -94,6 +95,7 @@ Skill。`status --json` 中 `installed_release` 是当前 scope 的 Skill releas
|
|
|
94
95
|
|
|
95
96
|
```bash
|
|
96
97
|
ml-platform list-parameters
|
|
98
|
+
ml-platform list-datasource-connections
|
|
97
99
|
ml-platform list-datasets --project <project>
|
|
98
100
|
ml-platform get-dataset <dataset_id> <dataset_version> --project <project>
|
|
99
101
|
ml-platform resolve-dataset <dataset_id> <dataset_version> --project <project>
|
|
@@ -129,8 +131,8 @@ npx --yes --registry=https://registry.npmjs.org/ \
|
|
|
129
131
|
## 包内容
|
|
130
132
|
|
|
131
133
|
- 一份由 TypeScript 编译的 business CLI;
|
|
132
|
-
-
|
|
133
|
-
`model-deployment-management`;
|
|
134
|
+
- 四份纯 Skill:`feature-management`、`data-source-exploration`、
|
|
135
|
+
`model-lifecycle-management` 与 `model-deployment-management`;
|
|
134
136
|
- 显式安装、升级、状态、校验和回滚逻辑。
|
|
135
137
|
|
|
136
138
|
不包含 Codex CLI、Python business-client wheel、Agent Host、平台服务端、模型训练或 Kubernetes
|
package/checksums.json
CHANGED
|
@@ -2,17 +2,17 @@
|
|
|
2
2
|
"files": [
|
|
3
3
|
{
|
|
4
4
|
"path": "runtime/business-client/README.md",
|
|
5
|
-
"sha256": "sha256:
|
|
6
|
-
"size_bytes":
|
|
5
|
+
"sha256": "sha256:30d921fabf5bc0339eda60169a1865f6007807b1d9a8d3e646b42f68cd563240",
|
|
6
|
+
"size_bytes": 8125
|
|
7
7
|
},
|
|
8
8
|
{
|
|
9
9
|
"path": "runtime/business-client/package-lock.json",
|
|
10
|
-
"sha256": "sha256:
|
|
10
|
+
"sha256": "sha256:ebc31820ae0d2985879efd87ba4b0d8843e44a175d4c9e1dc557737893f8bdae",
|
|
11
11
|
"size_bytes": 1628
|
|
12
12
|
},
|
|
13
13
|
{
|
|
14
14
|
"path": "runtime/business-client/package.json",
|
|
15
|
-
"sha256": "sha256:
|
|
15
|
+
"sha256": "sha256:4685ca172b5b23bd5de53bcccbd7f7412df281eff67533d72d1b598c63a28b47",
|
|
16
16
|
"size_bytes": 671
|
|
17
17
|
},
|
|
18
18
|
{
|
|
@@ -22,8 +22,8 @@
|
|
|
22
22
|
},
|
|
23
23
|
{
|
|
24
24
|
"path": "runtime/business-client/src/cli.js",
|
|
25
|
-
"sha256": "sha256:
|
|
26
|
-
"size_bytes":
|
|
25
|
+
"sha256": "sha256:c1beda5fd4e2d6f23f81529017679d85c0703d6859cfa0155a8597e50b7b6cdf",
|
|
26
|
+
"size_bytes": 60617
|
|
27
27
|
},
|
|
28
28
|
{
|
|
29
29
|
"path": "runtime/business-client/src/config.js",
|
|
@@ -35,6 +35,16 @@
|
|
|
35
35
|
"sha256": "sha256:4ae9e9ec10a74f36ed19ca7a63b0ba58a34cedea13ed95131ef1d07cd7d9e674",
|
|
36
36
|
"size_bytes": 17176
|
|
37
37
|
},
|
|
38
|
+
{
|
|
39
|
+
"path": "skills/data-source-exploration/SKILL.md",
|
|
40
|
+
"sha256": "sha256:3b609d61f3c73da3657ce2e6bb4d1eff95a90e4ac5e5445b8513536438b26b0b",
|
|
41
|
+
"size_bytes": 3853
|
|
42
|
+
},
|
|
43
|
+
{
|
|
44
|
+
"path": "skills/data-source-exploration/agents/openai.yaml",
|
|
45
|
+
"sha256": "sha256:5c7f7d497ad7738c672b8ee906d514284a2f2583a854f8fa487708a75836256f",
|
|
46
|
+
"size_bytes": 299
|
|
47
|
+
},
|
|
38
48
|
{
|
|
39
49
|
"path": "skills/feature-management/SKILL.md",
|
|
40
50
|
"sha256": "sha256:88898ea83ff339512d5512537db485a2764420abb67afceb9dd090cc780c00e0",
|
|
@@ -142,8 +152,8 @@
|
|
|
142
152
|
},
|
|
143
153
|
{
|
|
144
154
|
"path": "skills/model-lifecycle-management/SKILL.md",
|
|
145
|
-
"sha256": "sha256:
|
|
146
|
-
"size_bytes":
|
|
155
|
+
"sha256": "sha256:4ea40815d73b3a470b2572874dcad63640cbdd53d3c10293d7a5ce8621a13b96",
|
|
156
|
+
"size_bytes": 4272
|
|
147
157
|
},
|
|
148
158
|
{
|
|
149
159
|
"path": "skills/model-lifecycle-management/agents/openai.yaml",
|
|
@@ -152,8 +162,8 @@
|
|
|
152
162
|
},
|
|
153
163
|
{
|
|
154
164
|
"path": "skills/model-lifecycle-management/references/discovery.md",
|
|
155
|
-
"sha256": "sha256:
|
|
156
|
-
"size_bytes":
|
|
165
|
+
"sha256": "sha256:184472a79fb217dd8c74dffab20e75285b0c8f69207dc593ae3099b8a2b6d6d5",
|
|
166
|
+
"size_bytes": 5381
|
|
157
167
|
},
|
|
158
168
|
{
|
|
159
169
|
"path": "skills/model-lifecycle-management/references/evaluation.md",
|
|
@@ -167,13 +177,13 @@
|
|
|
167
177
|
},
|
|
168
178
|
{
|
|
169
179
|
"path": "skills/model-lifecycle-management/references/training-contracts.md",
|
|
170
|
-
"sha256": "sha256:
|
|
171
|
-
"size_bytes":
|
|
180
|
+
"sha256": "sha256:4cecf5f90464640d063ca721fded1c27b46bb79daa0f67c78a8fb6b0e3e60b3b",
|
|
181
|
+
"size_bytes": 14747
|
|
172
182
|
},
|
|
173
183
|
{
|
|
174
184
|
"path": "skills/model-lifecycle-management/references/training.md",
|
|
175
|
-
"sha256": "sha256:
|
|
176
|
-
"size_bytes":
|
|
185
|
+
"sha256": "sha256:97de7894da24a07b4146d6de5e00a0cf17166654fb219a145f35adff5e8a7f99",
|
|
186
|
+
"size_bytes": 7980
|
|
177
187
|
}
|
|
178
188
|
],
|
|
179
189
|
"schema_version": "data_platform.ml_platform_checksums/v1"
|
package/package.json
CHANGED
package/release.json
CHANGED
|
@@ -13,8 +13,8 @@
|
|
|
13
13
|
"entrypoint": "src/cli.js",
|
|
14
14
|
"name": "ml-platform",
|
|
15
15
|
"path": "runtime/business-client",
|
|
16
|
-
"sha256": "sha256:
|
|
17
|
-
"version": "0.8.
|
|
16
|
+
"sha256": "sha256:4e06df68d4f849eae92d1fcfb57f1a3b9e949f97ed8042ec549c8b460f3736c8",
|
|
17
|
+
"version": "0.8.3"
|
|
18
18
|
},
|
|
19
19
|
"policy_sha256": "sha256:02fbde0134696b78c0c365d6e0b76c9814227208da5b2b3d4e5dbd82dc4a3606",
|
|
20
20
|
"public_schema_versions": [
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"ml_data_platform.feature_set/v1",
|
|
26
26
|
"ml_data_platform.dataset_manifest/v1"
|
|
27
27
|
],
|
|
28
|
-
"release_version": "0.2.2-alpha.
|
|
28
|
+
"release_version": "0.2.2-alpha.4",
|
|
29
29
|
"runtime_requirements": {
|
|
30
30
|
"node": ">=18",
|
|
31
31
|
"os": [
|
|
@@ -35,24 +35,30 @@
|
|
|
35
35
|
},
|
|
36
36
|
"schema_version": "data_platform.ml_platform_release/v2",
|
|
37
37
|
"skills": {
|
|
38
|
+
"data-source-exploration": {
|
|
39
|
+
"path": "skills/data-source-exploration",
|
|
40
|
+
"requires_cli": ">=0.8.3 <0.9.0",
|
|
41
|
+
"revision": "0.2.2-alpha.4",
|
|
42
|
+
"sha256": "sha256:ae3cfc88f8f3f46a5b7e0179d850b2216bc37275f34679dbea1ff2ba99875a96"
|
|
43
|
+
},
|
|
38
44
|
"feature-management": {
|
|
39
45
|
"path": "skills/feature-management",
|
|
40
|
-
"requires_cli": ">=0.8.
|
|
41
|
-
"revision": "0.2.2-alpha.
|
|
46
|
+
"requires_cli": ">=0.8.3 <0.9.0",
|
|
47
|
+
"revision": "0.2.2-alpha.4",
|
|
42
48
|
"sha256": "sha256:d32f0306f129d7bf76eb037fe239c84dd3c29c64b61600799d6e40320784bb10"
|
|
43
49
|
},
|
|
44
50
|
"model-deployment-management": {
|
|
45
51
|
"path": "skills/model-deployment-management",
|
|
46
|
-
"requires_cli": ">=0.8.
|
|
47
|
-
"revision": "0.2.2-alpha.
|
|
52
|
+
"requires_cli": ">=0.8.3 <0.9.0",
|
|
53
|
+
"revision": "0.2.2-alpha.4",
|
|
48
54
|
"sha256": "sha256:e7d87c272a0dcc028ea628d5e36f5b7e52ed27cb9c1bc17ea76527478e28b46d"
|
|
49
55
|
},
|
|
50
56
|
"model-lifecycle-management": {
|
|
51
57
|
"path": "skills/model-lifecycle-management",
|
|
52
|
-
"requires_cli": ">=0.8.
|
|
53
|
-
"revision": "0.2.2-alpha.
|
|
54
|
-
"sha256": "sha256:
|
|
58
|
+
"requires_cli": ">=0.8.3 <0.9.0",
|
|
59
|
+
"revision": "0.2.2-alpha.4",
|
|
60
|
+
"sha256": "sha256:ec6ebb2587fd7acd024def75a8c0e9cd94b6cdd8ee6df439609291b496751b91"
|
|
55
61
|
}
|
|
56
62
|
},
|
|
57
|
-
"source_commit": "
|
|
63
|
+
"source_commit": "bbad8bc765cdfcfc1ab95e8954da2265ea8dba59"
|
|
58
64
|
}
|
|
@@ -147,6 +147,9 @@ resolve the same contracts as submission without registering or scheduling
|
|
|
147
147
|
workloads. Read `submitted=false` and `unchecked`: successful preflight does not
|
|
148
148
|
verify image pulls, cluster resources or runtime execution. Complete request
|
|
149
149
|
examples and identity propagation are in the bundled model lifecycle Skill.
|
|
150
|
+
Training Run and Job validation also return `effective_trainer_parameters`;
|
|
151
|
+
for Job preflight it must match `job_spec.trainer_parameters` and represents
|
|
152
|
+
the complete mapping delivered to the Worker after defaults are resolved.
|
|
150
153
|
|
|
151
154
|
Maintainers can export the current API, including uncommitted schema changes,
|
|
152
155
|
with `uv run python scripts/generate_openapi_schema.py --working-tree` from the
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zhuoyuezs/ml-platform-business-client",
|
|
3
|
-
"version": "0.8.
|
|
3
|
+
"version": "0.8.3",
|
|
4
4
|
"lockfileVersion": 3,
|
|
5
5
|
"requires": true,
|
|
6
6
|
"packages": {
|
|
7
7
|
"": {
|
|
8
8
|
"name": "@zhuoyuezs/ml-platform-business-client",
|
|
9
|
-
"version": "0.8.
|
|
9
|
+
"version": "0.8.3",
|
|
10
10
|
"license": "UNLICENSED",
|
|
11
11
|
"bin": {
|
|
12
12
|
"ml-platform": "src/cli.js"
|
|
@@ -13,7 +13,10 @@ const BUSINESS_COMMANDS = new Set([
|
|
|
13
13
|
"get-model-runtime-release", "get-training-run", "get-model-artifact", "get-executable-package",
|
|
14
14
|
"validate-training-job", "validate-model-package",
|
|
15
15
|
"version", "create-project", "list-projects", "get-project", "delete-project",
|
|
16
|
-
"configure", "show-config", "health",
|
|
16
|
+
"configure", "show-config", "health",
|
|
17
|
+
"list-datasource-connections", "list-datasource-namespaces", "list-datasource-objects",
|
|
18
|
+
"list-datasource-fields", "query-datasource-object",
|
|
19
|
+
"list-parameters", "list-operators", "list-features",
|
|
17
20
|
"list-feature-sets", "list-datasets", "get-dataset", "list-jobs", "resolve-manifest",
|
|
18
21
|
"resolve-dataset", "build-dataset", "build-registered-dataset",
|
|
19
22
|
"get-parameter", "get-feature", "get-feature-set", "get-operator",
|
|
@@ -49,6 +52,11 @@ const COMMAND_USAGE = {
|
|
|
49
52
|
configure: "configure --api-url URL",
|
|
50
53
|
"show-config": "show-config",
|
|
51
54
|
health: "health",
|
|
55
|
+
"list-datasource-connections": "list-datasource-connections [--limit N] [--cursor CURSOR]",
|
|
56
|
+
"list-datasource-namespaces": "list-datasource-namespaces CONNECTION_ID [--limit N] [--cursor CURSOR]",
|
|
57
|
+
"list-datasource-objects": "list-datasource-objects CONNECTION_ID NAMESPACE_ID [--limit N] [--cursor CURSOR]",
|
|
58
|
+
"list-datasource-fields": "list-datasource-fields CONNECTION_ID OBJECT_ID [--limit N] [--cursor CURSOR]",
|
|
59
|
+
"query-datasource-object": "query-datasource-object CONNECTION_ID OBJECT_ID SPEC_JSON [--cursor CURSOR]",
|
|
52
60
|
"create-project": "create-project NAME [--display-name TEXT] [--description TEXT] [--owner OWNER]",
|
|
53
61
|
"list-projects": "list-projects",
|
|
54
62
|
"get-project": "get-project NAME",
|
|
@@ -238,6 +246,13 @@ function readObject(file) {
|
|
|
238
246
|
}
|
|
239
247
|
function readDataset(file) { return normalizeDataset(readObject(file)); }
|
|
240
248
|
function readSpec(file) { return readObject(file); }
|
|
249
|
+
function datasourcePageParams(rest) {
|
|
250
|
+
const rawLimit = take(rest, "--limit");
|
|
251
|
+
const limit = rawLimit === undefined ? 100 : integer(rawLimit, "--limit");
|
|
252
|
+
if (limit < 1 || limit > 500)
|
|
253
|
+
throw new Error("--limit must be between 1 and 500");
|
|
254
|
+
return { limit, cursor: take(rest, "--cursor") };
|
|
255
|
+
}
|
|
241
256
|
function datasetEndpoint(project, datasetId, datasetVersion) {
|
|
242
257
|
return `/datasets/${encodeURIComponent(project)}/${encodeURIComponent(datasetId)}/${encodeURIComponent(datasetVersion)}`;
|
|
243
258
|
}
|
|
@@ -382,6 +397,32 @@ async function runBusinessCli(argv) {
|
|
|
382
397
|
}
|
|
383
398
|
else if (options.command === "health")
|
|
384
399
|
result = await client(options).get("/health");
|
|
400
|
+
else if (options.command === "list-datasource-connections") {
|
|
401
|
+
result = await client(options).get("/datasource-connections", datasourcePageParams(rest));
|
|
402
|
+
}
|
|
403
|
+
else if (options.command === "list-datasource-namespaces") {
|
|
404
|
+
const connectionId = positional(rest, "connection id");
|
|
405
|
+
result = await client(options).get(`/datasource-connections/${encodeURIComponent(connectionId)}/namespaces`, datasourcePageParams(rest));
|
|
406
|
+
}
|
|
407
|
+
else if (options.command === "list-datasource-objects") {
|
|
408
|
+
const connectionId = positional(rest, "connection id");
|
|
409
|
+
const namespaceId = positional(rest, "namespace id");
|
|
410
|
+
result = await client(options).get(`/datasource-connections/${encodeURIComponent(connectionId)}/namespaces/${encodeURIComponent(namespaceId)}/objects`, datasourcePageParams(rest));
|
|
411
|
+
}
|
|
412
|
+
else if (options.command === "list-datasource-fields") {
|
|
413
|
+
const connectionId = positional(rest, "connection id");
|
|
414
|
+
const objectId = positional(rest, "object id");
|
|
415
|
+
result = await client(options).get(`/datasource-connections/${encodeURIComponent(connectionId)}/objects/${encodeURIComponent(objectId)}/fields`, datasourcePageParams(rest));
|
|
416
|
+
}
|
|
417
|
+
else if (options.command === "query-datasource-object") {
|
|
418
|
+
const connectionId = positional(rest, "connection id");
|
|
419
|
+
const objectId = positional(rest, "object id");
|
|
420
|
+
const spec = readSpec(positional(rest, "query spec JSON"));
|
|
421
|
+
const cursor = take(rest, "--cursor");
|
|
422
|
+
if (cursor !== undefined)
|
|
423
|
+
spec.cursor = cursor;
|
|
424
|
+
result = await client(options).post(`/datasource-connections/${encodeURIComponent(connectionId)}/objects/${encodeURIComponent(objectId)}/query`, spec);
|
|
425
|
+
}
|
|
385
426
|
else if (options.command === "create-project") {
|
|
386
427
|
const name = positional(rest, "project name");
|
|
387
428
|
result = await client(options).post("/projects", { schema_version: "ml_data_platform.project/v1", name, display_name: take(rest, "--display-name", name), description: take(rest, "--description", ""), owner: take(rest, "--owner", "demo") });
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: data-source-exploration
|
|
3
|
+
description: Discover platform-configured PostgreSQL, MySQL, SQL Server, or InfluxDB objects and run bounded structured reads for local data and feature exploration. Use when an algorithm engineer wants to inspect visible databases, schemas, tables, measurements, fields, or limited rows before defining governed feature assets. Do not use for free-form SQL, bulk export, asset publication, dataset builds, or model training.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Data Source Exploration
|
|
7
|
+
|
|
8
|
+
Use the platform API through `ml-platform`; never connect to a business database directly and
|
|
9
|
+
never request credentials. The platform exposes configured connections and their discoverable objects.
|
|
10
|
+
|
|
11
|
+
## Establish The Target
|
|
12
|
+
|
|
13
|
+
Before discovery, verify the installed client and server target:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
command -v ml-platform
|
|
17
|
+
ml-platform health
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
If the server target is missing or unhealthy, stop. Do not fall back to a DSN, database client, SQL,
|
|
21
|
+
Flux, or a repository-local source adapter.
|
|
22
|
+
|
|
23
|
+
## Discover Hierarchically
|
|
24
|
+
|
|
25
|
+
Read one level at a time and preserve the returned opaque IDs:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
ml-platform list-datasource-connections
|
|
29
|
+
ml-platform list-datasource-namespaces <connection_id>
|
|
30
|
+
ml-platform list-datasource-objects <connection_id> <namespace_id>
|
|
31
|
+
ml-platform list-datasource-fields <connection_id> <object_id>
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Follow `next_cursor` with `--cursor` until it is null when complete traversal is needed. Do not derive an
|
|
35
|
+
ID from a database, schema, table, measurement, or field name. An absent object means it is unavailable
|
|
36
|
+
or no longer present; do not probe alternate names.
|
|
37
|
+
|
|
38
|
+
## Query One Object
|
|
39
|
+
|
|
40
|
+
Create a `DatasourceQuerySpec` JSON in the user's workspace. Select only fields needed for the current
|
|
41
|
+
question, use bounded time filters for time-series data, and start with a small `limit`:
|
|
42
|
+
|
|
43
|
+
```json
|
|
44
|
+
{
|
|
45
|
+
"schema_version": "ml_data_platform.datasource_query/v1",
|
|
46
|
+
"field_ids": ["fld_time", "fld_entity", "fld_value"],
|
|
47
|
+
"filters": [
|
|
48
|
+
{
|
|
49
|
+
"field_id": "fld_time",
|
|
50
|
+
"operator": "between",
|
|
51
|
+
"values": ["2026-09-01T00:00:00+08:00", "2026-09-02T00:00:00+08:00"]
|
|
52
|
+
}
|
|
53
|
+
],
|
|
54
|
+
"order_by": [{"field_id": "fld_time", "direction": "asc"}],
|
|
55
|
+
"limit": 200,
|
|
56
|
+
"cursor": null
|
|
57
|
+
}
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Run it with:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
ml-platform query-datasource-object \
|
|
64
|
+
<connection_id> <object_id> datasource-query.json
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Supported filters are `eq`, `in`, `gt`, `gte`, `lt`, `lte`, `between`, and `is_null`. Only fields marked
|
|
68
|
+
filterable or sortable may be used in those positions. InfluxDB reads require bounded `_time` start and
|
|
69
|
+
end filters. For the next page, keep projection, filters, ordering, and `limit` unchanged and pass the
|
|
70
|
+
returned cursor with `--cursor`.
|
|
71
|
+
|
|
72
|
+
## Explore In The Workspace
|
|
73
|
+
|
|
74
|
+
Analyze returned rows with local code such as pandas or DuckDB. Keep these outputs outside the platform
|
|
75
|
+
repository unless the user explicitly asks for a repository-owned fixture. Record:
|
|
76
|
+
|
|
77
|
+
- the connection, namespace, object, field IDs, query spec, `audit_ref`, and response hash;
|
|
78
|
+
- observed facts such as dtypes, missingness, frequency, ranges, duplicates, and candidate entity/time keys;
|
|
79
|
+
- hypotheses about business meaning, target suitability, leakage, transformations, and model features;
|
|
80
|
+
- unresolved items that require a data owner or business owner to confirm.
|
|
81
|
+
|
|
82
|
+
Do not present inferred semantics as facts. Query results are exploratory evidence, not training data and
|
|
83
|
+
not a substitute for Parameter, Feature, FeatureSet, DatasetManifest, or DatasetArtifact.
|
|
84
|
+
|
|
85
|
+
## Hand Off Confirmed Work
|
|
86
|
+
|
|
87
|
+
After a human confirms source mapping, event time, entity keys, units, availability, quality rules, target,
|
|
88
|
+
horizon, and feature formulas, use `$feature-management` to create governed assets and build a
|
|
89
|
+
DatasetArtifact. Use `$model-lifecycle-management` only after that artifact exists. Exploration does not
|
|
90
|
+
authorize publishing assets, building datasets, or submitting training jobs.
|
|
@@ -24,10 +24,12 @@ Before training a model that will be evaluated, also read
|
|
|
24
24
|
[references/evaluation.md](references/evaluation.md). Check the final dataset
|
|
25
25
|
schema, training split binding and evaluation runtime before submission.
|
|
26
26
|
OpenAPI free-form dictionaries are not complete nested contracts: use the
|
|
27
|
-
examples in these references and same-release
|
|
28
|
-
to discover fields. Preserve release/error
|
|
27
|
+
SystemModel `parameter_schema`, examples in these references and same-release
|
|
28
|
+
validation, not successive 422s to discover fields. Preserve release/error
|
|
29
|
+
evidence when deployment differs.
|
|
29
30
|
|
|
30
31
|
- Establish exact immutable inputs, canonical SystemModel `v1`, runtime-release image digest, model inputs/targets, resources, evaluation policy, and package destination from authoritative contracts. Do not infer them from model names.
|
|
32
|
+
- For Chronos-2, inspect the typed `lora_config` schema before authoring a Run and preserve `effective_trainer_parameters` from validation. Do not infer PEFT keys or rely on Worker defaults.
|
|
31
33
|
- Validate, register, submit, retry, cancel, and package creation are separate actions. Read-only discovery and validation do not authorize mutation or workload submission.
|
|
32
34
|
- Before submission, show the exact JSON path, immutable identities, expected workload, and target API. Obtain explicit authorization. Retry and cancel require separate authorization for the exact Job.
|
|
33
35
|
- Never use `latest` image tags. During development the canonical SystemModel stays at `v1`; image changes are reviewed Runtime Release updates and require confirming no active Job depends on the previous binding.
|
|
@@ -68,6 +68,10 @@ artifact eligibility, selected runtime, input modes and immutable-version confli
|
|
|
68
68
|
checks used by creation. Both return `status=valid`, the proposed `job_spec` or
|
|
69
69
|
`package`, `submitted=false` and `unchecked` execution checks. They do not register
|
|
70
70
|
Jobs/packages or invoke schedulers. Artifact reads can populate local caches.
|
|
71
|
+
For training, confirm that top-level `effective_trainer_parameters` exactly
|
|
72
|
+
matches `job_spec.trainer_parameters`; this is the complete parameter mapping
|
|
73
|
+
the Worker receives after platform defaults and reproducibility controls are
|
|
74
|
+
resolved.
|
|
71
75
|
|
|
72
76
|
A valid preview does not prove image availability, cluster capacity, credentials,
|
|
73
77
|
framework imports, training success or packaging test success. Read `unchecked`
|
|
@@ -81,9 +85,11 @@ Training Run `task`, `data`, `data.artifact`, `features.feature_set`, `splits`,
|
|
|
81
85
|
`temporal` and `trainer` are explicit objects in OpenAPI. Evaluation artifact,
|
|
82
86
|
config references, members and execution, plus ModelPackage requests, are also
|
|
83
87
|
structured. Unknown keys return 422 with `detail[].loc` identifying the field.
|
|
84
|
-
Keep algorithm-specific `trainer.parameters`
|
|
85
|
-
|
|
86
|
-
|
|
88
|
+
Keep algorithm-specific `trainer.parameters` governed by the SystemModel
|
|
89
|
+
parameter schema. A typed nested contract such as Chronos-2 `lora_config`
|
|
90
|
+
rejects unknown fields; a SystemModel that intentionally advertises
|
|
91
|
+
`additionalProperties` remains extensible. Do not infer one policy from the
|
|
92
|
+
other.
|
|
87
93
|
|
|
88
94
|
These HTTP checks preserve accepted values and do not rewrite historical stored
|
|
89
95
|
contracts or hashes. Legacy records remain readable. A previously ignored typo
|
|
@@ -68,6 +68,59 @@ Non-temporal regression can omit it. Do not change task kind just to evade check
|
|
|
68
68
|
`series_keys` must identify entity-key columns when multiple series coexist;
|
|
69
69
|
an empty list treats the entire artifact as one series.
|
|
70
70
|
|
|
71
|
+
## Chronos-2 LoRA parameters
|
|
72
|
+
|
|
73
|
+
Chronos-2 fine-tuning settings belong under `trainer.parameters`; they are
|
|
74
|
+
frozen with the immutable TrainingRun. This fragment overrides rank and keeps
|
|
75
|
+
the other released LoRA defaults explicit:
|
|
76
|
+
|
|
77
|
+
```json
|
|
78
|
+
{
|
|
79
|
+
"trainer": {
|
|
80
|
+
"project": "default",
|
|
81
|
+
"name": "chronos2_forecaster",
|
|
82
|
+
"version": "v1",
|
|
83
|
+
"parameters": {
|
|
84
|
+
"finetune_mode": "lora",
|
|
85
|
+
"num_steps": 1000,
|
|
86
|
+
"batch_size": 16,
|
|
87
|
+
"learning_rate": 0.00001,
|
|
88
|
+
"gradient_accumulation_steps": 2,
|
|
89
|
+
"logging_steps": 10,
|
|
90
|
+
"lora_config": {
|
|
91
|
+
"r": 16,
|
|
92
|
+
"lora_alpha": 16,
|
|
93
|
+
"lora_dropout": 0.0,
|
|
94
|
+
"target_modules": [
|
|
95
|
+
"self_attention.q",
|
|
96
|
+
"self_attention.v",
|
|
97
|
+
"self_attention.k",
|
|
98
|
+
"self_attention.o",
|
|
99
|
+
"output_patch_embedding.output_layer"
|
|
100
|
+
]
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
},
|
|
104
|
+
"reproducibility": {"seed": 42, "deterministic": true}
|
|
105
|
+
}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Always obtain the accepted fields, target-module enum and defaults from
|
|
109
|
+
`get-system-model chronos2_forecaster v1`; unknown nested LoRA keys are
|
|
110
|
+
rejected before scheduling. `r >= 1`, `lora_alpha >= 1`, and
|
|
111
|
+
`0 <= lora_dropout < 1`. `finetune_mode=full` rejects `lora_config`.
|
|
112
|
+
|
|
113
|
+
The platform materializes omitted values before Job creation. Run validation
|
|
114
|
+
returns `effective_trainer_parameters`; Job validation returns the same object
|
|
115
|
+
both at the top level and in `job_spec.trainer_parameters`. Review that object
|
|
116
|
+
instead of assuming library defaults. Changing any Trainer parameter requires
|
|
117
|
+
a new TrainingRun identity; Job retry cannot alter it.
|
|
118
|
+
|
|
119
|
+
Do not put `prediction_length` in this mapping. It is derived from the temporal
|
|
120
|
+
horizon and frequency. Trainer `context_length` defaults to the sequence input
|
|
121
|
+
adapter's context length, and `min_past` defaults to prediction length. GPU,
|
|
122
|
+
CPU, memory and deadline remain TrainingJob execution fields.
|
|
123
|
+
|
|
71
124
|
## Direct tabular forecasting with known-future features
|
|
72
125
|
|
|
73
126
|
Tree and other matrix-based models can forecast a variable target grid without
|
|
@@ -85,6 +85,70 @@ ml-platform --profile server cancel-training-job <job-id>
|
|
|
85
85
|
|
|
86
86
|
Retry and cancellation require exact-Job authorization. Never change a run contract while retrying; create a new immutable run when semantic inputs change.
|
|
87
87
|
|
|
88
|
+
## Chronos-2 fine-tuning parameters
|
|
89
|
+
|
|
90
|
+
Configure Chronos-2 algorithm parameters only under
|
|
91
|
+
`trainer.parameters` in the TrainingRun. Configure execution device and GPU
|
|
92
|
+
resources later under TrainingJob `execution`; do not mix the two contracts.
|
|
93
|
+
The Run is immutable after registration, so changing rank, learning rate,
|
|
94
|
+
batch size or fine-tuning mode requires a new `run_id`. Retrying an existing
|
|
95
|
+
Job must retain the original parameters.
|
|
96
|
+
|
|
97
|
+
Inspect the same-release contract first:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
ml-platform --profile server get-system-model \
|
|
101
|
+
chronos2_forecaster v1 --project default
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
The returned `parameter_schema` is authoritative. A typical explicit LoRA
|
|
105
|
+
fragment is:
|
|
106
|
+
|
|
107
|
+
```json
|
|
108
|
+
{
|
|
109
|
+
"trainer": {
|
|
110
|
+
"project": "default",
|
|
111
|
+
"name": "chronos2_forecaster",
|
|
112
|
+
"version": "v1",
|
|
113
|
+
"parameters": {
|
|
114
|
+
"finetune_mode": "lora",
|
|
115
|
+
"num_steps": 1000,
|
|
116
|
+
"batch_size": 16,
|
|
117
|
+
"learning_rate": 0.00001,
|
|
118
|
+
"gradient_accumulation_steps": 2,
|
|
119
|
+
"logging_steps": 10,
|
|
120
|
+
"lora_config": {
|
|
121
|
+
"r": 16,
|
|
122
|
+
"lora_alpha": 32,
|
|
123
|
+
"lora_dropout": 0.05
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
},
|
|
127
|
+
"reproducibility": {"seed": 42, "deterministic": true}
|
|
128
|
+
}
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
`r` and `lora_alpha` must be positive integers; `lora_dropout` is in `[0, 1)`.
|
|
132
|
+
Only target modules enumerated by the SystemModel schema are accepted. LoRA is
|
|
133
|
+
the default mode; full fine-tuning rejects `lora_config`. Omitted LoRA settings
|
|
134
|
+
resolve to `r=8`, `lora_alpha=16`, `lora_dropout=0` and the released Chronos-2
|
|
135
|
+
target-module list. Omitted learning rate resolves to `1e-5` for LoRA and
|
|
136
|
+
`1e-6` for full fine-tuning. `min_past` defaults to the derived prediction
|
|
137
|
+
length, while trainer `context_length` defaults to
|
|
138
|
+
`input_adapter.context_length`.
|
|
139
|
+
|
|
140
|
+
Set random seed and determinism only through top-level `reproducibility`.
|
|
141
|
+
Compatibility fields with the same names may appear in the parameter schema,
|
|
142
|
+
but the top-level values are authoritative. Do not configure
|
|
143
|
+
`prediction_length`; the platform derives it as `temporal.horizon /
|
|
144
|
+
temporal.frequency`.
|
|
145
|
+
|
|
146
|
+
After `validate-training-run`, inspect and preserve
|
|
147
|
+
`effective_trainer_parameters`. After Run registration, call
|
|
148
|
+
`validate-training-job` and confirm that its top-level
|
|
149
|
+
`effective_trainer_parameters` exactly equals
|
|
150
|
+
`job_spec.trainer_parameters` before requesting submission authorization.
|
|
151
|
+
|
|
88
152
|
## Chronos-2 future-known inputs
|
|
89
153
|
|
|
90
154
|
For storage forecasting, tomorrow's weather forecast and peak/valley schedule
|