@zhuoyuezs/ml-platform 0.1.1 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DEVELOPMENT.md +96 -9
- package/README.md +89 -38
- package/checksums.json +37 -32
- package/package.json +5 -1
- package/release-policy.json +10 -0
- package/release.json +13 -9
- package/runtime/business-client/README.md +13 -0
- package/runtime/business-client/package-lock.json +2 -2
- package/runtime/business-client/package.json +1 -1
- package/runtime/business-client/src/catalog.js +40 -17
- package/runtime/business-client/src/cli.js +150 -26
- package/runtime/business-client/src/config.js +6 -2
- package/runtime/business-client/src/http.js +88 -17
- package/scripts/lib.js +98 -4
- package/scripts/main.js +96 -11
- package/skills/feature-management/SKILL.md +187 -18
- package/skills/feature-management/assets/catalog-template/datasets/example_temperature_training.v1.json +16 -0
- package/skills/feature-management/assets/catalog-template/feature_sets/example_temperature_core.v1.json +1 -0
- package/skills/feature-management/assets/catalog-template/features/example_temperature_mean_5m.v1.json +9 -2
- package/skills/feature-management/assets/catalog-template/operator_package/tests/test_operator.py +51 -24
- package/skills/feature-management/assets/catalog-template/operators/example_temperature_features.v1.json +16 -1
- package/skills/feature-management/assets/catalog-template/parameters/example_temperature.v1.json +1 -0
- package/skills/feature-management/references/commands.md +66 -4
- package/skills/feature-management/references/contracts.md +43 -6
- package/skills/feature-management/references/operator-authoring.md +15 -7
- package/skills/feature-management/references/platform-capability-guide.md +44 -0
|
@@ -35,6 +35,9 @@ Use this reference for the implemented V1 schemas. Prefer the running CLI and `s
|
|
|
35
35
|
`project` field in its own JSON spec (it defaults to `default` when omitted).
|
|
36
36
|
So to place an asset in a non-default project, set `"project": "<name>"` in the
|
|
37
37
|
spec before `apply`.
|
|
38
|
+
- For a confirmed non-default target, include that field in every draft
|
|
39
|
+
Parameter, Operator, Feature, Feature input, FeatureSet, and DatasetManifest;
|
|
40
|
+
do not rely on Pydantic's `default` fallback during authoring.
|
|
38
41
|
- Reads and filters, by contrast, take the project out-of-band: `list-* --project`
|
|
39
42
|
and the leading project argument of the artifact commands. `apply --project`
|
|
40
43
|
scopes the comparison snapshot that publication is validated against.
|
|
@@ -91,6 +94,11 @@ Required shape:
|
|
|
91
94
|
|
|
92
95
|
Supported source modes are `direct_column`, `sql`, and `derived`. Prefer `direct_column` for ordinary business parameters.
|
|
93
96
|
|
|
97
|
+
For `sql`, `source.sql` must contain one statement without a trailing semicolon,
|
|
98
|
+
and named placeholders must be supplied through `source.params`; `start` and
|
|
99
|
+
`end` are reserved runtime window parameters. `source.parameters` is not a
|
|
100
|
+
supported field.
|
|
101
|
+
|
|
94
102
|
`availability.strategy` 支持 `source_field`、`fixed_delay` 和 `unsupported`。只有一个
|
|
95
103
|
事件时间列时,可以使用经过数据 owner 确认的固定延迟上界,例如
|
|
96
104
|
`{"strategy":"fixed_delay","delay":"PT10M","accuracy":"upper_bound"}`。
|
|
@@ -282,6 +290,14 @@ The list must be nonempty, unique, and ordered exactly as consumers expect.
|
|
|
282
290
|
"horizon": "10min"
|
|
283
291
|
},
|
|
284
292
|
"parameters": [],
|
|
293
|
+
"parameter_missing_policies": [
|
|
294
|
+
{
|
|
295
|
+
"parameter": "hot_blast_pressure",
|
|
296
|
+
"version": "v1",
|
|
297
|
+
"project": "default",
|
|
298
|
+
"policy": "report_only"
|
|
299
|
+
}
|
|
300
|
+
],
|
|
285
301
|
"feature_set": {"name": "pressure_core", "version": "v1"},
|
|
286
302
|
"preprocess": [],
|
|
287
303
|
"output": {
|
|
@@ -321,9 +337,13 @@ declared `realtime_fetch` policy contains meaningful defaults and changes the
|
|
|
321
337
|
manifest hash. Declaring `endpoint_policy` also changes the manifest hash.
|
|
322
338
|
|
|
323
339
|
Each explicit `parameters` request may set `missing_policy`. A
|
|
324
|
-
`parameter_missing_policies` entry
|
|
325
|
-
|
|
326
|
-
|
|
340
|
+
`parameter_missing_policies` entry is reserved for a Parameter used only through
|
|
341
|
+
the selected FeatureSet's input closure; it must not also appear in
|
|
342
|
+
`parameters`. Its exact V1 shape is `{"parameter": "<name>", "version":
|
|
343
|
+
"<version>", "project": "<project>", "policy": "<policy>"}`. Do not use
|
|
344
|
+
the `ParameterRequest` field names `name` or `missing_policy` in this list. For
|
|
345
|
+
a confirmed non-default target, set `project` explicitly instead of relying on
|
|
346
|
+
its `default` fallback. The policies are:
|
|
327
347
|
|
|
328
348
|
- `report_only`: retain all target rows and report source gaps.
|
|
329
349
|
- `fail_on_requested_range_gap`: fail when a detected gap affects the requested
|
|
@@ -424,13 +444,30 @@ near the end of the target range can have its lookback clipped by the range end,
|
|
|
424
444
|
and a window close to the end can remove most remaining rows.
|
|
425
445
|
|
|
426
446
|
`target` states which instant the label is aligned to, so reviewing a manifest
|
|
427
|
-
shows the target definition rather than only a Parameter version string.
|
|
428
|
-
|
|
447
|
+
shows the target definition rather than only a Parameter version string. When
|
|
448
|
+
the target is a Parameter, keep its complete versioned reference **inside**
|
|
449
|
+
`target.parameter`; do not flatten `version` or `project` onto `target`.
|
|
450
|
+
`si_time_source` is `weight_time` or `dispatch_time`. `interpolation` is an
|
|
451
|
+
object with a `method`, never a bare string.
|
|
429
452
|
|
|
430
453
|
```json
|
|
431
|
-
"target": {
|
|
454
|
+
"target": {
|
|
455
|
+
"parameter": {
|
|
456
|
+
"parameter": "hot_metal_si",
|
|
457
|
+
"version": "v1",
|
|
458
|
+
"project": "replace_with_project"
|
|
459
|
+
},
|
|
460
|
+
"si_time_source": "weight_time",
|
|
461
|
+
"offset_minutes": 0,
|
|
462
|
+
"interpolation": {"method": "linear"}
|
|
463
|
+
}
|
|
432
464
|
```
|
|
433
465
|
|
|
466
|
+
Allowed interpolation methods are `none`, `linear`, and `forward_fill`. A
|
|
467
|
+
business phrase such as "allow and report target nulls" belongs in an existing
|
|
468
|
+
Parameter missing policy, Feature configuration, or the semantic review; it is
|
|
469
|
+
not a free-form `target` or DatasetManifest field.
|
|
470
|
+
|
|
434
471
|
The declaration is checked against the selected Si Parameter, but only for
|
|
435
472
|
`direct_column` reads, which name their time column in the contract. A `sql` or
|
|
436
473
|
`derived` Parameter can apply an offset inside its query text, so those are
|
|
@@ -67,12 +67,17 @@ Use `context.cutoff_times` for causal feature windows. It is derived once from
|
|
|
67
67
|
`context.target_times - context.prediction_horizon` using the DatasetManifest
|
|
68
68
|
prediction contract; do not parse a duplicated Feature-level horizon.
|
|
69
69
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
70
|
+
An Operator requires an explicit forecast contract whenever its formula or code
|
|
71
|
+
uses `context.cutoff_times`, or any `input_schema.history_requirements` entry uses
|
|
72
|
+
`anchor: cutoff`. In that case always declare `input_schema.prediction` with
|
|
73
|
+
`required`, `minimum_horizon`, and `maximum_horizon`. This includes a strict
|
|
74
|
+
nowcast: declare `required=true` and both bounds as `0min`; do not omit the
|
|
75
|
+
section and silently accept the runtime's `default_nowcast` fallback. Declare
|
|
76
|
+
source history as `input_schema.history_requirements`, using `anchor: cutoff`
|
|
77
|
+
for causal windows and `anchor: target_time` only for intentional target-aligned
|
|
78
|
+
formulas. Scope a requirement with `output_columns` when only some outputs need
|
|
79
|
+
it. Include alignment slack such as hourly floor boundaries in the declared
|
|
80
|
+
lookback.
|
|
76
81
|
|
|
77
82
|
For realtime reads, `history_requirements` is also the per-Parameter fetch
|
|
78
83
|
contract. Do not rely on a package-local lookback map or a single fixed window
|
|
@@ -110,7 +115,10 @@ Never infer a rule from a Feature name alone. Require a business decision when a
|
|
|
110
115
|
|
|
111
116
|
## Required Tests
|
|
112
117
|
|
|
113
|
-
Use deterministic fixtures and cover
|
|
118
|
+
Use deterministic fixtures and cover each applicable item with a separately
|
|
119
|
+
named test. A broad formula or happy-path test does not substitute for the
|
|
120
|
+
cutoff-before, cutoff-at, cutoff-after, duplicate/missing, empty-history,
|
|
121
|
+
requested-output, event-time-order, or dtype/rounding cases below:
|
|
114
122
|
|
|
115
123
|
1. expected formula values;
|
|
116
124
|
2. the event immediately before cutoff;
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Platform Capability Guide
|
|
2
|
+
|
|
3
|
+
Use this guide to explain what the platform can execute. It is not business
|
|
4
|
+
evidence. A platform default never answers an unresolved business question.
|
|
5
|
+
|
|
6
|
+
## Public Assets
|
|
7
|
+
|
|
8
|
+
| Asset | Stores | Use when | Do not use for |
|
|
9
|
+
|---|---|---|---|
|
|
10
|
+
| Project | Namespace and isolation boundary | Every independent business release | A dataset version or a source table |
|
|
11
|
+
| Parameter | One readable source contract and its quality/time semantics | Raw or independently readable source values/events | Rolling means, ratios, trends, model columns |
|
|
12
|
+
| Operator | Versioned deterministic executable code | A formula needs computation over declared inputs | A business definition without confirmed formula |
|
|
13
|
+
| Feature | One immutable output-column contract | One model/input column produced by an Operator | A group of columns or an unnamed formula |
|
|
14
|
+
| FeatureSet | Ordered Feature references | Consumer column order is part of the contract | Copying formulas or creating multiple datasets |
|
|
15
|
+
| DatasetManifest | Dataset mode, time grid, rowsets, policy and FeatureSet reference | A reproducible dataset contract is ready | Filling unknown source or business semantics |
|
|
16
|
+
| DatasetArtifact | Output of an authorized build | Build was explicitly approved and completed | Proving a Catalog is semantically correct |
|
|
17
|
+
|
|
18
|
+
## Decision Rules
|
|
19
|
+
|
|
20
|
+
- Parameter answers **what source value is exposed**; Operator/Feature answers
|
|
21
|
+
**how a confirmed business formula is computed**.
|
|
22
|
+
- FeatureSet order is consumer-facing and must be confirmed; it is not inferred
|
|
23
|
+
from filesystem order or JSON discovery order.
|
|
24
|
+
- Dataset `time_range`, `prediction`, `rowset_splits`, `abnormal_windows` and
|
|
25
|
+
`endpoint_policy` are separate contracts. A platform default does not choose
|
|
26
|
+
a business policy.
|
|
27
|
+
- `apply --dry-run` checks structure, references, package and immutability. It
|
|
28
|
+
does not prove source correctness, formula correctness or artifact parity.
|
|
29
|
+
- `publish` creates immutable Registry resources. `build` creates an execution
|
|
30
|
+
Job and must have a separate explicit approval.
|
|
31
|
+
|
|
32
|
+
## Required Semantic Asset Review
|
|
33
|
+
|
|
34
|
+
Before writing Catalog JSON, produce a local review table with one row per
|
|
35
|
+
Parameter, Operator, Feature and Dataset field:
|
|
36
|
+
|
|
37
|
+
```text
|
|
38
|
+
asset_key | business_meaning | source/evidence | confirmed_by | unresolved | proposed_value
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Stop before Catalog generation if any required `source/evidence`, formula,
|
|
42
|
+
time boundary, unit, null policy, output dtype, FeatureSet order, read policy,
|
|
43
|
+
or approval field is unresolved. The review table is a proposal for the user;
|
|
44
|
+
it is not a Registry asset and must not contain guessed values.
|