@zhuoyuezs/ml-platform 0.1.0

package/skills/feature-management/references/commands.md

@@ -0,0 +1,244 @@
+# Platform Commands
+
+Use the `ml-platform` executable installed on `PATH` by the same ML Platform release as this Skill. The Skill contains no client runtime or launcher. Do not require a platform source checkout. Use the same API target for discovery, dry-run, publication, resolve, and build.
+
+## Contents
+
+1. Select a target
+2. Discover Registry state
+3. Test and build an Operator
+4. Validate and publish a catalog
+5. Resolve and build a dataset
+6. Fetch realtime inference data
+7. Inspect and download an artifact
+8. Prohibited shortcuts
+
+## Select A Target
+
+Verify the installed CLI before using the Skill:
+
+```bash
+command -v ml-platform
+ml-platform version
+ml-platform --help
+```
+
+Configure the server API for business users:
+
+```bash
+ml-platform configure --api-url http://<platform-api-host>:8060
+ml-platform --profile server show-config
+ml-platform --profile server health
+```
+
+The API URL for this environment is fixed at `http://10.36.9.212:30620`.
+Configure it before all discovery, dry-run, publication, build, and artifact
+commands. The address in the command block is a reusable placeholder; replace
+it with the fixed URL above for this deployment.
+
+Do not place database, MinIO, or source credentials in commands when the server profile can use managed Secrets.
+
+## Discover Registry State
+
+```bash
+ml-platform --profile server list-parameters -q <stable-name>
+ml-platform --profile server list-operators -q <stable-name>
+ml-platform --profile server list-features -q <stable-name>
+ml-platform --profile server list-feature-sets -q <stable-name>
+ml-platform --profile server list-datasets -q <stable-dataset-id>
+```
+
+By default each `list-*` command auto-pages through the whole result set and
+prints a **bare JSON array** of every matching item, so it never silently stops
+at the server's default page size. Use `-q` for a case-insensitive identifier
+substring search. To fetch one explicit page instead, pass `--limit` (1-500)
+and/or `--offset`; in that single-page mode the command prints the
+`{"items": [...], "total": ..., "limit": ..., "offset": ...}` envelope so you can
+resume with the next `--offset`. Note that `--offset` paging is not stable
+under concurrent writes: if items are added or removed between page requests,
+boundary rows can be skipped or repeated. A stable cursor is planned as a
+follow-up. Catalog `apply` and `apply --dry-run` page
+through the entire Registry automatically. Filter structured JSON with a JSON
+parser when the output is large. Do not scrape it with fragile text replacement.
+
+Every `list-*` registry command (except `list-jobs`) accepts `--project <name>`
+to restrict results to a single project. Omit it to list across all projects:
+
+```bash
+ml-platform --profile server list-parameters --project <project>
+ml-platform --profile server list-feature-sets --project <project>
+```
+
+## Projects
+
+A project is an isolated namespace over the whole registry. The real identity of
+every asset is `project/name:version`, so the same `name:version` can exist
+independently under different projects and never collide. The platform ships a
+built-in `default` project used whenever a project is not specified.
+
+Project is supplied differently for writes vs reads:
+
+Manage projects through the business client as well:
+
+```bash
+ml-platform create-project <project> --display-name "<display name>" --description "<description>"
+ml-platform list-projects
+ml-platform get-project <project>
+ml-platform delete-project <project>
+```
+
+`delete-project` is subject to the server's protection and only succeeds for an
+empty project. Project commands are server-only and do not modify local storage.
+
+- Creation: the project an asset is published under comes from the optional
+  `project` field in its own JSON spec (defaults to `default` when omitted). Set
+  `"project": "<name>"` in the spec to publish into a non-default project.
+- `apply --project <project>` scopes the comparison snapshot the catalog is
+  validated against (omit to compare against `default`);
+- `list-* --project <project>` filters discovery to that project;
+- `get-dataset-artifact` / `download-dataset-artifact` take the project as their
+  first positional argument (defaults to `default`).
+
+Before choosing names, discover within the target project: identical names in a
+different project are unrelated and must not be reused across projects to mean the
+same thing. Cross-project references are rejected by the platform.
+
+## Test And Build An Operator
+
+From the Operator package directory:
+
+```bash
+uv run python -m unittest discover -s tests -p 'test_*.py'
+```
+
+Build the package:
+
+```bash
+uv build <catalog>/operator_package \
+  --wheel \
+  --out-dir <catalog>/operator_package/dist
+```
+
+Confirm that `catalog.json` references the exact generated filename.
+
+## Validate And Publish A Catalog
+
+Validate against the intended server Registry without mutation:
+
+```bash
+ml-platform --profile server apply <catalog-directory> --dry-run
+```
+
+Review the returned counts and conflicts. After explicit user approval, publish:
+
+```bash
+ml-platform --profile server apply <catalog-directory>
+```
+
+Add `--project <project>` to publish into a specific project and compare against
+that project's snapshot only. Without it, the catalog is applied to the built-in
+`default` project:
+
+```bash
+ml-platform --profile server apply <catalog-directory> --project <project> --dry-run
+ml-platform --profile server apply <catalog-directory> --project <project>
+```
+
+`apply` processes dependencies in this order:
+
+```text
+Parameter -> Operator package/Operator -> Feature -> FeatureSet -> Dataset
+```
+
+The cross-store publication is not transactional. If an error occurs, report the returned progress summary and rerun only after understanding which immutable assets were already published.
+
+## Resolve And Build A Dataset
+
+Resolve without fetching source data:
+
+```bash
+ml-platform --profile server \
+  resolve-manifest <catalog>/datasets/<dataset>.json \
+  --out /tmp/<dataset>.resolved.json
+```
+
+Submit and wait for a server build:
+
+```bash
+ml-platform --profile server \
+  build-dataset <catalog>/datasets/<dataset>.json \
+  --partition-duration 1d \
+  --max-parallelism 1 \
+  --wait
+```
+
+`--partition-duration` is an execution control and does not change the
+DatasetManifest hash. The initial implementation is sequential, so keep
+`--max-parallelism 1`. Completed partitions are checkpointed and resumed by
+default; use `--no-resume` only to force a clean execution.
+
+Record the returned `job_id`, `dataset_id`, and `manifest_hash`. If a separate wait is needed:
+
+```bash
+ml-platform --profile server wait-job <job_id>
+ml-platform --profile server get-job <job_id>
+```
+
+## Fetch Realtime Inference Data
+
+Fetch one causal-cutoff row in memory:
+
+```bash
+ml-platform --profile server \
+  fetch-inference-data <catalog>/datasets/<dataset>.json \
+  --cutoff-time 2026-07-31T10:00:00+08:00
+```
+
+Optional flags `--max-workers`, `--max-source-lag-hours`,
+`--no-validate-freshness`, and `--allow-missing` override individual
+`realtime_fetch` fields for this request. Prefer the versioned manifest policy;
+use overrides only when the user explicitly requests different runtime behavior.
+
+This command calls `POST /inference-data/fetch`. It does not submit a build Job,
+write parquet, or publish a DatasetArtifact. Treat HTTP `503` as retryable source
+freshness/missing-data failure and HTTP `400` as a contract error that requires a
+manifest or request change.
+
+## Inspect And Download An Artifact
+
+```bash
+ml-platform --profile server \
+  get-dataset-artifact <dataset_id> <manifest_hash> --project <project>
+
+ml-platform --profile server \
+  download-dataset-artifact <dataset_id> <manifest_hash> \
+  --project <project> \
+  --out-dir /tmp/<dataset_id>-artifact
+```
+
+`--project` defaults to `default`; pass the owning project when the dataset lives
+elsewhere, or the lookup returns `artifact_not_found`.
+
+Inspect at least:
+
+```text
+dataset_manifest.json
+resolved_manifest.json
+validation.json
+lineage.json
+feature_computations/computation_manifest.json
+feature_computations/executions/*.json
+feature_dataset.parquet
+```
+
+Use a structured Parquet reader for schema, row count, column order, and missing-rate inspection.
+
+## Prohibited Shortcuts
+
+- Do not use `seed-demo v95` for business catalogs.
+- Do not call individual `add-*` commands when catalog `apply` is available for an end-to-end publication.
+- Do not use `delete-dataset`, `cancel-job`, or force overwrite operations unless separately requested.
+- Do not build images, deploy Kubernetes, or change service configuration in this workflow.
+- Do not claim a successful publication from `--dry-run` output.
+- Do not claim Chronon compile/backfill from a successful first-phase Operator build.
+- Do not use `build-dataset` to serve a single realtime inference cutoff.