kiteml-cli 0.3.1__tar.gz → 0.4.0__tar.gz

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: kiteml-cli
-Version: 0.3.1
+Version: 0.4.0
 Summary: Kite CLI and MCP server: train robot policies with reinforcement learning in simulation, augment LeRobot datasets, and build MuJoCo digital twins, from your terminal or an AI agent.
 Project-URL: Homepage, https://kiteml.com
 Project-URL: Documentation, https://kite-ml.mintlify.app/tools/cli-and-mcp
@@ -59,8 +59,9 @@ Model Context Protocol (MCP) client.
   Hugging Face.
 - **Digital twins (beta).** Point Kite at one episode of a LeRobot dataset and get an interactable MuJoCo scene of the
   room, with the objects the robot handles built to size.
-- **Sim runs (beta).** Run a Kite SDK spec on Kite's machines, on an NVIDIA L4 for a GPU policy: every episode traced, every
-  evaluator scored. You get a scorecard and the run directory, which `kite verify` checks.
+- **Evaluations (beta).** Evaluate a policy by running a Kite SDK spec on Kite's machines, on an NVIDIA L4 for a GPU
+  policy: every episode traced, every evaluator scored. You get a scorecard and the run directory, which
+  `kite verify` checks.
 
 ## Install
 
@@ -108,7 +109,7 @@ lerobot/pusht to look like this photo".
 | `kite_rl_status`, `kite_rl_metrics`, `kite_rl_report`, `kite_rl_list` | Follow training and read each run's evaluation verdict |
 | `kite_augment_create`, `kite_augment_status`, `kite_augment_list`, `kite_augment_cancel` | Augment LeRobot datasets |
 | `kite_twin_validate`, `kite_twin_create`, `kite_twin_status`, `kite_twin_list`, `kite_twin_cancel`, `kite_twin_resume` | Build MuJoCo digital twins (beta) |
-| `kite_sim_validate`, `kite_sim_run`, `kite_sim_status`, `kite_sim_list`, `kite_sim_report`, `kite_sim_logs`, `kite_sim_cancel`, `kite_sim_resume` | Run Kite SDK specs on Kite's machines and read their scorecards (beta) |
+| `kite_eval_validate`, `kite_eval_run`, `kite_eval_status`, `kite_eval_list`, `kite_eval_report`, `kite_eval_logs`, `kite_eval_cancel`, `kite_eval_resume` | Evaluate a policy on a Kite SDK spec on Kite's machines and read the scorecard (beta) |
 | `kite_doctor` | Check connectivity and authentication |
 
 ## Use it from the terminal
@@ -137,11 +138,11 @@ Build a digital twin (beta, enabled per account):
 kite twin create lerobot/svla_so101_pickplace --out ./twins   # -> ./twins/twin_.../scene.xml
 ```
 
-Run a Kite SDK spec on Kite's machines (beta):
+Evaluate a policy on a Kite SDK spec, on Kite's machines (beta):
 
 ```bash
-kite sim validate specs/smoke.yaml             # free: the plan and what a run reserves
-kite sim run specs/smoke.yaml --out ./runs     # -> ./runs/srun_.../scorecard.json
+kite eval validate specs/smoke.yaml            # free: the plan and what a run reserves
+kite eval run specs/smoke.yaml --out ./runs    # -> ./runs/eval_.../scorecard.json
 ```
 
 Every command prints JSON (`{"ok": true, "data": ...}`) so scripts and agents can parse it. Run `kite --help` for
@@ -152,7 +153,8 @@ the full list.
 - Docs: [CLI and MCP](https://kite-ml.mintlify.app/tools/cli-and-mcp) ·
   [RL runs](https://kite-ml.mintlify.app/platform-api/rl-runs) ·
   [Augmentations](https://kite-ml.mintlify.app/platform-api/augmentation) ·
-  [Twins](https://kite-ml.mintlify.app/platform-api/twins)
+  [Twins](https://kite-ml.mintlify.app/platform-api/twins) ·
+  [Evaluations](https://kite-ml.mintlify.app/platform-api/evaluations)
 - Dashboard: [app.kiteml.com](https://app.kiteml.com)
 - Website: [kiteml.com](https://kiteml.com)
 - Support: [raul@kiteml.com](mailto:raul@kiteml.com)

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/README.md

@@ -17,8 +17,9 @@ Model Context Protocol (MCP) client.
   Hugging Face.
 - **Digital twins (beta).** Point Kite at one episode of a LeRobot dataset and get an interactable MuJoCo scene of the
   room, with the objects the robot handles built to size.
-- **Sim runs (beta).** Run a Kite SDK spec on Kite's machines, on an NVIDIA L4 for a GPU policy: every episode traced, every
-  evaluator scored. You get a scorecard and the run directory, which `kite verify` checks.
+- **Evaluations (beta).** Evaluate a policy by running a Kite SDK spec on Kite's machines, on an NVIDIA L4 for a GPU
+  policy: every episode traced, every evaluator scored. You get a scorecard and the run directory, which
+  `kite verify` checks.
 
 ## Install
 
@@ -66,7 +67,7 @@ lerobot/pusht to look like this photo".
 | `kite_rl_status`, `kite_rl_metrics`, `kite_rl_report`, `kite_rl_list` | Follow training and read each run's evaluation verdict |
 | `kite_augment_create`, `kite_augment_status`, `kite_augment_list`, `kite_augment_cancel` | Augment LeRobot datasets |
 | `kite_twin_validate`, `kite_twin_create`, `kite_twin_status`, `kite_twin_list`, `kite_twin_cancel`, `kite_twin_resume` | Build MuJoCo digital twins (beta) |
-| `kite_sim_validate`, `kite_sim_run`, `kite_sim_status`, `kite_sim_list`, `kite_sim_report`, `kite_sim_logs`, `kite_sim_cancel`, `kite_sim_resume` | Run Kite SDK specs on Kite's machines and read their scorecards (beta) |
+| `kite_eval_validate`, `kite_eval_run`, `kite_eval_status`, `kite_eval_list`, `kite_eval_report`, `kite_eval_logs`, `kite_eval_cancel`, `kite_eval_resume` | Evaluate a policy on a Kite SDK spec on Kite's machines and read the scorecard (beta) |
 | `kite_doctor` | Check connectivity and authentication |
 
 ## Use it from the terminal
@@ -95,11 +96,11 @@ Build a digital twin (beta, enabled per account):
 kite twin create lerobot/svla_so101_pickplace --out ./twins   # -> ./twins/twin_.../scene.xml
 ```
 
-Run a Kite SDK spec on Kite's machines (beta):
+Evaluate a policy on a Kite SDK spec, on Kite's machines (beta):
 
 ```bash
-kite sim validate specs/smoke.yaml             # free: the plan and what a run reserves
-kite sim run specs/smoke.yaml --out ./runs     # -> ./runs/srun_.../scorecard.json
+kite eval validate specs/smoke.yaml            # free: the plan and what a run reserves
+kite eval run specs/smoke.yaml --out ./runs    # -> ./runs/eval_.../scorecard.json
 ```
 
 Every command prints JSON (`{"ok": true, "data": ...}`) so scripts and agents can parse it. Run `kite --help` for
@@ -110,7 +111,8 @@ the full list.
 - Docs: [CLI and MCP](https://kite-ml.mintlify.app/tools/cli-and-mcp) ·
   [RL runs](https://kite-ml.mintlify.app/platform-api/rl-runs) ·
   [Augmentations](https://kite-ml.mintlify.app/platform-api/augmentation) ·
-  [Twins](https://kite-ml.mintlify.app/platform-api/twins)
+  [Twins](https://kite-ml.mintlify.app/platform-api/twins) ·
+  [Evaluations](https://kite-ml.mintlify.app/platform-api/evaluations)
 - Dashboard: [app.kiteml.com](https://app.kiteml.com)
 - Website: [kiteml.com](https://kiteml.com)
 - Support: [raul@kiteml.com](mailto:raul@kiteml.com)

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/src/kite_cli/__init__.py

@@ -1,3 +1,3 @@
 """Kite CLI and MCP server for the Kite robotics training platform."""
 
-__version__ = "0.3.1"
+__version__ = "0.4.0"

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/src/kite_cli/archive.py

@@ -1,4 +1,4 @@
-"""Fetching a finished resource's `.tar.gz` (a twin, a sim run): to `<out>/<id>.tar.gz`, sha256-verified, and unpacked
+"""Fetching a finished resource's `.tar.gz` (a twin, an evaluation): to `<out>/<id>.tar.gz`, sha256-verified, and unpacked
 only inside `<out>/<id>/`.
 
 A signed storage URL is fetched with a plain HTTP client, never with the Kite key; the API's own archive route

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/src/kite_cli/cli/commands/augment.py

@@ -1,7 +1,8 @@
 """`kite augment` — dataset augmentation jobs via the public /v1 API.
 
-create / status / download / cancel / list. Authenticates with KITE_API_KEY like the
-rest of the CLI; calls the bare `/v1/augmentations` surface.
+create / status / download / cancel / list, and `environments list` (what the background and
+world engines put behind the work cell). Authenticates with KITE_API_KEY like the rest of the
+CLI; calls the bare `/v1/augmentations` surface.
 """
 
 from __future__ import annotations
@@ -18,6 +19,9 @@ from kite_cli.output import print_error, print_json
 from kite_cli.progress import wait_for_completion
 
 app = typer.Typer(no_args_is_help=True, help="Dataset augmentation jobs")
+environments_app = typer.Typer(no_args_is_help=True, help="Environments the background and world engines put "
+                                                          "behind the work cell")
+app.add_typer(environments_app, name="environments")
 
 _TERMINAL = {"succeeded", "failed", "canceled"}
 _MEDIA_TYPES = {".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".png": "image/png", ".webp": "image/webp"}
@@ -33,13 +37,57 @@ def reference_image(path: str, camera: Optional[str] = None) -> dict:
     return {"camera": camera, **entry} if camera else entry
 
 
+def scene_config(episodes, model: Optional[str], variants: Optional[int] = None,
+                 preview: Optional[bool] = None, allow_video_fallback: Optional[bool] = None) -> dict:
+    """The /v1 ``config`` for an episode selection ("all" or a count) and the scene engines' options: background /
+    world augment whole datasets and take ``episodes``; every other model takes a count as ``episode_count`` ("all"
+    sends none: relight covers every episode, video and auto keep their default)."""
+    config: dict = {}
+    if episodes is not None:
+        if str(episodes) != "all" and not str(episodes).isdigit():
+            raise typer.BadParameter("a count or 'all'", param_hint="--episodes")
+        if model in ("background", "world"):
+            config["episodes"] = "all" if str(episodes) == "all" else int(episodes)
+        elif str(episodes) != "all":
+            config["episode_count"] = int(episodes)
+    if variants is not None:
+        config["variants"] = variants
+    if preview is not None:
+        config["preview_first"] = preview
+    if allow_video_fallback:
+        config["allow_video_fallback"] = True
+    return config
+
+
 @app.command()
 def create(
     repo_id: str = typer.Option(..., "--repo-id", help="Source HF dataset id, e.g. lerobot/pusht"),
     instructions: str = typer.Option(..., "-i", "--instructions", help="Augmentation instruction"),
-    episodes: int = typer.Option(3, "-n", "--episodes", help="Episodes to generate with video augmentation (1-50); relight covers every episode"),
+    episodes: Optional[str] = typer.Option(
+        None, "-n", "--episodes",
+        help="A count (video: 1-50, default 3) or 'all' (background/world, their default); relight covers every "
+        "episode",
+    ),
     model: Optional[str] = typer.Option(
-        None, "--model", help="auto (default: pick relight or video) | relight (colour/exposure, every episode) | video (generative)"
+        None, "--model",
+        help="auto (default: the planner picks per camera) | relight (colour/exposure) | background (a panorama behind "
+        "the work cell) | world (a 3D world, for moving cameras) | video (generative)",
+    ),
+    environments: Optional[str] = typer.Option(
+        None, "--environments", help="Background/world: comma-separated environment ids (aenv_...) or library ids"
+    ),
+    env_count: Optional[int] = typer.Option(None, "--env-count", help="Background/world: environments to pick for the rig"),
+    must_keep: Optional[List[str]] = typer.Option(
+        None, "--must-keep", help="Something that must keep its recorded pixels (repeatable)"
+    ),
+    variants: Optional[int] = typer.Option(None, "--variants", help="Background/world: environments per source episode"),
+    preview: Optional[bool] = typer.Option(
+        None, "--preview/--no-preview", help="Background/world: check one episode's quality gates before the rest"
+    ),
+    allow_video_fallback: bool = typer.Option(
+        False, "--allow-video-fallback",
+        help="auto: use generative video (far more tokens, no pixel guarantee) where no scene engine can run, "
+        "instead of failing",
     ),
     reference_images: Optional[List[str]] = typer.Option(
         None, "--reference-image", help="Photo of the target look, [camera=]PATH (repeatable)"
@@ -59,13 +107,18 @@ def create(
     body: dict = {
         "source": {"type": "huggingface", "repo_id": repo_id},
         "instructions": instructions,
-        "config": {"episode_count": episodes},
+        "config": scene_config(episodes, model, variants, preview, allow_video_fallback),
         "output": {"type": output, "private": private},
     }
     if hf_repo:
         body["output"]["repo_id"] = hf_repo
     if model:
         body["model"] = model
+    if environments or env_count:
+        body["environments"] = {k: v for k, v in (
+            ("ids", [e.strip() for e in (environments or "").split(",") if e.strip()]), ("count", env_count)) if v}
+    if must_keep:
+        body["must_keep"] = list(must_keep)
     try:
         if relight:
             body["config"]["relight"] = json.loads(relight)
@@ -135,6 +188,23 @@ def cancel(aug_id: str = typer.Argument(..., help="Augmentation id (aug_...)")):
         print_error(e.message, code=e.code, details=e.details)
 
 
+@environments_app.command("list")
+def list_environments(
+    owner: str = typer.Option("user", "--owner", help="user (yours) or kite (the Kite library)"),
+    kind: Optional[str] = typer.Option(None, "--kind", help="panorama or world"),
+    category: Optional[str] = typer.Option(None, "--category", help="workshop, warehouse, hangar, lab, ..."),
+    lighting: Optional[str] = typer.Option(None, "--lighting", help="daylight or artificial"),
+    limit: int = typer.Option(20, "--limit", help="Max rows (1-100)"),
+):
+    """List environments: yours, or the Kite library's (--owner kite)."""
+    params = {"owner": owner, "kind": kind, "category": category, "lighting": lighting, "limit": limit}
+    try:
+        client = KiteClient()
+        print_json(client.get("/v1/augmentations/environments", params={k: v for k, v in params.items() if v}))
+    except KiteAPIError as e:
+        print_error(e.message, code=e.code, details=e.details)
+
+
 @app.command()
 def download(
     aug_id: str = typer.Argument(..., help="Augmentation id (aug_...)"),

kiteml_cli-0.3.1/src/kite_cli/cli/commands/sim.py → kiteml_cli-0.4.0/src/kite_cli/cli/commands/evaluations.py

@@ -1,22 +1,23 @@
-"""`kite sim` — a Kite SDK spec run on Kite's machines via the public /v1/sim_runs API: the spec in, a scorecard and a
-verifiable run directory out. The policy runs on an NVIDIA L4 if its device is `cuda`, on a CPU machine otherwise.
-
-  kite sim validate specs/smoke.yaml [--set episodes.count=20]   free: the plan, and what a run reserves
-  kite sim run      specs/smoke.yaml --wait --out ./runs         -> ./runs/srun_.../scorecard.json
-  kite sim status   srun_01K...   [--wait]
-  kite sim list     [--limit 20] [--after srun_...]
-  kite sim report   srun_01K...   the scorecard
-  kite sim logs     srun_01K...   [--tail 200]
-  kite sim files    srun_01K...   [--limit 100] [--after PATH] [--all]
-  kite sim download srun_01K...   [--out DIR] [--no-extract]
-  kite sim cancel   srun_01K...   -> canceling, then canceled once its job has stopped
-  kite sim resume   srun_01K...
+"""`kite eval` — an evaluation: a Kite SDK spec run on Kite's machines via the public /v1/evaluations API, the spec
+in, a scorecard and a verifiable run directory out. The policy runs on an NVIDIA L4 if its device is `cuda`, on a CPU
+machine otherwise. (Until 2026-10-18 this was `kite sim`, over /v1/sim_runs.)
+
+  kite eval validate specs/smoke.yaml [--set episodes.count=20]   free: the plan, and what a run reserves
+  kite eval run      specs/smoke.yaml --wait --out ./runs         -> ./runs/eval_.../scorecard.json
+  kite eval status   eval_01K...   [--wait]
+  kite eval list     [--limit 20] [--after eval_...]
+  kite eval report   eval_01K...   the scorecard
+  kite eval logs     eval_01K...   [--tail 200]
+  kite eval files    eval_01K...   [--limit 100] [--after PATH] [--all]
+  kite eval download eval_01K...   [--out DIR] [--no-extract]
+  kite eval cancel   eval_01K...   -> canceling, then canceled once its job has stopped
+  kite eval resume   eval_01K...
 
 SPEC is the file `kite run` takes (kite_spec_version "2"). The command sends its text, asks the API which paths it
 names (`files_required`), and sends each from this disk: a file or folder Kite ships by its sha256 alone, anything
 else inline (about 15 MB in all). A file on the Hugging Face Hub, such as a checkpoint's weights, goes as a
 reference: `--hf PATH=OWNER/NAME@COMMIT:FILE`, PATH as the spec names it (a folder's file as FOLDER/FILE).
-Where runs go: --out DIR, else $KITE_SIM_RUNS_DIR, else ./runs (`~` works). Paths printed are absolute. A run
+Where runs go: --out DIR, else $KITE_EVAL_RUNS_DIR, else ./runs (`~` works). Paths printed are absolute. A run
 directory extracts to <out>/<run_id>/; the SDK's `kite verify <out>/<run_id> --evals DIR` checks it, DIR being the
 evaluators the spec named (the run records where its job read them, not your folder).
 """
@@ -42,7 +43,7 @@ from kite_cli.client import KiteAPIError, KiteClient, post_create
 from kite_cli.output import print_error, print_json
 from kite_cli.progress import wait_for_completion
 
-app = typer.Typer(no_args_is_help=True, help="Sim runs: a Kite SDK spec on Kite's machines -> a scorecard and a "
+app = typer.Typer(no_args_is_help=True, help="Evaluations: a Kite SDK spec on Kite's machines -> a scorecard and a "
                                              "verifiable run directory")
 
 _TERMINAL = {"succeeded", "failed", "canceled"}  # not canceling: the cancel was accepted, the job has not stopped
@@ -104,37 +105,37 @@ def run(
                                      ("verify_determinism", verify_determinism)) if v}
         body = {**body, **({"options": options} if options else {}), **({"label": label} if label else {})}
         try:
-            srun = post_create(client, "/v1/sim_runs", body, key)
+            run = post_create(client, "/v1/evaluations", body, key)
         except (httpx.TimeoutException, httpx.NetworkError) as e:
             print_error(f"network error ({e}); re-run with --idempotency-key {key} to retry without running twice",
                         code="network_error", details={"idempotency_key": key})
             return
-        if not (wait or out) or not srun.get("id"):
-            print_json(srun)
+        if not (wait or out) or not run.get("id"):
+            print_json(run)
             return
-        srun = wait_for_completion(client, f"/v1/sim_runs/{srun['id']}", terminal_states=_TERMINAL,
-                                   poll_interval=_POLL_S)
-        if srun.get("status") != "succeeded":
-            print_json(srun, success=False)
+        run = wait_for_completion(client, f"/v1/evaluations/{run['id']}", terminal_states=_TERMINAL,
+                                  poll_interval=_POLL_S)
+        if run.get("status") != "succeeded":
+            print_json(run, success=False)
             raise typer.Exit(1)
         if out is None:
-            print_json(srun)
+            print_json(run)
             return
-        saved = _save(client, srun, _out_dir(out), extract=True)
-        print_json({"ok": True, "data": srun, "run_dir": saved["run_dir"], "archive": saved["archive"]})
+        saved = _save(client, run, _out_dir(out), extract=True)
+        print_json({"ok": True, "data": run, "run_dir": saved["run_dir"], "archive": saved["archive"]})
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
 
 @app.command()
 def status(
-    run_id: str = typer.Argument(..., help="Sim run id (srun_...)"),
+    run_id: str = typer.Argument(..., help="Evaluation id (eval_...)"),
     wait: bool = typer.Option(False, "--wait", help="Poll until succeeded | failed | canceled"),
 ):
     """A run's live status, phase, episodes done, and once it has succeeded its result and `download`."""
     try:
         client = KiteClient()
-        path = f"/v1/sim_runs/{run_id}"
+        path = f"/v1/evaluations/{run_id}"
         if wait:
             print_json(wait_for_completion(client, path, terminal_states=_TERMINAL, poll_interval=_POLL_S))
         else:
@@ -148,43 +149,43 @@ def list_runs(
     limit: int = typer.Option(20, "--limit", help="Max rows (1-100)"),
     after: Optional[str] = typer.Option(None, "--after", help="Cursor: the previous page's next_cursor"),
 ):
-    """List your sim runs (newest first, last status seen; `status` has live progress)."""
+    """List your evaluations (newest first, last status seen; `status` has live progress)."""
     try:
-        print_json(KiteClient().get("/v1/sim_runs", params=_page(limit, after)))
+        print_json(KiteClient().get("/v1/evaluations", params=_page(limit, after)))
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
 
 @app.command()
-def report(run_id: str = typer.Argument(..., help="Sim run id (srun_...)")):
+def report(run_id: str = typer.Argument(..., help="Evaluation id (eval_...)")):
     """A succeeded run's scorecard: every evaluator's value, interval and count, and what was not measured."""
     try:
-        print_json(KiteClient().get(f"/v1/sim_runs/{run_id}/report"))
+        print_json(KiteClient().get(f"/v1/evaluations/{run_id}/report"))
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
 
 @app.command()
 def logs(
-    run_id: str = typer.Argument(..., help="Sim run id (srun_...)"),
+    run_id: str = typer.Argument(..., help="Evaluation id (eval_...)"),
     tail: int = typer.Option(200, "--tail", help="The last N lines"),
 ):
     """What the SDK printed, oldest first: readable while the run builds and after."""
     try:
-        print_json(KiteClient().get(f"/v1/sim_runs/{run_id}/logs", params={"tail": tail}))
+        print_json(KiteClient().get(f"/v1/evaluations/{run_id}/logs", params={"tail": tail}))
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
 
 @app.command("files")
 def list_files(
-    run_id: str = typer.Argument(..., help="Sim run id (srun_...)"),
+    run_id: str = typer.Argument(..., help="Evaluation id (eval_...)"),
     limit: Optional[int] = typer.Option(None, "--limit", help="Files per page, 1-100 (default 100)"),
     after: Optional[str] = typer.Option(None, "--after", help="Cursor: the previous page's next_cursor (a file path)"),
     all_pages: bool = typer.Option(False, "--all", help="Follow next_cursor through every page; print one list"),
 ):
     """The files of a succeeded run's directory (path, bytes, sha256), sorted by path."""
-    path = f"/v1/sim_runs/{run_id}/files"
+    path = f"/v1/evaluations/{run_id}/files"
     size = limit or 100
     try:
         client = KiteClient()
@@ -206,8 +207,8 @@ def list_files(
 
 @app.command()
 def download(
-    run_id: str = typer.Argument(..., help="Sim run id (srun_...)"),
-    out: Optional[Path] = typer.Option(None, "-o", "--out", help="Output directory (default: $KITE_SIM_RUNS_DIR, "
+    run_id: str = typer.Argument(..., help="Evaluation id (eval_...)"),
+    out: Optional[Path] = typer.Option(None, "-o", "--out", help="Output directory (default: $KITE_EVAL_RUNS_DIR, "
                                                                  "else ./runs)"),
     extract: bool = typer.Option(True, "--extract/--no-extract", help="Unpack into <out>/<run_id>/, or keep only "
                                                                       "the .tar.gz"),
@@ -215,26 +216,26 @@ def download(
     """Download a succeeded run: <out>/<run_id>.tar.gz, sha256-verified, unpacked to <out>/<run_id>/."""
     try:
         client = KiteClient(timeout=120)
-        print_json(_save(client, client.get(f"/v1/sim_runs/{run_id}"), _out_dir(out), extract=extract))
+        print_json(_save(client, client.get(f"/v1/evaluations/{run_id}"), _out_dir(out), extract=extract))
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
 
 @app.command()
-def cancel(run_id: str = typer.Argument(..., help="Sim run id (srun_...)")):
+def cancel(run_id: str = typer.Argument(..., help="Evaluation id (eval_...)")):
     """Stop a queued or running run: canceling until its job has stopped, then canceled. It pays the minutes it
     ran."""
     try:
-        print_json(KiteClient().post(f"/v1/sim_runs/{run_id}/cancel"))
+        print_json(KiteClient().post(f"/v1/evaluations/{run_id}/cancel"))
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
 
 @app.command()
-def resume(run_id: str = typer.Argument(..., help="Sim run id (srun_...)")):
+def resume(run_id: str = typer.Argument(..., help="Evaluation id (eval_...)")):
     """Run a failed or canceled run again from its request, as a new attempt: it reserves its cap again."""
     try:
-        print_json(KiteClient().post(f"/v1/sim_runs/{run_id}/resume"))
+        print_json(KiteClient().post(f"/v1/evaluations/{run_id}/resume"))
     except KiteAPIError as e:
         print_error(e.message, code=e.code, details=e.details)
 
@@ -265,7 +266,7 @@ def request(client: KiteClient, spec: Path, overrides: List[str], hf: Dict[str,
         body["kite_version"] = kite_version
     for _ in range(_ROUNDS):
         try:
-            return body, client.post("/v1/sim_runs/validate", json_data=body), sent
+            return body, client.post("/v1/evaluations/validate", json_data=body), sent
         except KiteAPIError as e:
             if e.code == "files_required" and e.details.get("files"):
                 for item in e.details["files"]:
@@ -415,16 +416,17 @@ def _page(limit: int, after: Optional[str]) -> dict:
 
 
 def _out_dir(out: Optional[Path]) -> Path:
-    """--out, else $KITE_SIM_RUNS_DIR, else ./runs; absolute, so what is printed says exactly where the run is."""
-    return Path(out or os.environ.get("KITE_SIM_RUNS_DIR") or "./runs").expanduser().resolve()
+    """--out, else $KITE_EVAL_RUNS_DIR, else ./runs; absolute, so what is printed says exactly where the run is."""
+    return Path(out or os.environ.get("KITE_EVAL_RUNS_DIR") or "./runs").expanduser().resolve()
 
 
-def _save(client: KiteClient, srun: dict, out: Path, *, extract: bool) -> dict:
+def _save(client: KiteClient, run: dict, out: Path, *, extract: bool) -> dict:
     """Fetch a succeeded run's archive to <out>/<run_id>.tar.gz, verify its sha256, and (extract) unpack the run
     directory into <out>/<run_id>/."""
-    rid, d = str(srun.get("id") or ""), srun.get("download")
-    if srun.get("status") != "succeeded" or not d:
-        raise KiteAPIError(409, f"Sim run '{rid}' is {srun.get('status')}, not succeeded.", code="sim_run_not_ready")
+    rid, d = str(run.get("id") or ""), run.get("download")
+    if run.get("status") != "succeeded" or not d:
+        raise KiteAPIError(409, f"Evaluation '{rid}' is {run.get('status')}, not succeeded.",
+                           code="evaluation_not_ready")
     archive = verified(client, d, out, rid, storage=_storage_client)
     saved = {"id": rid, "archive": str(archive), "bytes": archive.stat().st_size, "sha256": str(d["sha256"]).lower()}
     if extract:

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/src/kite_cli/cli/main.py

@@ -5,7 +5,7 @@ Usage:
   kite auth login
   kite rl plan open_duck_mini_v2 "walk forward at a steady pace" -o walk.json
   kite augment create --repo-id lerobot/pusht -i "white marble table" --wait
-  kite sim run specs/smoke.yaml --wait --out ./runs
+  kite eval run specs/smoke.yaml --wait --out ./runs
   kite doctor
 """
 
@@ -16,7 +16,7 @@ from typing import Optional
 import typer
 
 from kite_cli import __version__
-from kite_cli.cli.commands import augment, auth_cmd, config_cmd, rl, sim, twin
+from kite_cli.cli.commands import augment, auth_cmd, config_cmd, evaluations, rl, twin
 
 app = typer.Typer(
     name="kite",
@@ -31,8 +31,8 @@ app.add_typer(config_cmd.app, name="config", help="CLI configuration")
 app.add_typer(augment.app, name="augment", help="Dataset augmentation: relighting + video augmentation of LeRobot datasets")
 app.add_typer(twin.app, name="twin", help="Digital twins (beta): one robot episode -> an interactable MuJoCo scene")
 app.add_typer(rl.app, name="rl", help="RL training in simulation: a task spec -> a trained, verified policy")
-app.add_typer(sim.app, name="sim", help="Sim runs (beta): a Kite SDK spec on Kite's machines -> a scorecard and a "
-                                        "verifiable run directory")
+app.add_typer(evaluations.app, name="eval", help="Evaluations (beta): a Kite SDK spec on Kite's machines -> a "
+                                                "scorecard and a verifiable run directory")
 
 # Kite's own install adds the dashboard's commands (projects, training, deployment, sensors); the published package
 # has only the /v1 commands above.

{kiteml_cli-0.3.1 → kiteml_cli-0.4.0}/src/kite_cli/mcp/server.py

@@ -86,8 +86,8 @@ def _twin_view(twin: dict, client: KiteClient) -> dict:
     return out
 
 
-def _sim_view(run: dict, client: KiteClient) -> dict:
-    """A sim run as an agent should see it: the resource plus `next`, the one thing to do now. A hosted MCP server
+def _eval_view(run: dict, client: KiteClient) -> dict:
+    """An evaluation as an agent should see it: the resource plus `next`, the one thing to do now. A hosted MCP server
     cannot write to the agent's disk, so a finished run's `next` is the exact command that fetches and extracts its
     run directory."""
     out = dict(run)
@@ -101,26 +101,26 @@ def _sim_view(run: dict, client: KiteClient) -> dict:
         else:
             fetch = f'curl -fL "{d["url"]}" -o {fname}'
         out["next"] = (f"{fetch} && tar xzf {fname}   # -> {rid}/, the run directory (`kite verify {rid} --evals <the spec's evaluators>` checks it)"
-                       + (f" (link valid until {d['expires_at']}; call kite_sim_status again for a fresh one)"
-                          if d.get("expires_at") else "") + f"; kite_sim_report {rid} has the scorecard")
+                       + (f" (link valid until {d['expires_at']}; call kite_eval_status again for a fresh one)"
+                          if d.get("expires_at") else "") + f"; kite_eval_report {rid} has the scorecard")
     elif status in ("queued", "processing"):
         eps = run.get("episodes") or {}
         where = (f" (phase {run['phase']}" + (f", {eps['done']}/{eps['total']} episodes" if eps.get("total") else "")
                  + ")") if run.get("phase") else ""
-        out["next"] = f"call kite_sim_status again in a minute{where}"
+        out["next"] = f"call kite_eval_status again in a minute{where}"
     elif status == "canceling":
-        out["next"] = ("call kite_sim_status again in a few seconds: the cancel was accepted and the job is "
+        out["next"] = ("call kite_eval_status again in a few seconds: the cancel was accepted and the job is "
                        "stopping; it then reads canceled")
     elif status in ("failed", "canceled"):
-        out["next"] = (f"kite_sim_resume {rid} to run it again" if (run.get("error") or {}).get("resumable", True)
+        out["next"] = (f"kite_eval_resume {rid} to run it again" if (run.get("error") or {}).get("resumable", True)
                        else "fix the spec as the error says, and run it again")
     return out
 
 
-def _sim_request(args: dict, client: KiteClient) -> tuple[dict, dict | None]:
-    """The request body a kite_sim_* tool sends, and the validation that accepted it when it was worked out here:
+def _eval_request(args: dict, client: KiteClient) -> tuple[dict, dict | None]:
+    """The request body a kite_eval_* tool sends, and the validation that accepted it when it was worked out here:
     from `spec_yaml` and `files` as given, or (the local server only) from `spec`, a spec file on this machine,
-    whose files are sent as `kite sim run` sends them."""
+    whose files are sent as `kite eval run` sends them."""
     if args.get("spec_yaml"):
         keys = ("spec_yaml", "spec_path", "files", "overrides", "kite_version")
         return {k: args[k] for k in keys if args.get(k)}, None
@@ -131,7 +131,7 @@ def _sim_request(args: dict, client: KiteClient) -> tuple[dict, dict | None]:
         raise ValueError("`spec` only works with the local (stdio) Kite MCP server; send `spec_yaml` and `files`")
     from pathlib import Path
 
-    from kite_cli.cli.commands.sim import hf_refs, request
+    from kite_cli.cli.commands.evaluations import hf_refs, request
 
     hf = hf_refs([f"{k}={v}" for k, v in (args.get("hf") or {}).items()])
     body, validation, _ = request(client, Path(args["spec"]), args.get("overrides") or [], hf, args.get("kite_version"))
@@ -190,7 +190,7 @@ _RL_SPEC = {
 }
 _RL_RUN_ID = {"run_id": {"type": "string", "description": "RL run id (rlr_...)"}}
 
-_SIM_SPEC = {
+_EVAL_SPEC = {
     "spec": {"type": "string", "description": "A Kite SDK spec file on this machine, best as an absolute path (the local server only): every path it names is sent from this disk, a file Kite ships by its hash alone"},
     "spec_yaml": {"type": "string", "description": "The spec's text, instead of `spec` (the hosted server): send `spec_path` and `files` with it"},
     "spec_path": {"type": "string", "description": "With spec_yaml: its path in your workspace, e.g. specs/smoke.yaml. Paths in the spec are read relative to it"},
@@ -199,7 +199,7 @@ _SIM_SPEC = {
     "hf": {"type": "object", "description": "With spec: {PATH: \"OWNER/NAME@COMMIT:FILE\"}, the paths that are a public Hugging Face file (a checkpoint's weights). Optional"},
     "kite_version": {"type": "string", "description": "The Kite release you expect; refused if it is not the one served. Optional"},
 }
-_SIM_RUN_ID = {"run_id": {"type": "string", "description": "Sim run id (srun_...)"}}
+_EVAL_ID = {"run_id": {"type": "string", "description": "Evaluation id (eval_...)"}}
 
 # ---------------------------------------------------------------------------
 # Tool definitions — every tool has title + annotations per Claude directory
@@ -208,15 +208,26 @@ _SIM_RUN_ID = {"run_id": {"type": "string", "description": "Sim run id (srun_...
 TOOLS = [
     # Augmentations (public /v1 API — relighting + video augmentation)
     Tool(name="kite_augment_create", title="Create Augmentation",
-         description="Create a dataset augmentation run from a HuggingFace LeRobot dataset. model 'auto' (default) lets a planner pick 'relight' (cheap, exact colour/exposure/warmth transform of every episode — e.g. match a darker, warmer deployment scene) or 'video' (generative video augmentation of content: textures, objects, backgrounds). Pass reference_images of the target look. Returns the augmentation resource with its id (aug_...) and, once planned, the chosen plan.",
+         description="Create a dataset augmentation run from a HuggingFace LeRobot dataset. model 'auto' (default) lets a planner pick per camera: 'relight' (cheap, exact colour/exposure/warmth transform of every episode — e.g. match a darker, warmer deployment scene), 'background' (a new place behind the work cell from a panorama; the robot, objects and table keep their recorded pixels), 'world' (a 3D world rendered from each camera's pose, for moving cameras) or 'video' (generative video augmentation of content: textures, objects). background and world run on whole datasets and may not be enabled for every account. Pass reference_images of the target look. Returns the augmentation resource with its id (aug_...) and, once planned, the chosen plan.",
          inputSchema={"type": "object", "properties": {
              "repo_id": {"type": "string", "description": "Source HF dataset id, e.g. lerobot/pusht"},
              "instructions": {"type": "string", "description": "Augmentation instruction"},
-             "episode_count": {"type": "integer", "description": "Episodes to generate (1-50, default 3)"},
+             "episode_count": {"type": "integer", "description": "Video: episodes to generate (1-50, default 3)"},
+             "episodes": {"anyOf": [{"type": "integer", "minimum": 1}, {"type": "string", "enum": ["all"]}],
+                          "description": "model background/world: source episodes to augment, 'all' (their default) or the first N. Any other model takes a count as episode_count; 'all' there sends none (relight covers every episode)"},
              "output_type": {"type": "string", "enum": ["download", "huggingface"], "description": "Where results go (default download)"},
              "hf_repo": {"type": "string", "description": "Target HF repo id (for output_type huggingface)"},
              "variation_prompts": {"type": "array", "items": {"type": "string"}, "description": "Per-episode prompts (overrides episode_count)"},
-             "model": {"type": "string", "enum": ["auto", "relight", "video"], "description": "Engine (default auto)"},
+             "model": {"type": "string", "enum": ["auto", "relight", "background", "world", "video"], "description": "Engine (default auto)"},
+             "environments": {"type": "object", "description": "background/world: what to put behind the work cell (default: 3 matched to the rig). One of ids, prompt, or filter/count", "properties": {
+                 "ids": {"type": "array", "items": {"type": "string"}, "description": "Environment ids (aenv_...) or library ids"},
+                 "filter": {"type": "object", "properties": {"category": {"type": "string"}, "lighting": {"type": "string", "enum": ["daylight", "artificial"]}}},
+                 "prompt": {"type": "string", "description": "Make new environments from a description (minutes)"},
+                 "count": {"type": "integer", "description": "How many to pick or make"}}},
+             "must_keep": {"type": "array", "items": {"type": "string"}, "description": "Things that must keep their recorded pixels, besides the robot and what it handles"},
+             "variants": {"type": "integer", "description": "background/world: environments per source episode (default 1)"},
+             "preview_first": {"type": "boolean", "description": "background/world: check one episode's quality gates (windows at its start, middle and end) before charging for the rest (default true)"},
+             "allow_video_fallback": {"type": "boolean", "description": "auto: where no scene engine can run, use generative video (far more tokens, no pixel guarantee) instead of failing with the reasons and the video price (default false)"},
              "reference_images": {"type": "array", "maxItems": 6, "description": "Photos of the target look", "items": {
                  "type": "object", "properties": {
                      "camera": {"type": "string", "description": "Camera it shows (e.g. 'top'); omit for all cameras"},
@@ -276,42 +287,42 @@ TOOLS = [
              "twin_id": {"type": "string", "description": "Twin id (twin_...)"}},
           "required": ["twin_id"]}, annotations=_WRITE),
 
-    # Sim runs (public /v1 API): a Kite SDK spec on Kite's machines -> a scorecard and a verifiable run directory
-    Tool(name="kite_sim_validate", title="Validate Sim Run",
+    # Evaluations (public /v1 API): a Kite SDK spec on Kite's machines -> a scorecard and a verifiable run directory
+    Tool(name="kite_eval_validate", title="Validate Evaluation",
          description="Free check of a Kite SDK spec (kite_spec_version \"2\") before it runs on Kite's machines, in seconds: the spec parses and plans (episodes, each evaluator, the estimate), every file it names is accounted for, and the price (price.reserved tokens at price.tokens_per_hour: an NVIDIA L4 for a cuda policy, a CPU machine otherwise). Creates nothing and charges nothing. Errors keep the SDK's own code, field, fix and line (spec_invalid, files_required listing each missing path, code_not_allowed for Python Kite does not ship).",
-         inputSchema={"type": "object", "properties": _SIM_SPEC}, annotations=_READ),
-    Tool(name="kite_sim_run", title="Start Sim Run",
-         description="Run a Kite SDK spec on Kite's machines: the policy in the spec's world for its episodes, every episode traced and every evaluator scored. Returns the queued run (id srun_...) with a `next` instruction; follow it with kite_sim_status. The run's cap (max_minutes, default 60 on CPU and 120 on an L4) is reserved now at its tier's rate, and the run is charged the minutes its job ran; a failed run costs nothing (insufficient_tokens when the balance is short). A finished run has a scorecard (kite_sim_report), its manifest hash (the run's identity, shared by a local `kite run` of the same spec) and the run directory as one .tar.gz.",
+         inputSchema={"type": "object", "properties": _EVAL_SPEC}, annotations=_READ),
+    Tool(name="kite_eval_run", title="Start Evaluation",
+         description="Run a Kite SDK spec on Kite's machines: the policy in the spec's world for its episodes, every episode traced and every evaluator scored. Returns the queued run (id eval_...) with a `next` instruction; follow it with kite_eval_status. The run's cap (max_minutes, default 60 on CPU and 120 on an L4) is reserved now at its tier's rate, and the run is charged the minutes its job ran; a failed run costs nothing (insufficient_tokens when the balance is short). A finished run has a scorecard (kite_eval_report), its manifest hash (the run's identity, shared by a local `kite run` of the same spec) and the run directory as one .tar.gz.",
          inputSchema={"type": "object", "properties": {
-             **_SIM_SPEC,
+             **_EVAL_SPEC,
              "max_minutes": {"type": "integer", "description": "The run's cap, 1-720: what it reserves and the most it can cost. Optional"},
              "label": {"type": "string", "description": "A short name for your own listing. Optional"},
              "no_eval": {"type": "boolean", "description": "Run the episodes only: no evaluators, no scorecard. Optional"},
              "verify_determinism": {"type": "boolean", "description": "Re-run episode 0 and fail the run on a differing hash. Optional"},
              "idempotency_key": {"type": "string", "description": "Reuse the same key when retrying, so a retry never runs twice. Optional"}}},
          annotations=_WRITE),
-    Tool(name="kite_sim_status", title="Get Sim Run Status",
-         description="Get a sim run's status (queued/processing/canceling/succeeded/failed/canceled), its phase (materializing, planning, fetching, running, verifying, packaging), the episodes done, the tokens reserved and refunded, and a `next` instruction. queued includes waiting for a machine: an L4 can take minutes to find. When succeeded it carries the result (manifest_hash, run_hash, verified, each evaluator's metric with its interval) and `next` is the exact command that fetches and extracts the run directory. A failed run carries the SDK's error (code, param, details.kite_code, details.remediation).",
-         inputSchema={"type": "object", "properties": _SIM_RUN_ID, "required": ["run_id"]}, annotations=_READ),
-    Tool(name="kite_sim_list", title="List Sim Runs",
-         description="List your sim runs, newest first, with their last known status (kite_sim_status has live progress).",
+    Tool(name="kite_eval_status", title="Get Evaluation Status",
+         description="Get an evaluation's status (queued/processing/canceling/succeeded/failed/canceled), its phase (materializing, planning, fetching, running, verifying, packaging), the episodes done, the tokens reserved and refunded, and a `next` instruction. queued includes waiting for a machine: an L4 can take minutes to find. When succeeded it carries the result (manifest_hash, run_hash, verified, each evaluator's metric with its interval) and `next` is the exact command that fetches and extracts the run directory. A failed run carries the SDK's error (code, param, details.kite_code, details.remediation).",
+         inputSchema={"type": "object", "properties": _EVAL_ID, "required": ["run_id"]}, annotations=_READ),
+    Tool(name="kite_eval_list", title="List Evaluations",
+         description="List your evaluations, newest first, with their last known status (kite_eval_status has live progress).",
          inputSchema={"type": "object", "properties": {
              "limit": {"type": "integer", "description": "Max rows (1-100, default 20)"}}}, annotations=_READ),
-    Tool(name="kite_sim_report", title="Get Sim Run Scorecard",
-         description="Get a succeeded sim run's scorecard, the SDK's scorecard.json: every evaluator's value, confidence interval and count, the calibration it was scored under, and what was not measured.",
-         inputSchema={"type": "object", "properties": _SIM_RUN_ID, "required": ["run_id"]}, annotations=_READ),
-    Tool(name="kite_sim_logs", title="Get Sim Run Logs",
-         description="Get the last lines the SDK printed for a sim run, oldest first; readable while it runs and after. Use it to see why a run failed.",
+    Tool(name="kite_eval_report", title="Get Evaluation Scorecard",
+         description="Get a succeeded evaluation's scorecard, the SDK's scorecard.json: every evaluator's value, confidence interval and count, the calibration it was scored under, and what was not measured.",
+         inputSchema={"type": "object", "properties": _EVAL_ID, "required": ["run_id"]}, annotations=_READ),
+    Tool(name="kite_eval_logs", title="Get Evaluation Logs",
+         description="Get the last lines the SDK printed for an evaluation, oldest first; readable while it runs and after. Use it to see why a run failed.",
          inputSchema={"type": "object", "properties": {
-             **_SIM_RUN_ID,
+             **_EVAL_ID,
              "tail": {"type": "integer", "description": "How many of the last lines (default 200)"}},
           "required": ["run_id"]}, annotations=_READ),
-    Tool(name="kite_sim_cancel", title="Cancel Sim Run",
-         description="Stop a queued or running sim run. It answers canceling, and reads canceled once its job has stopped (check with kite_sim_status). It is charged the minutes it ran.",
-         inputSchema={"type": "object", "properties": _SIM_RUN_ID, "required": ["run_id"]}, annotations=_DESTRUCTIVE),
-    Tool(name="kite_sim_resume", title="Resume Sim Run",
-         description="Run a failed or canceled sim run again from its request, as a new attempt: it reserves its cap again and is queued.",
-         inputSchema={"type": "object", "properties": _SIM_RUN_ID, "required": ["run_id"]}, annotations=_WRITE),
+    Tool(name="kite_eval_cancel", title="Cancel Evaluation",
+         description="Stop a queued or running evaluation. It answers canceling, and reads canceled once its job has stopped (check with kite_eval_status). It is charged the minutes it ran.",
+         inputSchema={"type": "object", "properties": _EVAL_ID, "required": ["run_id"]}, annotations=_DESTRUCTIVE),
+    Tool(name="kite_eval_resume", title="Resume Evaluation",
+         description="Run a failed or canceled evaluation again from its request, as a new attempt: it reserves its cap again and is queued.",
+         inputSchema={"type": "object", "properties": _EVAL_ID, "required": ["run_id"]}, annotations=_WRITE),
 
     # RL runs (public /v1 API): train a robot policy in simulation from a spec, one run per seed
     Tool(name="kite_rl_catalog", title="RL Training Catalog",
@@ -402,11 +413,15 @@ async def _dispatch(name: str, args: dict, client: KiteClient) -> list[TextConte
 
     # --- Augmentations (public /v1 API) ---
     if name == "kite_augment_create":
+        from kite_cli.cli.commands.augment import scene_config
+
         body = {
             "source": {"type": "huggingface", "repo_id": args["repo_id"]},
             "instructions": args["instructions"],
-            "config": {"episode_count": args.get("episode_count", 3)},
+            "config": scene_config(args.get("episodes", args.get("episode_count")), args.get("model"),
+                                   args.get("variants"), args.get("preview_first"), args.get("allow_video_fallback")),
             "output": {"type": args.get("output_type", "download")},
+            **{k: args[k] for k in ("environments", "must_keep") if args.get(k)},
         }
         if args.get("hf_repo"):
             body["output"]["repo_id"] = args["hf_repo"]
@@ -453,32 +468,32 @@ async def _dispatch(name: str, args: dict, client: KiteClient) -> list[TextConte
     if name == "kite_twin_resume":
         return _ok(_twin_view(client.post(f"/v1/twins/{args['twin_id']}/resume"), client))
 
-    # --- Sim runs (public /v1 API) ---
-    if name in ("kite_sim_validate", "kite_sim_run"):
+    # --- Evaluations (public /v1 API) ---
+    if name in ("kite_eval_validate", "kite_eval_run"):
         try:
-            body, validation = _sim_request(args, client)
+            body, validation = _eval_request(args, client)
         except ValueError as e:
             return _err(str(e))
-    if name == "kite_sim_validate":
-        return _ok(validation or client.post("/v1/sim_runs/validate", json_data=body))
-    if name == "kite_sim_run":
+    if name == "kite_eval_validate":
+        return _ok(validation or client.post("/v1/evaluations/validate", json_data=body))
+    if name == "kite_eval_run":
         options = {k: args[k] for k in ("max_minutes", "no_eval", "verify_determinism") if args.get(k)}
         body = {**body, **({"options": options} if options else {}),
                 **({"label": args["label"]} if args.get("label") else {})}
         headers = {"Idempotency-Key": args["idempotency_key"]} if args.get("idempotency_key") else None
-        return _ok(_sim_view(client.post("/v1/sim_runs", json_data=body, extra_headers=headers), client))
-    if name == "kite_sim_status":
-        return _ok(_sim_view(client.get(f"/v1/sim_runs/{args['run_id']}"), client))
-    if name == "kite_sim_list":
-        return _ok(client.get("/v1/sim_runs", params={"limit": args.get("limit", 20)}))
-    if name == "kite_sim_report":
-        return _ok(client.get(f"/v1/sim_runs/{args['run_id']}/report"))
-    if name == "kite_sim_logs":
-        return _ok(client.get(f"/v1/sim_runs/{args['run_id']}/logs", params={"tail": args.get("tail", 200)}))
-    if name == "kite_sim_cancel":
-        return _ok(_sim_view(client.post(f"/v1/sim_runs/{args['run_id']}/cancel"), client))
-    if name == "kite_sim_resume":
-        return _ok(_sim_view(client.post(f"/v1/sim_runs/{args['run_id']}/resume"), client))
+        return _ok(_eval_view(client.post("/v1/evaluations", json_data=body, extra_headers=headers), client))
+    if name == "kite_eval_status":
+        return _ok(_eval_view(client.get(f"/v1/evaluations/{args['run_id']}"), client))
+    if name == "kite_eval_list":
+        return _ok(client.get("/v1/evaluations", params={"limit": args.get("limit", 20)}))
+    if name == "kite_eval_report":
+        return _ok(client.get(f"/v1/evaluations/{args['run_id']}/report"))
+    if name == "kite_eval_logs":
+        return _ok(client.get(f"/v1/evaluations/{args['run_id']}/logs", params={"tail": args.get("tail", 200)}))
+    if name == "kite_eval_cancel":
+        return _ok(_eval_view(client.post(f"/v1/evaluations/{args['run_id']}/cancel"), client))
+    if name == "kite_eval_resume":
+        return _ok(_eval_view(client.post(f"/v1/evaluations/{args['run_id']}/resume"), client))
 
     # --- RL runs (public /v1 API) ---
     if name == "kite_rl_catalog":