euler-train 2.8.0__tar.gz → 2.9.0__tar.gz

{euler_train-2.8.0 → euler_train-2.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-train
-Version: 2.8.0
+Version: 2.9.0
 Summary: Lightweight file-based experiment logger for PyTorch
 Requires-Python: >=3.9
 Requires-Dist: numpy

{euler_train-2.8.0 → euler_train-2.9.0}/README.md

@@ -64,9 +64,9 @@ run = euler_train.init(
     config=cfg,
     stream={
         "base_url": os.environ["EULER_VIEW_BASE_URL"],
-        "model_id": 42,
         "api_token": os.environ["EULER_VIEW_API_TOKEN"],
         "stream_attach_token": os.environ.get("EULER_VIEW_STREAM_ATTACH_TOKEN"),
+        "model_id": 42,  # only needed for the SLURM fallback path
     },
 )
 ```
@@ -74,7 +74,7 @@ run = euler_train.init(
 What each field is for:
 
 - `base_url`: Euler View server base URL, for example `https://view.example.com`.
-- `model_id`: Euler View model that should receive this run.
+- `model_id`: Euler View model that should receive this run. This is optional when `stream_attach_token` is provided, and required for the SLURM fallback path.
 - `api_token` or `access_token`: credential used to open a stream session. This is required whenever the package needs to negotiate its own short-lived ingest token, even if `stream_attach_token` is also provided.
 - `stream_attach_token`: optional opaque launch-attachment token. When present, the producer sends it during the session handshake and Euler View attaches the stream to that launch directly. This is the preferred path for interactive or otherwise non-SLURM launches. Internally Euler View currently maps this token to `model_launches.id`, but the package treats it as an opaque token.
 - `stream_token`: optional pre-issued ingest token. If you pass this, the package skips the session handshake entirely and posts events directly to the ingest endpoint.
@@ -85,9 +85,9 @@ What each field is for:
 How the handshake works:
 
 1. `euler_train` creates the local run directory and writes `meta.json`, `config.json`, and the other standard files as usual.
-2. When the first stream flush happens, the package requests a session from Euler View at `/api/models/{model_id}/runs/{run_id}/stream/session`.
+2. When the first stream flush happens, the package requests a session from Euler View at `POST /api/model-run-stream/session`.
 3. If `stream_attach_token` is configured, it is sent as `streamAttachToken` and used as the explicit launch attachment key.
-4. If no `stream_attach_token` is configured, the package falls back to the SLURM metadata already captured in `meta.json["slurm"]["job_id"]`.
+4. If no `stream_attach_token` is configured, the package falls back to the SLURM metadata already captured in `meta.json["slurm"]["job_id"]`. In that fallback mode, `model_id` must be configured so Euler View knows which model namespace to match against.
 5. Euler View returns a short-lived ingest token plus the canonical `streamAttachToken`; the package caches that returned token and reuses it on later session refreshes.
 6. Subsequent events are sent to `/api/model-run-stream/ingest`.
 
@@ -100,27 +100,25 @@ If you need to verify from a cluster node whether Euler View is reachable and wh
 ```bash
 euler-train.stream.check \
   --api-url https://view.example.com \
-  --model-id 42 \
   --api-key "$EULER_VIEW_API_TOKEN" \
   --stream-attach-token "$EULER_VIEW_STREAM_ATTACH_TOKEN"
 ```
 
-This command calls:
-
-- `POST /api/models/{model_id}/runs/{run_id}/stream/check`
+This command calls `POST /api/model-run-stream/check`.
 
 It validates:
 
 - network reachability to Euler View
 - API token validity and scope
-- model existence
 - stream attachment resolution via `stream_attach_token`, when provided
+- model existence, when the request uses the `model_id` + `--slurm-job-id` fallback path
 - SLURM fallback matching via `--slurm-job-id`, when no attach token is provided
 
 Useful flags:
 
 - `--run-id`: override the dry-run run ID. If omitted, the CLI generates a temporary `stream-check-*` ID.
 - `--stream-attach-token`: test the explicit launch attachment path.
+- `--model-id`: required for the fallback SLURM matching path, optional when `--stream-attach-token` is used.
 - `--slurm-job-id`: test the fallback SLURM matching path.
 - `--datasource-id`: include the datasource hint that a real stream session would send.
 - `--euler-train-dir` and `--run-dir`: include path hints in the dry-run payload.
@@ -131,8 +129,8 @@ The dry-run check does not start a run and does not mint an ingest token. It onl
 What has to be configured where:
 
 - In the training script or job wrapper: pass `stream={...}` to `euler_train.init(...)`.
-- In Euler View / the launcher: if the run should attach to a specific launch regardless of SLURM availability, provide a `stream_attach_token` to the runtime and forward it into the `stream` config.
-- Under plain SLURM launches: no extra attachment config is required as long as Euler View can uniquely match the run by `slurm.job_id`.
+- In Euler View / the launcher: if the run should attach to a specific launch regardless of SLURM availability, provide a `stream_attach_token` to the runtime and forward it into the `stream` config. In that mode the producer does not need to know `model_id`.
+- Under plain SLURM launches: no extra attachment config is required as long as Euler View can uniquely match the run by `slurm.job_id`, but `model_id` still has to be configured for the fallback lookup.
 - If neither `stream_attach_token` nor a unique SLURM match is available, run streaming still works, but launch-to-run linking in Euler View may remain unresolved.
 
 For new integrations, prefer `stream_attach_token`. The older `launch_id` / `launchId` config keys are still accepted as compatibility aliases, but new code should treat the attachment value as an opaque stream token rather than a launch ID.

{euler_train-2.8.0 → euler_train-2.9.0}/euler_train/stream.py

@@ -11,7 +11,6 @@ from datetime import datetime
 from pathlib import Path
 from typing import Any, Protocol
 from urllib.error import HTTPError, URLError
-from urllib.parse import quote
 from urllib.request import Request, urlopen
 
 from .serialization import _json_default
@@ -146,9 +145,9 @@ class EulerViewStreamConfig:
         )
 
         if self.stream_token is None:
-            if self.model_id is None:
+            if self.model_id is None and self.stream_attach_token is None:
                 raise ValueError(
-                    "stream.model_id is required when stream.stream_token is not provided",
+                    "stream.model_id is required when neither stream.stream_token nor stream.stream_attach_token is provided",
                 )
             if self.api_token is None and self.access_token is None:
                 raise ValueError(
@@ -312,11 +311,10 @@ class EulerViewStreamConsumer:
         if not force and now < self._next_retry_at:
             return False
 
-        session_url = (
-            f"{self.config.base_url}/api/models/{self.config.model_id}"
-            f"/runs/{quote(self._context.run_id, safe='')}/stream/session"
-        )
+        session_url = f"{self.config.base_url}/api/model-run-stream/session"
         session_body = _build_stream_session_body(
+            run_id=self._context.run_id,
+            model_id=self.config.model_id,
             run_dir=self.config.run_dir or str(self._context.run_dir),
             euler_train_dir=self.config.euler_train_dir or str(self._context.runs_dir),
             stream_attach_token=(
@@ -447,9 +445,6 @@ def check_stream_handshake(
             "stream.stream_token bypasses the session handshake; a dry-run handshake requires api_token or access_token instead",
         )
 
-    if resolved_config.model_id is None:
-        raise ValueError("stream.model_id is required for handshake checks")
-
     normalized_run_id = (
         _normalize_optional_text(run_id, field_name="run_id")
         if run_id is not None
@@ -473,6 +468,13 @@ def check_stream_handshake(
         raise ValueError(
             "stream_attach_token argument does not match stream.stream_attach_token",
         )
+    resolved_stream_attach_token = (
+        normalized_stream_attach_token or resolved_config.stream_attach_token
+    )
+    if resolved_config.model_id is None and resolved_stream_attach_token is None:
+        raise ValueError(
+            "stream.model_id is required for handshake checks when stream_attach_token is not provided",
+        )
 
     resolved_datasource_id = (
         _require_positive_int(
@@ -506,16 +508,13 @@ def check_stream_handshake(
         else resolved_config.run_dir
     )
 
-    url = (
-        f"{resolved_config.base_url}/api/models/{resolved_config.model_id}"
-        f"/runs/{quote(normalized_run_id, safe='')}/stream/check"
-    )
+    url = f"{resolved_config.base_url}/api/model-run-stream/check"
     payload = _build_stream_session_body(
+        run_id=normalized_run_id,
+        model_id=resolved_config.model_id,
         run_dir=resolved_run_dir,
         euler_train_dir=resolved_euler_train_dir,
-        stream_attach_token=(
-            normalized_stream_attach_token or resolved_config.stream_attach_token
-        ),
+        stream_attach_token=resolved_stream_attach_token,
         slurm_job_id=resolved_slurm_job_id,
         datasource_id=resolved_datasource_id,
         session_expires_in_sec=resolved_config.session_expires_in_sec,
@@ -633,6 +632,8 @@ def _extract_session_stream_attach_token(payload: Mapping[str, Any]) -> str | No
 
 def _build_stream_session_body(
     *,
+    run_id: str,
+    model_id: int | None,
     run_dir: str | None,
     euler_train_dir: str | None,
     stream_attach_token: str | None,
@@ -640,7 +641,9 @@ def _build_stream_session_body(
     datasource_id: int | None,
     session_expires_in_sec: int | None,
 ) -> dict[str, Any]:
-    body: dict[str, Any] = {}
+    body: dict[str, Any] = {"runId": run_id}
+    if model_id is not None:
+        body["modelId"] = model_id
     if run_dir is not None:
         body["runDir"] = run_dir
     if euler_train_dir is not None:

{euler_train-2.8.0 → euler_train-2.9.0}/euler_train/stream_check_cli.py

@@ -14,7 +14,11 @@ def build_parser() -> argparse.ArgumentParser:
         description="Dry-run the Euler View stream handshake",
     )
     parser.add_argument("--api-url", required=True, help="Euler View base URL")
-    parser.add_argument("--model-id", required=True, type=int, help="Euler View model ID")
+    parser.add_argument(
+        "--model-id",
+        type=int,
+        help="Euler View model ID. Required when using the SLURM fallback path instead of --stream-attach-token.",
+    )
     parser.add_argument(
         "--api-key",
         required=True,
@@ -64,6 +68,9 @@ def main(argv: Sequence[str] | None = None) -> int:
     parser = build_parser()
     args = parser.parse_args(argv)
 
+    if args.model_id is None and not args.stream_attach_token:
+        parser.error("--model-id is required unless --stream-attach-token is provided")
+
     try:
         result = check_stream_handshake(
             {

{euler_train-2.8.0 → euler_train-2.9.0}/euler_train.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: euler-train
-Version: 2.8.0
+Version: 2.9.0
 Summary: Lightweight file-based experiment logger for PyTorch
 Requires-Python: >=3.9
 Requires-Dist: numpy

{euler_train-2.8.0 → euler_train-2.9.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "euler-train"
-version = "2.8.0"
+version = "2.9.0"
 description = "Lightweight file-based experiment logger for PyTorch"
 requires-python = ">=3.9"
 dependencies = ["numpy", "euler-dataset-contract"]

{euler_train-2.8.0 → euler_train-2.9.0}/tests/test_runlog.py

@@ -1963,7 +1963,7 @@ class TestStreaming:
                     "timeout": timeout,
                 },
             )
-            if request.full_url.endswith("/stream/session"):
+            if request.full_url == "https://sync.example/api/model-run-stream/session":
                 return _FakeHttpResponse(
                     {
                         "token": "stream-token",
@@ -1997,12 +1997,12 @@ class TestStreaming:
         run.log({"loss": 0.25}, step=3, epoch=1)
         run.finish()
 
-        assert calls[0]["url"] == (
-            f"https://sync.example/api/models/42/runs/{run.run_id}/stream/session"
-        )
+        assert calls[0]["url"] == "https://sync.example/api/model-run-stream/session"
         assert calls[0]["method"] == "POST"
         assert calls[0]["headers"]["authorization"] == "Bearer user-token"
         assert calls[0]["body"] == {
+            "runId": run.run_id,
+            "modelId": 42,
             "runDir": str(run.dir),
             "eulerTrainDir": str(run.dir.parent),
             "datasourceId": 7,
@@ -2028,6 +2028,73 @@ class TestStreaming:
         finish_event = ingest_calls[2]["body"]["events"][0]
         assert finish_event["status"] == "completed"
 
+    def test_euler_view_http_consumer_can_handshake_without_model_id_when_attach_token_is_present(
+        self,
+        tmp_path,
+        monkeypatch,
+    ):
+        calls: list[dict] = []
+
+        def fake_urlopen(request, timeout=0):
+            headers = {k.lower(): v for k, v in request.header_items()}
+            body = json.loads(request.data.decode("utf-8"))
+            calls.append(
+                {
+                    "url": request.full_url,
+                    "method": request.get_method(),
+                    "headers": headers,
+                    "body": body,
+                    "timeout": timeout,
+                },
+            )
+            if request.full_url.endswith("/api/model-run-stream/session"):
+                return _FakeHttpResponse(
+                    {
+                        "token": "stream-token",
+                        "expiresAt": "2099-01-01T00:00:00+00:00",
+                        "ingestUrl": "https://sync.example/api/model-run-stream/ingest",
+                        "run": {
+                            "modelId": 42,
+                            "runId": "placeholder",
+                            "streamAttachToken": "attach-123",
+                        },
+                    },
+                )
+            if request.full_url.endswith("/api/model-run-stream/ingest"):
+                return _FakeHttpResponse(
+                    {
+                        "success": True,
+                        "events": [],
+                        "latestCursor": len(calls),
+                    },
+                )
+            raise AssertionError(f"Unexpected URL {request.full_url}")
+
+        monkeypatch.setattr(stream_mod, "urlopen", fake_urlopen)
+
+        run = euler_train.init(
+            dir=str(tmp_path / "r"),
+            config={"lr": 1e-3},
+            stream={
+                "base_url": "https://sync.example",
+                "api_token": "user-token",
+                "stream_attach_token": "attach-123",
+                "batch_size": 1,
+            },
+        )
+        run.log({"loss": 0.25}, step=3, epoch=1)
+        run.finish()
+
+        assert calls[0]["url"] == "https://sync.example/api/model-run-stream/session"
+        assert calls[0]["method"] == "POST"
+        assert calls[0]["headers"]["authorization"] == "Bearer user-token"
+        assert calls[0]["body"] == {
+            "runId": run.run_id,
+            "runDir": str(run.dir),
+            "eulerTrainDir": str(run.dir.parent),
+            "streamAttachToken": "attach-123",
+        }
+
     def test_stream_network_failures_do_not_break_training(
         self,
         tmp_path,
@@ -2072,7 +2139,7 @@ class TestStreaming:
                     "timeout": timeout,
                 },
             )
-            if request.full_url.endswith("/stream/check"):
+            if request.full_url == "https://sync.example/api/model-run-stream/check":
                 return _FakeHttpResponse(
                     {
                         "success": True,
@@ -2096,7 +2163,6 @@ class TestStreaming:
         payload = stream_mod.check_stream_handshake(
             {
                 "base_url": "https://sync.example",
-                "model_id": 42,
                 "api_token": "user-token",
                 "stream_attach_token": "attach-123",
                 "datasource_id": 7,
@@ -2107,7 +2173,7 @@ class TestStreaming:
         assert payload["success"] is True
         assert calls == [
             {
-                "url": "https://sync.example/api/models/42/runs/dry-run-1/stream/check",
+                "url": "https://sync.example/api/model-run-stream/check",
                 "method": "POST",
                 "headers": {
                     "accept": "application/json",
@@ -2115,6 +2181,7 @@ class TestStreaming:
                     "content-type": "application/json",
                 },
                 "body": {
+                    "runId": "dry-run-1",
                     "streamAttachToken": "attach-123",
                     "datasourceId": 7,
                 },
@@ -2128,7 +2195,7 @@ class TestStreaming:
         capsys,
     ):
         def fake_urlopen(request, timeout=0):
-            if not request.full_url.endswith("/stream/check"):
+            if request.full_url != "https://sync.example/api/model-run-stream/check":
                 raise AssertionError(f"Unexpected URL {request.full_url}")
             return _FakeHttpResponse(
                 {
@@ -2153,12 +2220,12 @@ class TestStreaming:
             [
                 "--api-url",
                 "https://sync.example",
-                "--model-id",
-                "42",
                 "--api-key",
                 "user-token",
                 "--run-id",
                 "dry-run-2",
+                "--stream-attach-token",
+                "attach-123",
                 "--json",
             ],
         )