letify 2.8.2__tar.gz → 2.8.3__tar.gz

{letify-2.8.2 → letify-2.8.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: letify
-Version: 2.8.2
+Version: 2.8.3
 Summary: Declarations that become infrastructure.
 Project-URL: Homepage, https://github.com/DarkPyonix/letify
 Project-URL: Documentation, https://github.com/DarkPyonix/letify/tree/main/docs

{letify-2.8.2 → letify-2.8.3}/docs/SPEC.md

@@ -1162,7 +1162,7 @@ The archive is `<workspace root>/env-archives/<env key>-<platform>.tar`, uncompr
 
 The client's inbound link is the weak one, so the store path is slower than the build in both directions. Compression costs more than it saves on the volume, so the archive is not compressed.
 
-After a successful sync and interpreter check, the runtime says `caching the environment on the workspace root, <size>`, runs tar from the project directory's parent into a temporary name beside the destination, and renames it into place, so a concurrent session never reads a half written archive. Nothing is sent to the client, and no measured rate gates the pack, because the transfer stays inside the provider.
+After a successful sync and interpreter check, the runtime says `caching the environment on the workspace root, <size>`, runs tar from the project directory's parent into a temporary name beside the destination, and renames it into place, so a concurrent session never reads a half written archive. The temporary name carries 16 random hex digits, `<archive>.<hex>.partial`, not the process id, because every Modal sandbox runs its worker under the same small pid, so two sessions packing the same environment at once wrote one file and one of them failed with tar exit status 2. A pack that fails for any reason removes its temporary file, says `letify: <runtime>: not caching the environment on the workspace root (<reason>)`, and the session goes on: the environment is already built and checked, so a failed cache never fails the session or its call. Nothing is sent to the client, and no measured rate gates the pack, because the transfer stays inside the provider.
 
 At the start of `install_env`, when that archive exists on the runtime, the runtime says `restoring the environment from the workspace root`, extracts it into the project directory's parent, and validates the interpreter with the same check an archive restore uses. The restore refuses a member whose path escapes the destination and allows the absolute symlinks a `.venv` needs. A directory an earlier session left under the same name is removed first. When validation fails, the runtime removes the extracted tree, syncs, and replaces the archive. `Runtime.env_source` is `"workspace"` after this restore. This path is taken before and instead of the store path, and such a provider never uses the client store or the bucket store for its environment.
 
@@ -1931,7 +1931,7 @@ A jump host that letify cannot log in to, or whose `authorized_keys` it cannot e
 
 > Colab is a `Shell` whose rendezvous is `colab exec`. It has no forward SSH, and its fallback is `colab exec` with the Colab file API.
 
-The Colab CLI runs as `uv tool run --from google-colab-cli colab`, with `jupyter-kernel-client<1` pinned, because release 0.6.0 of the CLI calls an API that jupyter-kernel-client 1.0 removed. `colab new` and `colab stop` manage the session. `Colab.sessions()` reads `colab sessions` and returns the first word of each listing line. The CLI prints a session as `[<name>] <id> | Hardware: <hardware> | Variant: <variant>`, so a first word in square brackets gives the name inside them. A line starting with `[colab]` is a message from the CLI, such as `[colab] No active sessions found on server.`, and names no session, so an account with no session returns an empty list.
+The Colab CLI runs as `uv tool run --from google-colab-cli colab`, with `jupyter-kernel-client<1` pinned, because release 0.6.0 of the CLI calls an API that jupyter-kernel-client 1.0 removed. `colab new` and `colab stop` manage the session. `Colab.sessions()` reads `colab sessions` and returns the first word of each listing line. The CLI prints a session as `[<name>] <id> | Hardware: <hardware> | Variant: <variant>`, so a first word in square brackets gives the name inside them. A line starting with `[colab]` is a message from the CLI, such as `[colab] No active sessions found on server.`, and names no session, so an account with no session returns an empty list. `colab new` runs through the CLI's own Python with a request timeout of 600 s instead of google-auth's 120 s. google-auth binds `google.auth.transport.requests._DEFAULT_TIMEOUT` into `AuthorizedSession.request`'s signature when the module loads, so setting the module value alone changes nothing: the script sets it to 600 and wraps `AuthorizedSession.request` to default `timeout` to it, then calls the CLI's entry point `colab_cli.cli:main`. That is needed because the assignment request for an A100 can take longer than 120 s to answer and the CLI then fails with `ReadTimeout` while the server goes on assigning the machine. letify's own limit on `colab new` stays 900 s.
 
 **An assignment with no session.** `colab new` can fail after Colab has already assigned a machine, for example with `ReadTimeout` after 120 s, and the machine then bills (14.2 compute units per hour for an A100) with no local session state. `colab sessions` lists it as `[?] <endpoint> | ...`, and `colab stop -s <name>` cannot reach it. `Colab.create_session` therefore lists the account's assignments before `colab new`. When `colab new` raises, for any reason, it lists them again, releases every `?` assignment whose endpoint was not in the first list, and re-raises the original error. An assignment that existed before the attempt is never released. Each release prints one line on standard error naming the endpoint. When the first or the second listing fails, one line on standard error names the endpoint check and the error, and the original error is still the one raised. An assignment is released by endpoint with `Client.unassign` of the Colab CLI, run as `uv tool run --from google-colab-cli python -c <script> <endpoint>` with the account's environment, because the CLI has no command for it. `Colab.orphans()` returns the endpoints of the `?` rows of `colab sessions` and `Colab.stop_orphan(endpoint)` releases one, so `letify stop --orphans` covers Colab as it covers Kaggle.
 

{letify-2.8.2 → letify-2.8.3}/letify/__init__.py

@@ -68,7 +68,7 @@ distributed: Final = Host.distributed
 # The enum class stays importable from letify.declare.instance for letify's own use.
 del Host
 
-__version__ = "2.8.2"
+__version__ = "2.8.3"
 
 
 def fetch(tensor):  # type: ignore[no-untyped-def]

{letify-2.8.2 → letify-2.8.3}/letify/providers/colab.py

@@ -131,6 +131,26 @@ UNASSIGN_SCRIPT = (
 )
 
 
+#: Run in the CLI's own Python: ``colab`` with google-auth's request timeout raised to 600 s.
+#: The CLI passes no timeout, so a slow A100 assignment fails at the 120 s default while the
+#: server goes on assigning. google-auth binds the 120 s default into the signature of
+#: ``AuthorizedSession.request`` when it is defined, so setting the module value alone changes
+#: nothing and the method is wrapped as well. Spec "Colab", the Colab CLI paragraph.
+NEW_SCRIPT = (
+    "import sys\n"
+    "import google.auth.transport.requests as requests\n"
+    "requests._DEFAULT_TIMEOUT = 600\n"
+    "_request = requests.AuthorizedSession.request\n"
+    "def _slow_request(self, *args, **kwargs):\n"
+    "    kwargs.setdefault('timeout', requests._DEFAULT_TIMEOUT)\n"
+    "    return _request(self, *args, **kwargs)\n"
+    "requests.AuthorizedSession.request = _slow_request\n"
+    "from colab_cli.cli import main\n"
+    "sys.argv = ['colab', *sys.argv[1:]]\n"
+    "sys.exit(main())\n"
+)
+
+
 def _parse_assignments(listing: str) -> list[tuple[str, str]]:
     """The ``(name, endpoint)`` pairs in ``colab sessions`` output, ``?`` for no local state."""
     found = []
@@ -275,8 +295,21 @@ class Colab(Shell):
 
     # -- the CLI -------------------------------------------------------------
 
-    def _cli(self, *args: str, timeout: float | None = None, stdin: str | None = None) -> str:
-        command = [*self._colab(), *args]
+    def _cli(
+        self,
+        *args: str,
+        timeout: float | None = None,
+        stdin: str | None = None,
+        script: str | None = None,
+    ) -> str:
+        """Run the CLI; with ``script``, run that code in the CLI's Python with ``args`` as argv."""
+        if script is None:
+            command = [*self._colab(), *args]
+        else:
+            uv = tools.find_uv()
+            if uv is None:
+                raise ProviderUnavailable(self.kind, tools.missing_uv_message())
+            command = [*tools.python_command(tools.COLAB, uv), "-c", script, *args]
         result = subprocess.run(
             command,
             input=stdin,
@@ -286,9 +319,10 @@ class Colab(Shell):
             env=self._env(),
         )
         if result.returncode != 0:
+            shown = ["<script>" if part == script else part for part in command]
             raise RuntimeFailure(
-                f"`{' '.join(command)}` exited {result.returncode}",
-                command=" ".join(command),
+                f"`{' '.join(shown)}` exited {result.returncode}",
+                command=" ".join(shown),
                 stderr=(result.stderr or "").strip(),
             )
         return result.stdout
@@ -482,7 +516,7 @@ class Colab(Shell):
                 file=sys.stderr,
             )
         try:
-            self._cli(*args, timeout=900)
+            self._cli(*args, timeout=900, script=NEW_SCRIPT)
         except BaseException:
             # A timed out `colab new` can still leave a server-side assignment behind.
             if before is not None:

{letify-2.8.2 → letify-2.8.3}/letify/runtime/bootstrap.py

@@ -281,13 +281,15 @@ def workspace_pack_source(root: str, archive: str) -> str:
     Spec "Volumes on a persistent runtime": runs ``tar`` from the directory's parent into a
     temporary name and renames it into place, so a concurrent session never reads a half
     written archive. Uncompressed, because gzip costs more than it saves on a local volume.
+    The temporary name carries random hex, not the process id, because every Modal sandbox
+    runs its worker under the same pid. A failed pack removes its temporary file.
     """
     return (
-        "import os, subprocess\n"
+        "import os, secrets, subprocess\n"
         f"_root = os.path.expanduser({root!r}).rstrip('/')\n"
         f"_archive = os.path.expanduser({archive!r})\n"
         "os.makedirs(os.path.dirname(_archive), exist_ok=True)\n"
-        "_partial = _archive + '.' + str(os.getpid()) + '.partial'\n"
+        "_partial = _archive + '.' + secrets.token_hex(8) + '.partial'\n"
         "try:\n"
         "    subprocess.run(['tar', '-cf', _partial, '-C', os.path.dirname(_root),\n"
         "                    os.path.basename(_root)], check=True, capture_output=True)\n"

{letify-2.8.2 → letify-2.8.3}/letify/runtime/session.py

@@ -826,7 +826,16 @@ class Runtime:
             "caching the environment on the workspace root"
             + (f", {size / 1_000_000_000:.1f} GB" if size else "")
         )
-        self.eval(bootstrap.workspace_pack_source(root, self._workspace_archive()), timeout=3600)
+        source = bootstrap.workspace_pack_source(root, self._workspace_archive())
+        try:
+            self.eval(source, timeout=3600)
+        except Exception as error:
+            # The environment is built and checked, so a failed cache must not fail the call.
+            lines = str(error).strip().splitlines()
+            reason = f"{type(error).__name__}: {lines[0]}" if lines else type(error).__name__
+            if len(reason) > 200:
+                reason = reason[:197] + "..."
+            self._say(f"not caching the environment on the workspace root ({reason})")
 
     def _environment_volumes(self) -> tuple[Volume, ...]:
         """Explicit archive volumes, or the automatic cache where the project does not survive.

{letify-2.8.2 → letify-2.8.3}/letify-core/Cargo.lock

@@ -138,7 +138,7 @@ dependencies = [
 
 [[package]]
 name = "letify-agent"
-version = "2.8.2"
+version = "2.8.3"
 dependencies = [
  "letify-wire",
  "libloading",
@@ -146,7 +146,7 @@ dependencies = [
 
 [[package]]
 name = "letify-driver"
-version = "2.8.2"
+version = "2.8.3"
 dependencies = [
  "letify-wire",
 ]
@@ -164,7 +164,7 @@ dependencies = [
 
 [[package]]
 name = "letify-wire"
-version = "2.8.2"
+version = "2.8.3"
 
 [[package]]
 name = "libc"

{letify-2.8.2 → letify-2.8.3}/letify-core/Cargo.toml

@@ -8,7 +8,7 @@ resolver = "3"
 members = ["letify-wire", "letify-driver", "letify-agent", "letify-quic"]
 
 [workspace.package]
-version = "2.8.2"
+version = "2.8.3"
 edition = "2024"
 license = "MIT"
 repository = "https://github.com/DarkPyonix/letify"

{letify-2.8.2 → letify-2.8.3}/letify-ext/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "letify-status",
-  "version": "2.8.2",
+  "version": "2.8.3",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "letify-status",
-      "version": "2.8.2",
+      "version": "2.8.3",
       "license": "SEE LICENSE IN LICENSE",
       "devDependencies": {
         "@types/node": "^18.19.0",

{letify-2.8.2 → letify-2.8.3}/letify-ext/package.json

@@ -2,7 +2,7 @@
   "name": "letify-status",
   "displayName": "letify Status",
   "description": "letify quota and GPU activity in the status bar, read from the letify CLI.",
-  "version": "2.8.2",
+  "version": "2.8.3",
   "publisher": "darkpyonix",
   "license": "SEE LICENSE IN LICENSE",
   "repository": {

{letify-2.8.2 → letify-2.8.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "letify"
-version = "2.8.2"
+version = "2.8.3"
 description = "Declarations that become infrastructure."
 readme = "README.md"
 requires-python = ">=3.11"

{letify-2.8.2 → letify-2.8.3}/tests/conftest.py

@@ -2040,7 +2040,7 @@ class FakeColabCli:
         self.unassigned: list[str] = []
 
     def __call__(self, command):
-        if "python" in command and "-c" in command:
+        if "python" in command and "-c" in command and "new" not in command:
             self.unassigned.append(command[-1])
             self.assignments = [a for a in self.assignments if a[1] != command[-1]]
             return FakeCompleted()

{letify-2.8.2 → letify-2.8.3}/tests/test_drive_store.py

@@ -43,6 +43,32 @@ def declare_drive(server: Any, **settings: Any) -> None:
     paths.config_path().write_text("\n".join(lines) + "\n", encoding="utf-8")
 
 
+#: Tests that pin the gate's own rates and arithmetic, so they see the real constants.
+REAL_GATE = {
+    "test_the_gate_estimates_a_restore_as_size_over_the_download_and_extraction_rates",
+    "test_the_gate_packs_only_when_the_build_took_twice_the_estimated_restore",
+}
+
+
+@pytest.fixture(autouse=True)
+def pack_whatever_the_build_time(
+    request: pytest.FixtureRequest, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """Make the estimated restore negligible, so a session packs however fast it built.
+
+    A test session builds in one or two seconds, and whether that clears twice the
+    estimated restore of its small tree depends on the machine. A test that needs the
+    pack would otherwise pass or fail by timing. Tests in ``REAL_GATE`` and tests that set
+    the rates themselves see their own values.
+    """
+    if request.node.name in REAL_GATE:
+        return
+    from letify.runtime import session
+
+    monkeypatch.setattr(session, "DRIVE_DOWNLOAD_BPS", 1e18)
+    monkeypatch.setattr(session, "EXTRACT_BPS", 1e18)
+
+
 def drive_options(server: Any, **extra: Any) -> dict[str, Any]:
     """Account options of an ephemeral account, after the machine has declared Drive."""
     declare_drive(server)

{letify-2.8.2 → letify-2.8.3}/tests/test_providers.py

@@ -398,6 +398,9 @@ COLAB_CLI = [
 ]
 
 
+COLAB_PYTHON = [*COLAB_CLI[:-1], "python"]
+
+
 def test_a_missing_uv_says_how_to_get_it(isolated_home, patch_which) -> None:
     # The Colab CLI runs through uv, so uv is the one thing that has to be installed.
     patch_which(tools_module, present=False)
@@ -414,7 +417,9 @@ def test_creating_a_colab_session_names_the_accelerator_the_cli_accepts(
     recorder = patch_run(colab_module)
     provider = provider_of(Colab, "colab_a")
     provider.create_session(provider.RTX_PRO_6000, "letify-g4-1")
-    assert recorder.command == [*COLAB_CLI, "new", "-s", "letify-g4-1", "--gpu", "G4"]
+    assert recorder.command == [
+        *COLAB_PYTHON, "-c", colab_module.NEW_SCRIPT, "new", "-s", "letify-g4-1", "--gpu", "G4"
+    ]
 
 
 def test_creating_a_colab_session_for_a_tpu_asks_for_a_tpu(
@@ -424,7 +429,9 @@ def test_creating_a_colab_session_for_a_tpu_asks_for_a_tpu(
     recorder = patch_run(colab_module)
     provider = provider_of(Colab, "colab_a")
     provider.create_session(provider.v5e1, "letify-v5e1-1")
-    assert recorder.command == [*COLAB_CLI, "new", "-s", "letify-v5e1-1", "--tpu", "v5e1"]
+    assert recorder.command == [
+        *COLAB_PYTHON, "-c", colab_module.NEW_SCRIPT, "new", "-s", "letify-v5e1-1", "--tpu", "v5e1"
+    ]
 
 
 def test_colab_offers_a_cpu_instance_under_the_name_local_uses(isolated_home) -> None:
@@ -441,7 +448,69 @@ def test_creating_a_colab_session_for_the_cpu_asks_for_no_accelerator(
     recorder = patch_run(colab_module)
     provider = provider_of(Colab, "colab_a")
     provider.create_session(provider.cpu, "letify-cpu-1")
-    assert recorder.command == [*COLAB_CLI, "new", "-s", "letify-cpu-1"]
+    assert recorder.command == [
+        *COLAB_PYTHON, "-c", colab_module.NEW_SCRIPT, "new", "-s", "letify-cpu-1"
+    ]
+
+
+def test_colab_new_raises_the_request_timeout_to_600_s_and_keeps_the_900_s_limit(
+    isolated_home, patch_which, patch_run
+) -> None:
+    # Spec "Colab": google-auth's 120 s default is too short for an A100 assignment.
+    patch_which(tools_module, present=True)
+    recorder = patch_run(colab_module)
+    provider = provider_of(Colab, "colab_a")
+    provider.create_session(provider.A100, "letify-a100-1")
+    call = next(c for c in recorder.calls if "new" in c["command"])
+    assert "_DEFAULT_TIMEOUT = 600" in colab_module.NEW_SCRIPT
+    assert "colab_cli.cli" in colab_module.NEW_SCRIPT
+    assert call["timeout"] == 900
+    assert call["command"][-5:] == ["new", "-s", "letify-a100-1", "--gpu", "A100"]
+
+
+def test_the_new_script_sets_the_timeout_before_calling_the_cli_and_exits_with_its_code() -> None:
+    # Run the script with a stub colab_cli, to see the order and the exit code.
+    import subprocess as sp
+    import tempfile
+    from pathlib import Path
+
+    with tempfile.TemporaryDirectory() as root:
+        stub = Path(root, "stub")
+        (stub / "colab_cli").mkdir(parents=True)
+        (stub / "colab_cli" / "__init__.py").write_text("")
+        (stub / "colab_cli" / "cli.py").write_text(
+            "import sys\n"
+            "import google.auth.transport.requests as r\n"
+            "def main():\n"
+            "    print(r._DEFAULT_TIMEOUT, r.AuthorizedSession().request('GET', 'u'), sys.argv)\n"
+            "    raise SystemExit(3)\n"
+        )
+        google = Path(root, "stub", "google", "auth", "transport")
+        google.mkdir(parents=True)
+        for d in (google.parent.parent, google.parent, google):
+            (d / "__init__.py").write_text("")
+        (google / "requests.py").write_text(
+            "_DEFAULT_TIMEOUT = 120\n"
+            "class AuthorizedSession:\n"
+            "    def request(self, method, url, timeout=_DEFAULT_TIMEOUT):\n"
+            "        return timeout\n"
+        )
+        done = sp.run(
+            [sys.executable, "-c", colab_module.NEW_SCRIPT, "new", "-s", "x"],
+            capture_output=True, text=True, env={"PYTHONPATH": str(stub)},
+        )
+    assert done.returncode == 3, done.stderr
+    assert done.stdout.strip() == "600 600 ['colab', 'new', '-s', 'x']"
+
+
+def test_a_failed_colab_new_raises_with_the_command_and_stderr(
+    isolated_home, patch_which, patch_run
+) -> None:
+    provider = _colab_with(patch_which, patch_run, FakeColabCli(new_fails=True))
+    with pytest.raises(letify.RuntimeFailure) as caught:
+        provider.create_session(provider.A100, "letify-a100-1")
+    assert "ReadTimeout" in caught.value.stderr
+    assert "new -s letify-a100-1 --gpu A100" in caught.value.command
 
 
 def test_a_failing_cli_command_names_the_command_and_the_end_of_its_stderr(

{letify-2.8.2 → letify-2.8.3}/tests/test_runtime.py

@@ -2706,6 +2706,81 @@ def test_a_persistent_provider_that_builds_its_venv_elsewhere_archives_on_its_wo
     assert "caching the environment on the workspace root" in err, err
 
 
+def test_the_workspace_pack_names_its_temporary_file_without_the_process_id(
+    tmp_path: Path, monkeypatch
+) -> None:
+    # Spec "Volumes on a persistent runtime": every Modal sandbox runs its worker under the
+    # same small pid, so the temporary name carries 16 random hex digits instead.
+    import re
+    import subprocess
+
+    project = tmp_path / "proj"
+    project.mkdir()
+    (project / "a.txt").write_text("a")
+    archive = tmp_path / "vol" / "env.tar"
+    seen: list[str] = []
+    real = subprocess.run
+
+    def record(cmd, *args, **kwargs):
+        seen.append(cmd[2])
+        return real(cmd, *args, **kwargs)
+
+    monkeypatch.setattr(subprocess, "run", record)
+    source = bootstrap.workspace_pack_source(str(project), str(archive))
+    for _ in range(2):
+        exec(source, {})
+    assert len(seen) == 2 and seen[0] != seen[1]
+    for name in seen:
+        assert re.fullmatch(re.escape(str(archive)) + r"\.[0-9a-f]{16}\.partial", name), name
+    assert archive.is_file()
+    assert not list(archive.parent.glob("*.partial"))
+
+
+def test_a_pack_that_fails_removes_its_temporary_file(tmp_path: Path) -> None:
+    # Spec "Volumes on a persistent runtime": a failed pack leaves no partial behind.
+    project = tmp_path / "proj"
+    project.mkdir()
+    archive = tmp_path / "vol" / "env.tar"
+    source = bootstrap.workspace_pack_source(str(project), str(archive)).replace(
+        "['tar', '-cf', _partial,", "['sh', '-c', 'echo x > \"$0\"; exit 2', _partial,"
+    )
+    with pytest.raises(subprocess.CalledProcessError):
+        exec(source, {})
+    assert not list(archive.parent.glob("*.partial")), "a temporary archive was left behind"
+    assert not archive.exists()
+
+
+def test_a_failed_workspace_pack_does_not_fail_the_session(
+    uv_project: Path, tmp_path: Path, monkeypatch, capsys
+) -> None:
+    # Spec "Volumes on a persistent runtime": the environment is already built, so a failed
+    # cache is said and the session goes on.
+    provider, volume = _elsewhere_provider(tmp_path)
+    env = Env()
+    real = bootstrap.workspace_pack_source
+
+    def failing(root: str, archive: str) -> str:
+        return real(root, archive).replace(
+            "['tar', '-cf', _partial,", "['sh', '-c', 'echo x > \"$0\"; exit 2', _partial,"
+        )
+
+    monkeypatch.setattr(bootstrap, "workspace_pack_source", failing)
+    instance = Instance(provider, gpu=None)._placed("remote")
+    runtime = provider.start(instance, env, name="lab-1", volumes=(volume,))
+    try:
+        assert runtime.env_source == "sync"
+        assert runtime.eval("__letify_value__ = 6 * 7") == 42
+        archive = _workspace_archive(env, runtime.platform)
+    finally:
+        runtime.shutdown()
+    err = capsys.readouterr().err
+    assert (
+        "not caching the environment on the workspace root (RemoteError: CalledProcessError" in err
+    ), err
+    assert not archive.exists()
+    assert not list(archive.parent.glob("*.partial"))
+
+
 def test_a_second_session_restores_the_environment_from_the_workspace_root(
     uv_project: Path, tmp_path: Path, monkeypatch, capsys
 ) -> None: