letify 2.6.2__tar.gz → 2.6.4__tar.gz

{letify-2.6.2 → letify-2.6.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: letify
-Version: 2.6.2
+Version: 2.6.4
 Summary: Declarations that become infrastructure.
 Project-URL: Homepage, https://github.com/DarkPyonix/letify
 Project-URL: Documentation, https://github.com/DarkPyonix/letify/tree/main/docs

{letify-2.6.2 → letify-2.6.4}/docs/INTENT.md

@@ -74,6 +74,28 @@ cross cloud to cloud and the client carries the mappings and nothing else. Measu
 inbound on the network this was designed on is capped near 0.25 MiB/s, against 21 MiB/s
 outbound to a near runtime, which is why this is a claim rather than an optimization.
 
+**TCP hole punching does not reach between two Kaggle runtimes.** Measured on 2026-10-13 between two
+Google Cloud VMs on two Kaggle accounts: both NATs preserved the port, 45117 on each, so the
+signalling and the mappings were right, and every SYN was dropped for the whole 45 second window from
+both sides at once. So the mechanism this claim names works client to runtime, where it was measured,
+and not runtime to runtime on that network. The claim is not refuted, because what it asserts is that
+the reduction must not pass through the client, and that still holds. What is refuted is that a TCP
+punch is the way to arrange it on Kaggle.
+
+**A UDP punch does not reach either.** Measured the same way on the same pair of accounts: both NATs
+preserved the port, 45223 on each, both sides sent datagrams for 30 seconds, and neither received
+one. So neither protocol reaches between two Kaggle runtimes with a mapping learned from a third
+party, and the distributed placement has no runtime to runtime transport on that pair today. A pair
+that cannot connect refuses the reduction, which is what letify does, so the behaviour is right and
+the placement is unusable there rather than slow.
+
+Two paths remain, and which one letify takes is an open decision rather than something to assume.
+Either both runtimes dial outward to a third host that is not the client, which is a relay and has to
+be argued against the constraint that a slow link is a failure, or they exchange parameters through
+object storage in one of their clouds, which letify already reaches with the `bucket` account option
+and the GCS backend. The second needs no inbound anywhere, which is the property the measurements
+above say is missing.
+
 ## How an experiment reports efficiency
 
 This standard sits here rather than in `docs/SPEC.md` because it governs how a claim is tested, not what the system does. The spec records the design; this records what a report has to show before a claim above may be called supported. Every experiment pull request follows it.
@@ -108,7 +130,7 @@ Each of these would change a claim or a default. Answering one is a good first e
 
 1. **How many concurrent sessions does one Colab account allow?** Undocumented, and it moves with tier, credit balance and demand. Until it is measured, a `devices` count in the provider entry is where the answer goes, so a user who has measured their own account is not overruled by a number letify guessed.
 2. **What is the real host synchronization count per step, `k`, for the target workload?** Measurable with `torch.cuda.set_sync_debug_mode("warn")`. This sets whether PyTorch forwarding is worth using for that workload.
-3. **How often does TCP hole punching succeed on the networks researchers actually use?** It succeeded between a Colab VM and a university network in Korea, where both NATs preserved the port. Home routers, office networks and mobile tethering are unmeasured. This decides how often the pipeline falls to UDP or to the provider's own path.
+3. **How often does TCP hole punching succeed on the networks researchers actually use?** It succeeded between a Colab VM and a university network in Korea, where both NATs preserved the port. It failed between two Google Cloud VMs on two Kaggle accounts, where both NATs also preserved the port and every SYN was dropped, so a preserved port is not sufficient. Home routers, office networks and mobile tethering are unmeasured. This decides how often the pipeline falls to UDP or to the provider's own path, and runtime to runtime it decides whether the distributed placement has a transport on a given pair of providers.
 4. **Is NVFP4 reachable in a stock Colab runtime?** Needs the CUDA version, the compute capability and whether the quantization stack installs.
 5. **Is the Elice SSH port stable across a restart?** If it is not, the configuration needs a command that resolves the current port.
 6. **What does Elice spot pricing cost?** The API exposes a pricing id, which suggests preemptible instances are available. This is a direct cost lever.

{letify-2.6.2 → letify-2.6.4}/docs/SPEC.md

@@ -111,6 +111,30 @@ parameters into the weak direction every round. A pair of replicas that cannot p
 fails the reduction and says so; it does not fall back to the client, for the reason
 [Link floor](#link-floor) gives.
 
+A replica joins a round with two addresses for the listener it opened: the one its own interface
+reports, and the one a STUN server says its NAT gives that port. The broker passes both to every
+other joined replica, with a token the pair agrees a connection on and the time both start dialing,
+which is what [TCP hole punching](#tcp-punch) already needs from the client when the client is one
+of the two ends. A replica then tries the address its peer reported for itself first, because two
+replicas on one machine or one network reach each other without a punch, and punches to the mapped
+address when that connection is refused. The punch is the same `punch` call a strategy makes. The
+replica that would have dialed anyway, the one that is not the leader, is the initiator, so the two
+sides never both take that role whatever their ranks are.
+
+The distributed placement is supported on a pair of runtimes that can reach each other, and nowhere
+else. Two Kaggle runtimes cannot: measured on 2026-10-13, both NATs preserved the port and every TCP
+SYN was dropped for 45 s, and a UDP punch on the same pair sent for 30 s each way without a datagram
+arriving. letify therefore refuses such a pair rather than routing the reduction through the client
+or a relay, which is the decision `docs/INTENT.md` records against claim N10. A declaration naming
+two runtimes that cannot connect fails, and the measurement to make before using a provider pair is
+whether they punch at all.
+
+Punching is between a pair, because one punched connection takes one local port on each side and
+the two sides have to agree which pair that port belongs to. So a round of more than two replicas
+whose members cannot reach each other directly is refused, naming the ranks that could not connect,
+rather than punching a mesh letify has not measured. Two replicas is the case claim N9's boundary is
+measured on, and more than two over direct addresses already works.
+
 ### How often to synchronize <!-- id: distributed-local-steps -->
 
 > `letify.local_steps(param_bytes, step_seconds)` answers how many steps to take between
@@ -783,7 +807,11 @@ A `read_until` blocks until the sandbox prints a line, and the adapter answers o
 - `Modal.stop` calls `abort()` when the adapter is out of step or its lock is still held after 5 s. The call timeout watchdog calls it through `SandboxChannel._kill`, so a call past its `timeout` ends its sandbox too.
 - The adapter's `terminate` for an id it did not create finds the sandbox with `modal.Sandbox.from_id` and terminates it. A sandbox that is already gone is not an error.
 
-Modal also bounds a sandbox that nothing terminates. `create` passes `timeout`, the entry option `timeout`, 3600 s by default, as the sandbox's maximum lifetime. It passes `idle_timeout`, the entry option `idle_timeout`, 600 s by default, after which Modal terminates a sandbox that is idle.
+Modal also bounds a sandbox that nothing terminates. `create` passes `idle_timeout`, the entry option `idle_timeout`, 600 s by default, after which Modal terminates a sandbox that is idle. That is the bound that stops an abandoned sandbox billing, because an abandoned one is idle by definition once its worker has no one to answer.
+
+It passes `timeout`, the entry option `timeout`, as the sandbox's maximum lifetime, and the default is the longest Modal accepts, 86400 s, measured as accepted. It is not a shorter number letify picked, for the reason The declaration gives for `timeout` having no default: a deadline letify invented would end a two hour training run at whatever hour it guessed. The default was 3600 s, exactly one hour, and it ended two Modal sessions of a real training run at one hour each with nothing saying why, which is that sentence happening. An account that wants a hard cap sets `timeout` itself.
+
+A sandbox that reaches that lifetime takes its worker with it, so the channel sees a worker that stopped without replying. Where the session's age is within a minute of the lifetime letify asked for, the failure says so instead of listing causes: `the Modal sandbox reached the <seconds> s lifetime letify asked for; raise timeout on the account to run longer`.
 
 ### Modal data channel <!-- id: modal-data-channel -->
 
@@ -1078,6 +1106,8 @@ Before a blob is written to a destination that is not unpacked, the local proces
 
 A persistent provider builds its environment with `uv sync` in `<workspace root>/project/<env key>` every session, and never packs the project directory into a volume or unpacks an archive from one. A sync over an existing `.venv` checks it and installs nothing, so it is faster than any archive transfer, and the `.venv` is already on the disk the archive would be unpacked to.
 
+That holds only while the project directory is under the workspace root. A provider that keeps its workspace root and still builds the `.venv` elsewhere archives like an ephemeral one, because the directory the sync writes does not survive the session however durable the workspace root is. Modal is that provider: its workspace root is a Modal volume and its `env_root` is the sandbox's own disk, chosen because a cold import from the volume reads thousands of small files over the network. Without this, Modal rebuilt 230 packages on every session, measured at 428 s and 534 s on an A10, and had neither the persistence its workspace root offers nor the archive an ephemeral provider gets. Whether the archive is then worth packing is the measurement Materializing into a runtime already makes.
+
 An ephemeral provider keeps the behaviour of Materializing into a runtime: every file is sent each session, and the environment archive is packed and restored.
 
 Nothing hands a session to the caller. There is no call that returns one, no argument that takes one, and no way to hold the wrong one, because which session serves a call is the pool's answer to work out from the declaration.

{letify-2.6.2 → letify-2.6.4}/letify/__init__.py

@@ -66,7 +66,7 @@ distributed: Final = Host.distributed
 # The enum class stays importable from letify.declare.instance for letify's own use.
 del Host
 
-__version__ = "2.6.2"
+__version__ = "2.6.4"
 
 
 def fetch(tensor):  # type: ignore[no-untyped-def]

{letify-2.6.2 → letify-2.6.4}/letify/declare/reduce.py

@@ -68,6 +68,7 @@ def all_reduce(
     op: Callable[[list], Any] | None = None,
     timeout: float = DEFAULT_TIMEOUT,
     min_peers: int | None = None,
+    _watch: dict | None = None,
 ) -> int:
     """Replace ``tensor``'s contents with the reduction of every replica's, in place.
 
@@ -111,19 +112,16 @@ def all_reduce(
         round_id = _ROUND
 
     deadline = time.monotonic() + timeout
-    listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    listener, port, mapped = _listener(world)
+    host = _local_address()
     try:
-        listener.bind(("0.0.0.0", 0))
-        listener.listen(world)
-        port = listener.getsockname()[1]
-        host = _local_address()
-
         main._reduce_join(
             {
                 "op": "reduce_join",
                 "round": round_id,
                 "rank": my_rank,
                 "address": [host, port],
+                "mapped": list(mapped) if mapped is not None else None,
                 "timeout": timeout,
                 "min_peers": needed,
             }
@@ -138,22 +136,58 @@ def all_reduce(
 
         leader = int(table["leader"])
         count = int(table["count"])
-        peers = {int(r): tuple(address) for r, address in table["peers"].items()}
-        header = {"rank": my_rank, "nbytes": len(raw)}
+        token = str(table.get("token", ""))
+        start_at = float(table.get("start_at", time.time()))
+        peers = {
+            int(r): {
+                "address": tuple(entry["address"]),
+                "mapped": tuple(entry["mapped"]) if entry.get("mapped") else None,
+            }
+            for r, entry in table["peers"].items()
+        }
+        header = {"rank": my_rank, "round": round_id, "nbytes": len(raw)}
+
+        if _watch is not None:
+            _watch["peers"] = table.get("peers")
+            _watch["token"] = token
+            _watch["start_at"] = start_at
 
         if my_rank == leader:
-            result = _lead(listener, deadline, my_rank, tensor, header, raw, count, op)
+            result = _lead(
+                listener, deadline, my_rank, tensor, header, raw, count, op, peers, token,
+                start_at, port, round_id,
+            )
         else:
-            result = _follow(peers[leader], deadline, header, raw)
+            result = _follow(leader, peers, deadline, header, raw, token, start_at, port, count)
     except OSError as exc:
         raise ReductionFailed(f"all_reduce round {round_id} failed: {exc}") from exc
-    finally:
-        listener.close()
 
     _write_back(tensor, result)
     return count
 
 
+#: The replica's listener, its port and its STUN mapping, opened once and kept.
+#: Spec "Reducing between replicas": a loop that reduces every H steps would otherwise
+#: pay a STUN round trip and a fresh punch on every round.
+_LISTENER: list = []
+_LISTENER_LOCK = threading.Lock()
+
+
+def _listener(world: int) -> tuple[Any, int, tuple[str, int] | None]:
+    """This replica's listening socket, its port and its mapped address, made once."""
+    with _LISTENER_LOCK:
+        if not _LISTENER:
+            sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            if hasattr(socket, "SO_REUSEPORT"):
+                sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+            sock.bind(("0.0.0.0", 0))
+            sock.listen(max(1, world))
+            port = sock.getsockname()[1]
+            _LISTENER.append((sock, port, _mapped_address(port)))
+        return _LISTENER[0]
+
+
 def _local_address() -> str:
     """The host peers can reach this one on. No configuration knob: it is the machine's
     own address, as spec "Reducing between replicas" scopes the local transport."""
@@ -163,6 +197,40 @@ def _local_address() -> str:
         return "127.0.0.1"
 
 
+def _mapped_address(port: int) -> tuple[str, int] | None:
+    """The STUN mapping of the listener's own ``port``, or None when it cannot be had.
+
+    Spec "Reducing between replicas": a replica joins a round with two addresses, its
+    own and this one. Never raises: a replica with no mapping still joins, with
+    ``mapped`` absent, and is still reachable on its own address.
+    """
+    try:
+        from ..transport import nat
+    except ImportError:
+        return None
+    try:
+        return nat.stun_mapping(port)
+    except Exception:
+        return None
+
+
+def _pair_for_punching(*, count: int, ranks: list[int]) -> tuple[int, int]:
+    """The two ranks a punch is between, or a refusal naming every rank in the round.
+
+    Spec "Reducing between replicas": one punched connection takes one local port on
+    each side, so punching a round of more than two replicas is refused rather than
+    attempted as a mesh letify has not measured.
+    """
+    ordered = sorted(ranks)
+    if count == 2 and len(ordered) == 2:
+        return ordered[0], ordered[1]
+    named = ", ".join(str(r) for r in ordered)
+    raise ReductionFailed(
+        f"a round of {count} replicas over ranks {named} cannot punch to reach each "
+        "other directly: punching is between a pair, not a mesh of more than two"
+    )
+
+
 def _await_table(main: Any, round_id: int, deadline: float) -> dict:
     """Block until the client answers this round's join, or the deadline passes."""
     cond = main._REDUCE_LOCK
@@ -187,18 +255,62 @@ def _lead(
     raw: bytes,
     count: int,
     op: Callable[[list], Any],
+    peers: dict[int, dict],
+    token: str,
+    start_at: float,
+    port: int,
+    round_id: int,
 ) -> Any:
-    """Accept every other joined peer, reduce, and send the result back to each."""
+    """Accept every other joined peer, punching for one that never connects, then reduce
+    and send the result back to each.
+
+    Spec "Reducing between replicas": the leader accepts on its own listener first,
+    because two replicas on one machine or one network reach each other without a
+    punch, and punches with ``initiator=False`` only for a peer that never arrives.
+    """
     collected: dict[int, Any] = {my_rank: tensor}
     conns: list[socket.socket] = []
     try:
+        window_deadline = min(deadline, start_at + _window())
         while len(collected) < count:
-            listener.settimeout(_remaining(deadline))
-            conn, _addr = listener.accept()
+            remaining_window = window_deadline - time.monotonic()
+            if remaining_window <= 0:
+                break
+            listener.settimeout(max(0.01, min(remaining_window, _remaining(deadline))))
+            try:
+                conn, _addr = listener.accept()
+            except TimeoutError:
+                break
             conn.settimeout(_remaining(deadline))
             peer_header, payload = _recv_message(conn)
+            if int(peer_header.get("round", round_id)) != round_id:
+                # The listener outlives a round, so a connection a previous one left
+                # behind can be accepted here. Spec "Reducing between replicas".
+                conn.close()
+                continue
             collected[int(peer_header["rank"])] = _rebuild(tensor, payload)
             conns.append(conn)
+
+        missing = sorted(r for r in peers if r not in collected)
+        if missing:
+            _pair_for_punching(count=count, ranks=sorted(peers))
+            from ..transport import nat
+
+            for peer_rank in missing:
+                mapped = peers[peer_rank]["mapped"]
+                if mapped is None:
+                    raise ReductionFailed(
+                        f"replica {peer_rank} never connected directly and reported no "
+                        "mapped address to punch to"
+                    )
+                conn = nat.punch(
+                    port, mapped, bytes.fromhex(token), initiator=False, start_at=start_at
+                )
+                conn.settimeout(_remaining(deadline))
+                peer_header, payload = _recv_message(conn)
+                collected[int(peer_header["rank"])] = _rebuild(tensor, payload)
+                conns.append(conn)
+
         arrays = [collected[r] for r in sorted(collected)]
         result = op(arrays)
         result_header, result_raw = _encode(result)
@@ -211,9 +323,34 @@ def _lead(
             conn.close()
 
 
-def _follow(leader_address: tuple, deadline: float, header: dict, raw: bytes) -> bytes:
-    """Send this replica's array to the leader, and wait for the reduced result's bytes."""
-    conn = socket.create_connection(leader_address, timeout=_remaining(deadline))
+def _follow(
+    leader_rank: int,
+    peers: dict[int, dict],
+    deadline: float,
+    header: dict,
+    raw: bytes,
+    token: str,
+    start_at: float,
+    port: int,
+    count: int,
+) -> bytes:
+    """Send this replica's array to the leader, and wait for the reduced result's bytes.
+
+    Spec "Reducing between replicas": the address the leader reported for itself is
+    tried first, and the mapped address is punched to with ``initiator=True`` only when
+    that direct connection fails.
+    """
+    peer = peers[leader_rank]
+    try:
+        conn = socket.create_connection(peer["address"], timeout=_remaining(deadline))
+    except OSError:
+        _pair_for_punching(count=count, ranks=sorted(peers))
+        mapped = peer["mapped"]
+        if mapped is None:
+            raise
+        from ..transport import nat
+
+        conn = nat.punch(port, mapped, bytes.fromhex(token), initiator=True, start_at=start_at)
     try:
         conn.settimeout(_remaining(deadline))
         _send_message(conn, header, raw)
@@ -224,6 +361,15 @@ def _follow(leader_address: tuple, deadline: float, header: dict, raw: bytes) ->
         conn.close()
 
 
+def _window() -> float:
+    """The punch window, from ``letify.transport.nat`` when it can be imported."""
+    try:
+        from ..transport import nat
+    except ImportError:
+        return 45.0
+    return nat.DEFAULT_WINDOW
+
+
 def _remaining(deadline: float) -> float:
     left = deadline - time.monotonic()
     if left <= 0:

{letify-2.6.2 → letify-2.6.4}/letify/providers/modal.py

@@ -102,6 +102,26 @@ STOP_WAIT = 5.0
 #: ``idle_timeout``.
 IDLE_TIMEOUT = 600
 
+#: The sandbox's maximum lifetime, the longest Modal accepts, measured as accepted.
+#: Spec "Modal": not a shorter number letify picked, for the reason The declaration gives
+#: for ``timeout`` having no default. The previous 3600 s ended two sessions of a real
+#: training run at one hour each with nothing saying why.
+SANDBOX_TIMEOUT = 86400
+
+
+def lifetime_detail(*, age_s: float, lifetime_s: int) -> str | None:
+    """What to say when a sandbox died at the lifetime letify asked for, or None.
+
+    Spec "Modal": the channel otherwise reports a worker that stopped without replying and
+    lists causes, none of which is the deadline letify set itself.
+    """
+    if lifetime_s <= 0 or age_s < lifetime_s - 60.0:
+        return None
+    return (
+        f"the Modal sandbox reached the {lifetime_s} s lifetime letify asked for; "
+        f"raise timeout on the account to run longer"
+    )
+
 
 class VolumePathMissing(RuntimeFailure):
     """The adapter answered that a volume path does not exist."""
@@ -377,6 +397,18 @@ class Modal(Provider):
         self._sandboxes: dict[str, str] = {}
         self._adapter: Adapter | None = None
 
+    @property
+    def sandbox_timeout_s(self) -> int:
+        """The sandbox's maximum lifetime, from the account or the longest Modal accepts."""
+        value = self.config.option("timeout", SANDBOX_TIMEOUT)
+        return max(1, int(value))
+
+    @property
+    def sandbox_idle_timeout_s(self) -> int:
+        """Seconds a sandbox may be idle before Modal terminates it. Spec "Modal"."""
+        value = self.config.option("idle_timeout", IDLE_TIMEOUT)
+        return max(1, int(value))
+
     def adapter(self) -> Adapter:
         """The adapter acting as this account, started on its first request."""
         if self._adapter is None or self._adapter.closed:
@@ -440,14 +472,15 @@ class Modal(Provider):
             args=["python3", "-u", "-c", BOOTSTRAP],
             packages=list(WORKER_PACKAGES),
             gpu=self.wire_name(runtime.instance) or None,
-            timeout=int(self.config.option("timeout", 3600)),
-            idle_timeout=int(self.config.option("idle_timeout", IDLE_TIMEOUT)),
+            timeout=self.sandbox_timeout_s,
+            idle_timeout=self.sandbox_idle_timeout_s,
             ports=[] if data_port is None else [data_port],
         )
         sandbox = str(created["sandbox"])
         self._sandboxes[runtime.name] = sandbox
         return SandboxChannel(
-            adapter, sandbox, name=runtime.name, data_port=data_port, streams=streams
+            adapter, sandbox, name=runtime.name, data_port=data_port, streams=streams,
+            lifetime_s=self.sandbox_timeout_s,
         )
 
     def stop(self, runtime: Runtime) -> None:
@@ -598,11 +631,15 @@ class SandboxChannel(FramedChannel):
         name: str,
         data_port: int | None = None,
         streams: int = 1,
+        lifetime_s: int = 0,
     ):
         self.adapter = adapter
         self.sandbox = sandbox
         self.name = name
         self.data_port = data_port
+        #: The sandbox's maximum lifetime, so a death at it is named rather than guessed.
+        self.lifetime_s = lifetime_s
+        self.opened_at = time.monotonic()
         #: TCP connections the data channel carries its frames over.
         self.streams = streams
         self._sockets: list[socket.socket] = []
@@ -628,7 +665,13 @@ class SandboxChannel(FramedChannel):
         self._await_ready()
 
     def _raw_text(self) -> str:
-        return bytes(self._raw[-2000:]).decode("utf-8", "replace")
+        text = bytes(self._raw[-2000:]).decode("utf-8", "replace")
+        # Spec "Modal": a sandbox that reached the lifetime letify asked for takes its
+        # worker with it, and the channel otherwise lists causes, none of them this one.
+        said = lifetime_detail(
+            age_s=time.monotonic() - self.opened_at, lifetime_s=self.lifetime_s
+        )
+        return f"{said}\n{text}" if said else text
 
     def _send_worker(self) -> None:
         # After a reexec the data connection is gone, so the source goes back over stdio.

{letify-2.6.2 → letify-2.6.4}/letify/runtime/broker.py

@@ -10,6 +10,7 @@ claim N10.
 from __future__ import annotations
 
 import json
+import secrets
 import threading
 import time
 from typing import Any
@@ -24,6 +25,10 @@ _TOTAL_BYTES = 0
 #: Seconds between polls of a replica's joins, and between sweeps for an expired round.
 _POLL_S = 0.02
 
+#: Seconds ahead of now a round's agreed start moment is set, so every replica's
+#: answer arrives before dialing is meant to begin.
+_START_AHEAD_S = 1.0
+
 
 def _account(payload: dict) -> None:
     """Add the size of one brokered payload to the running total."""
@@ -134,10 +139,26 @@ class Broker:
             self._fail(round_id, joins)
 
     def _answer(self, round_id: int, joins: dict[int, dict]) -> None:
-        """Send every joined replica the peer table, naming the lowest rank leader."""
+        """Send every joined replica the peer table, naming the lowest rank leader.
+
+        Spec "Reducing between replicas": each entry carries both addresses a replica
+        joined with, and the table carries one token and one start moment, the same
+        for every replica of this round, so a pair can agree a punch without the client
+        relaying anything more than this.
+        """
         leader = min(joins)
-        peers = {str(r): list(join["address"]) for r, join in joins.items()}
-        payload = {"round": round_id, "peers": peers, "leader": leader, "count": len(joins)}
+        peers = {
+            str(r): {"address": list(join["address"]), "mapped": join.get("mapped")}
+            for r, join in joins.items()
+        }
+        payload = {
+            "round": round_id,
+            "peers": peers,
+            "leader": leader,
+            "count": len(joins),
+            "token": secrets.token_hex(16),
+            "start_at": time.time() + _START_AHEAD_S,
+        }
         self._send_to(joins, payload)
 
     def _fail(self, round_id: int, joins: dict[int, dict]) -> None:

{letify-2.6.2 → letify-2.6.4}/letify/runtime/session.py

@@ -174,7 +174,11 @@ class Runtime:
             # Not retried, so nothing else would end this session.
             self.shutdown(reason="interpreter_mismatch")
             raise
-        if self.env_source == "sync" and not self.provider.persistent:
+        # The same rule as _environment_volumes: a project directory that does not
+        # survive the session is worth archiving whatever the provider calls itself.
+        if self.env_source == "sync" and not (
+            self.provider.persistent and self._project_survives()
+        ):
             from . import bootstrap
 
             root = bootstrap.project_dir(
@@ -643,8 +647,15 @@ class Runtime:
         self.workspace = self.eval(bootstrap.workspace_source(root), timeout=120)
 
     def _environment_volumes(self) -> tuple[Volume, ...]:
-        """Explicit archive volumes, or the automatic cache for an ephemeral provider."""
-        if self.provider.persistent:
+        """Explicit archive volumes, or the automatic cache where the project does not survive.
+
+        Spec "Volumes on a persistent runtime": a persistent provider needs no archive
+        because its project directory is on a disk that lasts, which holds only while that
+        directory is under the workspace root. One that keeps its workspace and builds the
+        .venv elsewhere archives like an ephemeral one, because the sync's output dies with
+        the session however durable the root is.
+        """
+        if self.provider.persistent and self._project_survives():
             return ()
         if self.volumes:
             return self.volumes
@@ -652,6 +663,14 @@ class Runtime:
 
         return (environment_volume(self.provider),)
 
+    def _project_survives(self) -> bool:
+        """Whether the directory the sync writes lives on the disk the workspace root is on."""
+        root = (self.provider.env_root or "").rstrip("/")
+        if not root:
+            return True
+        workspace = (self.workspace or self.provider.workspace_root or "").rstrip("/")
+        return bool(workspace) and (root == workspace or root.startswith(workspace + "/"))
+
     def _measured_rate(self) -> float | None:
         """This session's measured upload rate in bytes per second, or None where none was.
 

{letify-2.6.2 → letify-2.6.4}/letify-core/Cargo.lock

@@ -138,7 +138,7 @@ dependencies = [
 
 [[package]]
 name = "letify-agent"
-version = "2.6.2"
+version = "2.6.4"
 dependencies = [
  "letify-wire",
  "libloading",
@@ -146,7 +146,7 @@ dependencies = [
 
 [[package]]
 name = "letify-driver"
-version = "2.6.2"
+version = "2.6.4"
 dependencies = [
  "letify-wire",
 ]
@@ -164,7 +164,7 @@ dependencies = [
 
 [[package]]
 name = "letify-wire"
-version = "2.6.2"
+version = "2.6.4"
 
 [[package]]
 name = "libc"

{letify-2.6.2 → letify-2.6.4}/letify-core/Cargo.toml

@@ -8,7 +8,7 @@ resolver = "3"
 members = ["letify-wire", "letify-driver", "letify-agent", "letify-quic"]
 
 [workspace.package]
-version = "2.6.2"
+version = "2.6.4"
 edition = "2024"
 license = "MIT"
 repository = "https://github.com/DarkPyonix/letify"

{letify-2.6.2 → letify-2.6.4}/letify-ext/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "letify-status",
-  "version": "2.6.2",
+  "version": "2.6.4",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "letify-status",
-      "version": "2.6.2",
+      "version": "2.6.4",
       "license": "SEE LICENSE IN LICENSE",
       "devDependencies": {
         "@types/node": "^18.19.0",

{letify-2.6.2 → letify-2.6.4}/letify-ext/package.json

@@ -2,7 +2,7 @@
   "name": "letify-status",
   "displayName": "letify Status",
   "description": "letify quota and GPU activity in the status bar, read from the letify CLI.",
-  "version": "2.6.2",
+  "version": "2.6.4",
   "publisher": "darkpyonix",
   "license": "SEE LICENSE IN LICENSE",
   "repository": {

{letify-2.6.2 → letify-2.6.4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "letify"
-version = "2.6.2"
+version = "2.6.4"
 description = "Declarations that become infrastructure."
 readme = "README.md"
 requires-python = ">=3.11"