letify 2.6.7__tar.gz → 2.7.0__tar.gz

{letify-2.6.7 → letify-2.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: letify
-Version: 2.6.7
+Version: 2.7.0
 Summary: Declarations that become infrastructure.
 Project-URL: Homepage, https://github.com/DarkPyonix/letify
 Project-URL: Documentation, https://github.com/DarkPyonix/letify/tree/main/docs
@@ -186,7 +186,7 @@ letify login colab colab_pro_plus
 letify login shell lab_a100
 ```
 
-Each command writes two files. The account goes to `~/.letify/config.toml`, which belongs to your machine and never to the repository:
+Each command writes two files. The account goes to `~/.letify/config.toml`, which belongs to your machine and never to the repository. Set `LETIFY_HOME` to an absolute path to keep all of letify's local state in another directory instead of `~/.letify`:
 
 ```toml
 [colab_pro_plus]

{letify-2.6.7 → letify-2.7.0}/README.md

@@ -161,7 +161,7 @@ letify login colab colab_pro_plus
 letify login shell lab_a100
 ```
 
-Each command writes two files. The account goes to `~/.letify/config.toml`, which belongs to your machine and never to the repository:
+Each command writes two files. The account goes to `~/.letify/config.toml`, which belongs to your machine and never to the repository. Set `LETIFY_HOME` to an absolute path to keep all of letify's local state in another directory instead of `~/.letify`:
 
 ```toml
 [colab_pro_plus]

{letify-2.6.7 → letify-2.7.0}/docs/SPEC.md

@@ -1076,15 +1076,29 @@ less time than the build did. The archive is smaller than the directory, so the
 overstates the transfer and errs toward building, which is the cheaper of the two to get wrong. The
 ref records the build seconds beside the archive.
 
-With no measured rate for the link, letify packs only when that directory is at most 1 GiB, and says
-why when it declines. Packing moves the archive as one payload that the worker and the client each
-hold whole, so a tree of several gigabytes is held twice in memory with nothing reporting progress:
-measured on a Modal A10, a 7.2 GB environment sat in that step for over 30 minutes while the session
-billed. Packing unconditionally was right while only an ephemeral provider packed, where the
-alternative was a certain full rebuild; a provider that keeps its workspace root and builds its venv
-elsewhere reaches the same branch, and there the alternative is a sync that may take two minutes.
-Making the pack stream rather than hold the archive whole is the fix that removes this limit, and
-until then the limit is what keeps the step from costing more than it saves.
+With no measured rate for the link, letify packs a directory of any size when the archive streams,
+and says `letify: <runtime>: caching the environment, <size> GB` first. The archive streams when the
+first volume's store keeps blobs as files (the filesystem backend, which the automatic environment
+store uses) and the channel is persistent. Both directions then move the archive in pieces and
+neither side holds it whole:
+
+- Pack: the worker op `pack_blob` writes the same tar.gz as `pack_dir`, with the same member rules,
+  into a staging directory beside the project directory while hashing the bytes it writes. It
+  returns `{"digest", "size", "dir"}`. The client receives the file with the stream a write-back
+  uses, checks the digest as the pieces arrive into a scratch file beside the store, and renames it
+  into the store with `Store.adopt_file`. The staging directory is removed afterwards. The digest
+  is the store's content digest (blake3 truncated to 16 bytes, blake2b without blake3).
+- Restore: a blob of more than one piece (8 MiB) that the store holds as a file is sent with
+  `data_put` pieces into a staging directory beside the destination, then moved into place and
+  unpacked by the worker op `place_blob`. Smaller blobs, and any blob from a store that does not
+  keep files, go as one `put_file` payload.
+
+With no measured rate and a store that does not stream (the GCS backend) or a channel that is not
+persistent, letify packs only when the directory is at most 1 GiB, and says why when it declines,
+because that pack moves the archive as one payload that the worker and the client each hold whole.
+Measured: a 7.2 GB Modal environment sat in that whole-payload pack for over 30 minutes while the
+session billed, and a Kaggle session restoring a 3.0 GB archive held whole took 540 s. A streamed
+transfer ran at 85 to 88 MiB/s. A bucket would need a resumable upload to stream.
 
 A later session restores only when the archive's size over the link measured for that session would
 take less than the recorded build. Otherwise it syncs, and says why:
@@ -1123,7 +1137,7 @@ The follow-up design stores each model shard as one content addressed blob and p
 
 ### Volumes on a persistent runtime <!-- id: persistent-volumes -->
 
-> On a persistent provider the volume directory under the workspace root is the runtime's copy of the volume. A file is sent only when that copy does not already hold its digest, and no environment archive is packed or restored.
+> On a persistent provider the volume directory under the workspace root is the runtime's copy of the volume. A file is sent only when that copy does not already hold its digest, and no environment archive is packed or restored, except on the workspace root as below.
 
 A persistent runtime keeps `<workspace root>/volumes/<volume name>` between sessions, so a later session already holds what an earlier one received. The volume directory holds a manifest, `.letify-manifest.json`, mapping each materialized destination path to the digest, size in bytes and modification time in nanoseconds it had when it was written.
 
@@ -1131,7 +1145,27 @@ Before a blob is written to a destination that is not unpacked, the local proces
 
 A persistent provider builds its environment with `uv sync` in `<workspace root>/project/<env key>` every session, and never packs the project directory into a volume or unpacks an archive from one. A sync over an existing `.venv` checks it and installs nothing, so it is faster than any archive transfer, and the `.venv` is already on the disk the archive would be unpacked to.
 
-That holds only while the project directory is under the workspace root. A provider that keeps its workspace root and still builds the `.venv` elsewhere archives like an ephemeral one, because the directory the sync writes does not survive the session however durable the workspace root is. Modal is that provider: its workspace root is a Modal volume and its `env_root` is the sandbox's own disk, chosen because a cold import from the volume reads thousands of small files over the network. Without this, Modal rebuilt 230 packages on every session, measured at 428 s and 534 s on an A10, and had neither the persistence its workspace root offers nor the archive an ephemeral provider gets. Whether the archive is then worth packing is the measurement Materializing into a runtime already makes.
+That holds only while the project directory is under the workspace root. A provider that keeps its workspace root and builds the `.venv` elsewhere keeps the environment archive on that workspace root, not in a store and never through the client. Modal is that provider: its workspace root is a Modal volume that outlives every sandbox, and its `env_root` is the sandbox's own disk, chosen because a cold import from the volume reads thousands of small files over the network.
+
+The archive is `<workspace root>/env-archives/<env key>-<platform>.tar`, uncompressed. Measured on a Modal T4 with a 5.5 GB project directory of 33 packages including torch:
+
+| Step | Time |
+|---|---|
+| `uv sync` build | 216 s |
+| pack through the client's filesystem store | about 700 s |
+| restore through the client's filesystem store | 727 s |
+| `tar -cf` to the volume, uncompressed | 70.8 s |
+| `tar -xf` from the volume, uncompressed | 31.9 s |
+| `tar.gz` to the volume | 240.1 s, gzip is CPU bound |
+| `tar.gz` from the volume | 39.8 s, 3.0 GB |
+
+The client's inbound link is the weak one, so the store path is slower than the build in both directions. Compression costs more than it saves on the volume, so the archive is not compressed.
+
+After a successful sync and interpreter check, the runtime says `caching the environment on the workspace root, <size>`, runs tar from the project directory's parent into a temporary name beside the destination, and renames it into place, so a concurrent session never reads a half written archive. Nothing is sent to the client, and no measured rate gates the pack, because the transfer stays inside the provider.
+
+At the start of `install_env`, when that archive exists on the runtime, the runtime says `restoring the environment from the workspace root`, extracts it into the project directory's parent, and validates the interpreter with the same check an archive restore uses. The restore refuses a member whose path escapes the destination and allows the absolute symlinks a `.venv` needs. A directory an earlier session left under the same name is removed first. When validation fails, the runtime removes the extracted tree, syncs, and replaces the archive. `Runtime.env_source` is `"workspace"` after this restore. This path is taken before and instead of the store path, and such a provider never uses the client store or the bucket store for its environment.
+
+A session sees the archive only once the volume shows it, which is not at once on Modal: a session started right after the one that packed rebuilt and packed again, and a session started a few minutes later restored and its whole call took 44.2 s, against 106.6 s and 183.8 s for the two that built. The rebuild that follows such a miss is not wasted work so much as repeated work, and it leaves the same archive behind.
 
 An ephemeral provider keeps the behaviour of Materializing into a runtime: every file is sent each session, and the environment archive is packed and restored.
 
@@ -1975,25 +2009,27 @@ The worker's working directory is the root, so a relative path in user code reso
 
 ### Declared filesystem paths <!-- id: declared-paths -->
 
-> letify writes exclusively to declared directories: locally under ~/.letify, and on runtimes within the designated workspace root, with explicit documented exceptions for system SSH and daemon runtime directories.
+> letify writes exclusively to declared directories: locally under `~/.letify` or the directory `LETIFY_HOME` names, and on runtimes within the designated workspace root, with explicit documented exceptions for system SSH and daemon runtime directories.
 
 Every path letify creates or writes is declared here. The system temporary directory (/tmp, /var/tmp, or tempfile.gettempdir()) is never used, locally or on a runtime.
 
 #### Local machine
 
-Local filesystem modifications are consolidated under the single root `~/.letify`. The declared local paths are:
+Local filesystem modifications are consolidated under the single root `~/.letify`. The declared local paths are listed below, written relative to `~/.letify`:
+
+`LETIFY_HOME`, when set and non-empty, replaces `~/.letify` as the root of every local path in the table below, and `~` in it is expanded. A relative value raises `ConfigError` naming the value and saying it must be absolute, because a root that moves with the working directory would split one machine's state across directories. Unset or empty means `~/.letify`. `LETIFY_CONFIG` moves only the config file and wins over `LETIFY_HOME` for that file. The fallbacks that read `~/.cache/letify/...` apply only when `LETIFY_HOME` is unset, because a moved root is a separate letify state and does not read the default one's caches. With both `LETIFY_HOME` and `XDG_RUNTIME_DIR` set, the SSH control directory is `$XDG_RUNTIME_DIR/letify-<h>`, where `<h>` is the first 8 hex characters of the SHA-256 of the resolved root, so two roots on one machine never share multiplexed connections and the socket path stays short. The Local provider workspace follows the root. Messages that tell a person which file to edit name the path in use.
 
 | Path | Purpose | Movable by |
 |---|---|---|
-| `~/.letify/config.toml` | Machine configuration declaring accounts and global providers | `LETIFY_CONFIG` environment variable or project `.letify/config.toml` |
-| `~/.letify/accounts/<alias>/` | Account credentials and secrets (mode 0700 directory, mode 0600 files). Holds secret files only: `token.json`, `modal.toml`, `eci.yaml`, `cookie`, `access_token`, `username`, `notebook_id`, `known_hosts`, `link.json`, `password`. Never contains CLI cache directories or python packages | not movable |
-| `~/.letify/tools/<tool>/<version>/` | Installed standalone binaries (`tailcat`, `eci`). Older versions under `~/.letify/tools/<tool>/` are automatically pruned on install | not movable |
-| `~/.letify/cache/digests.json` | Local project file digest cache. Falls back to reading `~/.cache/letify/digests.json` if it exists from an earlier install | not movable |
-| `~/.letify/cache/storage/<name>` | Filesystem storage backend root for store `<name>`. Falls back to reading `~/.cache/letify/<name>` if it exists | not movable |
-| `~/.letify/cache/tools/<alias>/` | Redirected `HOME` and `USERPROFILE` for provider CLIs (`colab`, `kaggle`). Mode 0700. Keeps CLI temporary files, packages, and `.config/` out of the credential directory. The CLI finds its token via a symbolic link pointing to `~/.letify/accounts/<alias>/token.json` (or copy fallback on systems without symlink support) | not movable |
-| `~/.letify/tmp/` | Temporary directory for ephemeral keys during reverse SSH negotiation and adapter stderr files. Mode 0700 | not movable |
-| `~/.letify/ssh/` | OpenSSH multiplexing control sockets (`ControlPath`). Mode 0700 | `XDG_RUNTIME_DIR` (moved to `$XDG_RUNTIME_DIR/letify`) |
-| `~/.letify/runtime/` | Local provider workspace root for calls and local volumes. Holds `data/blobs`, `data/calls`, `tmp`. Falls back to reading `~/.letify-runtime` if it exists | not movable |
+| `~/.letify/config.toml` | Machine configuration declaring accounts and global providers | `LETIFY_CONFIG` environment variable (the file only), `LETIFY_HOME`, or project `.letify/config.toml` |
+| `~/.letify/accounts/<alias>/` | Account credentials and secrets (mode 0700 directory, mode 0600 files). Holds secret files only: `token.json`, `modal.toml`, `eci.yaml`, `cookie`, `access_token`, `username`, `notebook_id`, `known_hosts`, `link.json`, `password`. Never contains CLI cache directories or python packages | `LETIFY_HOME` |
+| `~/.letify/tools/<tool>/<version>/` | Installed standalone binaries (`tailcat`, `eci`). Older versions under `~/.letify/tools/<tool>/` are automatically pruned on install | `LETIFY_HOME` |
+| `~/.letify/cache/digests.json` | Local project file digest cache. Falls back to reading `~/.cache/letify/digests.json` if it exists from an earlier install | `LETIFY_HOME` |
+| `~/.letify/cache/storage/<name>` | Filesystem storage backend root for store `<name>`. Falls back to reading `~/.cache/letify/<name>` if it exists | `LETIFY_HOME` |
+| `~/.letify/cache/tools/<alias>/` | Redirected `HOME` and `USERPROFILE` for provider CLIs (`colab`, `kaggle`). Mode 0700. Keeps CLI temporary files, packages, and `.config/` out of the credential directory. The CLI finds its token via a symbolic link pointing to `~/.letify/accounts/<alias>/token.json` (or copy fallback on systems without symlink support) | `LETIFY_HOME` |
+| `~/.letify/tmp/` | Temporary directory for ephemeral keys during reverse SSH negotiation and adapter stderr files. Mode 0700 | `LETIFY_HOME` |
+| `~/.letify/ssh/` | OpenSSH multiplexing control sockets (`ControlPath`). Mode 0700 | `XDG_RUNTIME_DIR` (moved to `$XDG_RUNTIME_DIR/letify`), `LETIFY_HOME` (with `XDG_RUNTIME_DIR`: `$XDG_RUNTIME_DIR/letify-<h>`; without it: `$LETIFY_HOME/ssh`) |
+| `~/.letify/runtime/` | Local provider workspace root for calls and local volumes. Holds `data/blobs`, `data/calls`, `tmp`. Falls back to reading `~/.letify-runtime` if it exists | `LETIFY_HOME` |
 
 **Documented local exceptions:**
 

{letify-2.6.7 → letify-2.7.0}/docs/locales/README_ko.md

@@ -155,7 +155,7 @@ letify login colab colab_pro_plus
 letify login shell lab_a100
 ```
 
-명령 하나가 파일 두 개를 씁니다. 계정은 `~/.letify/config.toml`에 들어갑니다. 머신에 속하는 정보이고, 저장소에는 들어가지 않습니다.
+명령 하나가 파일 두 개를 씁니다. 계정은 `~/.letify/config.toml`에 들어갑니다. 머신에 속하는 정보이고, 저장소에는 들어가지 않습니다. letify의 로컬 상태 전체를 `~/.letify` 대신 다른 디렉터리에 두려면 `LETIFY_HOME`을 절대 경로로 설정하세요.
 
 ```toml
 [colab_pro_plus]

{letify-2.6.7 → letify-2.7.0}/letify/__init__.py

@@ -66,7 +66,7 @@ distributed: Final = Host.distributed
 # The enum class stays importable from letify.declare.instance for letify's own use.
 del Host
 
-__version__ = "2.6.7"
+__version__ = "2.7.0"
 
 
 def fetch(tensor):  # type: ignore[no-untyped-def]

{letify-2.6.7 → letify-2.7.0}/letify/config/__init__.py

@@ -16,6 +16,7 @@ from pathlib import Path
 from typing import Any
 
 from ..errors import ConfigError
+from ..paths import config_label, config_path
 from . import inventory, login, writer
 from .schema import RESERVED_ALIASES, Config, ProviderConfig
 from .secrets import CONFIG_DIRECTORY, account_directory, resolve_secret
@@ -49,7 +50,7 @@ def load(path: str | Path | None = None, *, home: bool = True) -> Config:
     which is what tests do to stay isolated from the developer's own accounts.
     """
     config = Config()
-    home_path = Path.home() / CONFIG_DIRECTORY / CONFIG_FILE
+    home_path = config_path()
     project_path = _project_file(path)
 
     home_entries = _read_entries(home_path, config, require_kind=True) if home else {}
@@ -66,14 +67,14 @@ def load(path: str | Path | None = None, *, home: bool = True) -> Config:
         if WORKSPACE_FIELD in body:
             raise ConfigError(
                 f"{project_path}: {alias!r} sets 'workspace', which belongs in the home file "
-                f"~/.letify/config.toml, because where letify may write is a rule of one "
+                f"{config_label()}, because where letify may write is a rule of one "
                 f"machine rather than of the repository"
             )
         base = home_entries.get(alias, {})
         kind = body.get("kind", base.get("kind"))
         if not isinstance(kind, str):
             raise ConfigError(
-                f"{project_path}: {alias!r} names an account that ~/.letify/config.toml does not "
+                f"{project_path}: {alias!r} names an account that {config_label()} does not "
                 f"have. "
                 f"Run 'letify login <kind> {alias}' to declare it on this machine, or give "
                 f"the table a 'kind' to declare it here."

{letify-2.6.7 → letify-2.7.0}/letify/config/login.py

@@ -995,7 +995,9 @@ def forget_secret(alias: str) -> bool:
 
 
 def home_path() -> Path:
-    return Path.home() / CONFIG_DIRECTORY / CONFIG_FILE
+    from ..paths import config_path
+
+    return config_path()
 
 
 def project_path(path: str | Path | None = None) -> Path:

{letify-2.6.7 → letify-2.7.0}/letify/config/secrets.py

@@ -28,7 +28,9 @@ ACCOUNTS_DIRECTORY = "accounts"
 
 def account_directory(alias: str) -> Path:
     """``~/.letify/accounts/<alias>/``, whether or not it exists yet."""
-    return Path.home() / CONFIG_DIRECTORY / ACCOUNTS_DIRECTORY / alias
+    from ..paths import account_directory as declared
+
+    return declared(alias)
 
 
 def resolve_secret(

{letify-2.6.7 → letify-2.7.0}/letify/install.py

@@ -140,7 +140,9 @@ def _executable(tool: str) -> str:
 
 
 def tools_home() -> Path:
-    return Path.home() / ".letify" / "tools"
+    from .paths import tools_directory
+
+    return tools_directory()
 
 
 def cache_path(tool: str) -> Path:
@@ -165,7 +167,9 @@ def auto_install_enabled() -> bool:
     value = os.environ.get("LETIFY_AUTO_INSTALL")
     if value is not None:
         return value.strip().lower() not in ("0", "false", "no")
-    home_file = Path.home() / ".letify" / "config.toml"
+    from .paths import config_path
+
+    home_file = config_path()
     try:
         settings = tomllib.loads(home_file.read_text(encoding="utf-8"))
     except (OSError, tomllib.TOMLDecodeError):

{letify-2.6.7 → letify-2.7.0}/letify/launcher.py

@@ -42,6 +42,7 @@ from .declare.env import Env
 from .declare.function import Function
 from .declare.instance import AnyInstance, Host, Instance, RemoteOnlyInstance
 from .errors import LetifyError, UnknownInstance, UnknownProvider
+from .paths import config_label
 from .runtime.pool import RuntimePool
 
 if TYPE_CHECKING:
@@ -168,7 +169,7 @@ class Launcher:
             known = ", ".join(self.config.order) or "none"
             raise UnknownProvider(
                 f"no provider is declared under {alias!r}. Declared: {known}. "
-                f"Name it in .letify/config.toml, and declare the account in ~/.letify/config.toml."
+                f"Name it in .letify/config.toml, and declare the account in {config_label()}."
             )
         provider = provider_registry.build(entry)
         # Connection decisions are printed only when the session start line is.

{letify-2.6.7 → letify-2.7.0}/letify/paths.py

@@ -1,17 +1,20 @@
 """Declared filesystem paths for letify.
 
 Spec section "Declared filesystem paths":
-letify writes exclusively to declared directories: locally under ~/.letify, and on
-runtimes within the designated workspace root, with explicit documented exceptions
-for system SSH and daemon runtime directories. The system temporary directory (/tmp)
-is never used.
+letify writes exclusively to declared directories: locally under ~/.letify (or the
+directory LETIFY_HOME names), and on runtimes within the designated workspace root, with
+explicit documented exceptions for system SSH and daemon runtime directories. The system
+temporary directory (/tmp) is never used.
 """
 
 from __future__ import annotations
 
+import hashlib
 import os
 from pathlib import Path
 
+from .errors import ConfigError
+
 #: Root directory for letify state, tools and caches on the local machine.
 CONFIG_DIRECTORY = ".letify"
 
@@ -80,16 +83,58 @@ DECLARED_REMOTE_EXCEPTIONS = frozenset(
 )
 
 
+def home_is_moved() -> bool:
+    """Whether ``LETIFY_HOME`` is set and non-empty."""
+    return bool(os.environ.get("LETIFY_HOME"))
+
+
 def letify_root() -> Path:
-    """The local state root directory: ~/.letify."""
-    return Path.home() / CONFIG_DIRECTORY
+    """The local state root directory: ``LETIFY_HOME`` when set, else ~/.letify.
+
+    ``~`` in ``LETIFY_HOME`` is expanded. A relative value raises ``ConfigError``, because a
+    root that moves with the working directory would split one machine's state.
+    """
+    value = os.environ.get("LETIFY_HOME")
+    if not value:
+        return Path.home() / CONFIG_DIRECTORY
+    root = Path(value).expanduser()
+    if not root.is_absolute():
+        raise ConfigError(f"LETIFY_HOME={value!r} must be an absolute path")
+    return root
 
 
 def config_path() -> Path:
-    """The local machine configuration file: ~/.letify/config.toml."""
+    """The local machine configuration file: ~/.letify/config.toml.
+
+    ``LETIFY_CONFIG`` names another file and wins over ``LETIFY_HOME`` for this file only.
+    """
+    named = os.environ.get("LETIFY_CONFIG")
+    if named:
+        return Path(named).expanduser()
     return letify_root() / "config.toml"
 
 
+def config_label() -> str:
+    """The machine configuration file as a message names it: the path in use."""
+    if os.environ.get("LETIFY_CONFIG") or home_is_moved():
+        return str(config_path())
+    return "~/.letify/config.toml"
+
+
+def tools_label() -> str:
+    """The installed tools directory as a message names it: the path in use."""
+    if home_is_moved():
+        return str(tools_directory())
+    return "~/.letify/tools"
+
+
+def accounts_label() -> str:
+    """The account credentials directory as a message names it: the path in use."""
+    if home_is_moved():
+        return str(accounts_directory())
+    return "~/.letify/accounts"
+
+
 def accounts_directory() -> Path:
     """The account credentials directory: ~/.letify/accounts."""
     return letify_root() / "accounts"
@@ -114,9 +159,11 @@ def digest_cache_path() -> Path:
     """The path to the client digest cache file.
 
     Returns ~/.letify/cache/digests.json, falling back to reading
-    ~/.cache/letify/digests.json if it exists from an earlier install.
+    ~/.cache/letify/digests.json if it exists from an earlier install, unless LETIFY_HOME is set.
     """
     new_path = cache_directory() / "digests.json"
+    if home_is_moved():
+        return new_path
     old_path = Path.home() / ".cache" / "letify" / "digests.json"
     if not new_path.exists() and old_path.exists():
         return old_path
@@ -130,7 +177,7 @@ def storage_cache_directory(name: str) -> Path:
     if it pre-exists from an earlier install.
     """
     old_path = Path.home() / ".cache" / "letify" / name
-    if old_path.exists():
+    if not home_is_moved() and old_path.exists():
         return old_path
     return cache_directory() / "storage" / name
 
@@ -148,17 +195,24 @@ def local_tmp_directory() -> Path:
 def ssh_control_directory() -> Path:
     """The directory for OpenSSH multiplexing control sockets.
 
-    Returns $XDG_RUNTIME_DIR/letify if set, otherwise ~/.letify/ssh.
+    Returns $XDG_RUNTIME_DIR/letify if set, otherwise ~/.letify/ssh. With LETIFY_HOME set
+    the directory is $XDG_RUNTIME_DIR/letify-<8 hex of the root's SHA-256>, so two roots
+    never share connections and the socket path stays short.
     The system temporary directory (/tmp) is never used.
     """
     runtime = os.environ.get("XDG_RUNTIME_DIR")
     if runtime:
+        if home_is_moved():
+            digest = hashlib.sha256(str(letify_root().resolve()).encode("utf-8")).hexdigest()
+            return Path(runtime) / f"letify-{digest[:8]}"
         return Path(runtime) / "letify"
     return letify_root() / "ssh"
 
 
 def local_runtime_root() -> str:
-    """The local provider workspace root string: ~/.letify/runtime."""
+    """The local provider workspace root string: ~/.letify/runtime, or under LETIFY_HOME."""
+    if home_is_moved():
+        return str(local_runtime_directory())
     return "~/.letify/runtime"
 
 

{letify-2.6.7 → letify-2.7.0}/letify/protocol/worker.py

@@ -2263,6 +2263,81 @@ def _op_pack_dir(request):
     }
 
 
+class _HashedFile:
+    """A write-only file that hashes every byte it passes on, so no second read is needed."""
+
+    def __init__(self, handle, hasher):
+        self._handle = handle
+        self._hasher = hasher
+        self.size = 0
+
+    def write(self, data):
+        self._handle.write(data)
+        self._hasher.update(data)
+        self.size += len(data)
+        return len(data)
+
+    def flush(self):
+        self._handle.flush()
+
+    def tell(self):
+        return self.size
+
+
+def _op_pack_blob(request):
+    """Pack a directory into the blob cache as one committed file, hashing as it writes.
+
+    Same archive and member rules as ``pack_dir``, but the archive never exists as one
+    bytes object: it is written to a partial file in pieces, and the caller reads it back
+    with ``data_stream``. The reply names the directory it used, with ``~`` expanded.
+    """
+    root = os.path.expanduser(request["path"])
+    directory = os.path.expanduser(request["dir"])
+    os.makedirs(directory, exist_ok=True)
+    partial = os.path.join(directory, ".pack.%d.partial" % os.getpid())
+    hasher = _data_hasher()
+    if hasher is None:
+        hasher = hashlib.blake2b(digest_size=16)
+    try:
+        with open(partial, "wb") as handle:
+            hashed = _HashedFile(handle, hasher)
+            with tarfile.open(fileobj=hashed, mode="w:gz") as archive:
+                archive.add(root, arcname=os.path.basename(root.rstrip("/")), recursive=True)
+        digest = hasher.hexdigest() if hasher.name == "blake2b" else hasher.hexdigest(length=16)
+        final = _data_file(directory, digest)
+        os.makedirs(os.path.dirname(final), exist_ok=True)
+        os.replace(partial, final)
+    finally:
+        if os.path.exists(partial):
+            os.remove(partial)
+    return {"ok": True, "value": {"digest": digest, "size": hashed.size, "dir": directory}}
+
+
+def _op_place_blob(request):
+    """Move a received file blob to its destination path, and unpack it when asked.
+
+    The sender wrote the blob into a staging directory with ``data_put``. The staging
+    directory is removed once the file has moved.
+    """
+    import shutil
+
+    staging = request["dir"]
+    source = _data_file(staging, request["digest"])
+    path = request["path"]
+    directory = os.path.dirname(path)
+    if directory:
+        os.makedirs(directory, exist_ok=True)
+    os.replace(source, path)
+    try:
+        os.chmod(path, 0o644)
+    except OSError:
+        pass
+    shutil.rmtree(staging, ignore_errors=True)
+    if request.get("unpack"):
+        _unpack(path, request.get("target") or os.path.dirname(path), request.get("links"))
+    return {"ok": True, "value": {"path": path, "size": os.path.getsize(path)}}
+
+
 def _op_exec(request):
     """Run plain source inside the worker, sharing its globals."""
     exec(compile(request["source"], "<letify>", "exec"), globals())
@@ -2380,6 +2455,8 @@ _OPS = {
     "pull": _op_pull,
     "get_file": _op_get_file,
     "pack_dir": _op_pack_dir,
+    "pack_blob": _op_pack_blob,
+    "place_blob": _op_place_blob,
     "exec": _op_exec,
     "eval": _op_eval,
     "release": _op_release,

{letify-2.6.7 → letify-2.7.0}/letify/providers/elice.py

@@ -43,6 +43,7 @@ from ..errors import (
     SpotPreempted,
     UnsupportedMode,
 )
+from ..paths import accounts_label, tools_label
 from .naming import normalize_gpu
 from .shell import Shell
 from .usage import Usage
@@ -205,7 +206,7 @@ def install_question() -> str:
 
     return (
         "eci, Elice's command line, is not installed. letify can download eci "
-        f"{install.ECI_VERSION} from Elice's GitHub release into ~/.letify/tools; it is "
+        f"{install.ECI_VERSION} from Elice's GitHub release into {tools_label()}; it is "
         "Elice's software, not part of letify. Install it now? [y/N]: "
     )
 
@@ -426,7 +427,7 @@ class Elice(Shell):
             raise ProviderUnavailable(
                 self.kind,
                 f"{self.alias} needs an access token. Run 'letify login elice {self.alias}', "
-                f"which keeps it in ~/.letify/accounts/{self.alias}/, or set access_token_env",
+                f"which keeps it in {accounts_label()}/{self.alias}/, or set access_token_env",
             )
         return token
 

{letify-2.6.7 → letify-2.7.0}/letify/remoting/device/value.py

@@ -14,7 +14,6 @@ import math
 import os
 import tomllib
 from collections.abc import Callable
-from pathlib import Path
 from typing import Any
 
 #: Attributes answered without resolving. Everything else waits for the value first.
@@ -33,7 +32,9 @@ def auto_fetch_enabled(configured: bool | None) -> bool:
 
 def _home_setting() -> bool:
     """``auto_fetch`` at the top level of ``~/.letify/config.toml``, on when absent."""
-    home_file = Path.home() / ".letify" / "config.toml"
+    from ...paths import config_path
+
+    home_file = config_path()
     try:
         settings = tomllib.loads(home_file.read_text(encoding="utf-8"))
     except (OSError, tomllib.TOMLDecodeError):

{letify-2.6.7 → letify-2.7.0}/letify/runtime/bootstrap.py

@@ -275,6 +275,69 @@ def venv_check_source(python: str, archive: str | None = None) -> str:
     return "\n".join(lines) + "\n"
 
 
+def workspace_pack_source(root: str, archive: str) -> str:
+    """Source that packs a project directory into an uncompressed tar beside its destination.
+
+    Spec "Volumes on a persistent runtime": runs ``tar`` from the directory's parent into a
+    temporary name and renames it into place, so a concurrent session never reads a half
+    written archive. Uncompressed, because gzip costs more than it saves on a local volume.
+    """
+    return (
+        "import os, subprocess\n"
+        f"_root = os.path.expanduser({root!r}).rstrip('/')\n"
+        f"_archive = os.path.expanduser({archive!r})\n"
+        "os.makedirs(os.path.dirname(_archive), exist_ok=True)\n"
+        "_partial = _archive + '.' + str(os.getpid()) + '.partial'\n"
+        "try:\n"
+        "    subprocess.run(['tar', '-cf', _partial, '-C', os.path.dirname(_root),\n"
+        "                    os.path.basename(_root)], check=True, capture_output=True)\n"
+        "    os.replace(_partial, _archive)\n"
+        "finally:\n"
+        "    if os.path.exists(_partial):\n"
+        "        os.remove(_partial)\n"
+        "__letify_value__ = os.path.getsize(_archive)\n"
+    )
+
+
+def workspace_restore_source(archive: str, target: str) -> str:
+    """Source that extracts a workspace archive into ``target``, or answers False without one.
+
+    Spec "Volumes on a persistent runtime": a member whose path escapes ``target`` is
+    refused, and the absolute symlinks a ``.venv`` needs are allowed, as the worker's
+    environment unpacking does with ``links=True``.
+    """
+    return (
+        "import os, shutil, tarfile\n"
+        f"_archive = os.path.expanduser({archive!r})\n"
+        f"_target = os.path.expanduser({target!r})\n"
+        "if not os.path.isfile(_archive):\n"
+        "    __letify_value__ = False\n"
+        "else:\n"
+        "    os.makedirs(_target, exist_ok=True)\n"
+        "    _real = os.path.abspath(_target)\n"
+        "    with tarfile.open(_archive, 'r:') as _packed:\n"
+        "        for _member in _packed.getmembers():\n"
+        "            # Lexical, because a tree from an earlier session already holds symlinks\n"
+        "            # that realpath would follow out of the destination.\n"
+        "            _where = os.path.normpath(os.path.join(_real, _member.name))\n"
+        "            if os.path.commonpath([_real, _where]) != _real:\n"
+        "                raise ValueError('archive member would escape the destination')\n"
+        "        # A tree left by an earlier session holds symlinks that extraction would\n"
+        "        # follow, so the top level names the archive holds are replaced whole.\n"
+        "        for _top in {_m.name.split('/')[0] for _m in _packed.getmembers()}:\n"
+        "            _old = os.path.join(_real, _top)\n"
+        "            if os.path.islink(_old) or os.path.isfile(_old):\n"
+        "                os.remove(_old)\n"
+        "            elif os.path.isdir(_old):\n"
+        "                shutil.rmtree(_old)\n"
+        "        try:\n"
+        "            _packed.extractall(_target, filter='tar')\n"
+        "        except TypeError:\n"
+        "            _packed.extractall(_target)\n"
+        "    __letify_value__ = True\n"
+    )
+
+
 def tree_size_source(root: str) -> str:
     """Source that sums the byte size of every regular file under a directory.
 
@@ -455,4 +518,6 @@ __all__ = [
     "uv_bin_dir",
     "uv_env",
     "venv_check_source",
+    "workspace_pack_source",
+    "workspace_restore_source",
 ]