gate-cache 0.3.0__tar.gz → 0.4.0__tar.gz

{gate_cache-0.3.0 → gate_cache-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: gate-cache
-Version: 0.3.0
+Version: 0.4.0
 Summary: Skip lint and test gates that already passed on the exact same git tree.
 Author: Nate Landau
 Author-email: Nate Landau <github@natenate.org>
@@ -31,6 +31,7 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Requires-Dist: cyclopts>=5.2.0,<6.0.0
 Requires-Dist: nclutils>=3.4.4,<4.0.0
+Requires-Dist: platformdirs>=4.13.0,<5.0.0
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 
@@ -40,7 +41,7 @@ Skip a command that already passed on the same code.
 
 A lint or test command often runs many times on code that did not change. It runs when you finish the work, in a pre-commit hook, after a rebase that changed nothing, and in a pre-push hook. When an agent works in the repository, it runs the same command again. If the command takes minutes, the repeat runs take most of the time between finished code and a pushed branch.
 
-gate-cache remembers which commands passed on which code. It wraps a command and gives it a name. When the command passes, gate-cache writes a marker for that name and for the exact content of the tracked files. The next time the same name runs on the same content, gate-cache finds the marker, skips the command, and exits 0 at once.
+gate-cache remembers which commands passed on which code. It wraps a command and gives it a name. When the command passes, gate-cache writes a marker for that name and for the exact content of the files in the working copy that git does not ignore. The next time the same name runs on the same content, gate-cache finds the marker, skips the command, and exits 0 at once.
 
 ```console
 $ gate-cache --name pytest -- uv run pytest
@@ -50,7 +51,7 @@ $ gate-cache --name pytest -- uv run pytest
 gate-cache: pytest already passed on tree aaff74984ccc, skipping
 ```
 
-A change to a tracked file changes the content, so no marker matches and the command runs again. A CI runner that starts from a fresh clone has no markers, so it runs every command.
+A change to any file that git does not ignore changes the content, so no marker matches and the command runs again. A CI runner that starts from a fresh clone has no markers, so it runs every command.
 
 ## Install
 
@@ -75,7 +76,7 @@ Each name has its own markers. A command run under one name never skips because
 
 gate-cache exits with the exit code of the command, or with 0 when it skips the command. It writes a marker only when the command passes, so a command that failed runs again the next time.
 
-When the working copy has uncommitted edits or files that git does not track, the content on disk is not a commit-ready snapshot. gate-cache then runs the command and writes no marker.
+The tree includes uncommitted edits and untracked files that git does not ignore. A gate that passes on uncommitted work therefore skips in the commit hook when you commit the same content. gate-cache computes the tree from a private copy of the index, so it never stages files or changes your index.
 
 ## Run it as a hook
 
@@ -119,20 +120,33 @@ After `just test` passes, the next `just test` on the same content skips pytest.
 
 ### Recipes with arguments
 
-gate-cache identifies a command by its name alone, not by its arguments. If `just test -k unit` and `just test` ran under the same name, a pass of the unit tests would also skip the full suite. Giving each set of arguments its own name, or caching only the bare form, keeps the two apart.
-
-This recipe caches `just test` and runs `just test -k unit` without the cache:
+gate-cache identifies a command by its name alone, not by its arguments. If `just test -k unit` and `just test` ran under the same name, a pass of the unit tests would also skip the full suite. To keep them apart, put the arguments in the name:
 
 ```just
 test *args:
-    #!/usr/bin/env bash
-    set -euo pipefail
-    if [[ -z "{{ args }}" ]]; then
-        exec gate-cache --name test -- uv run pytest
-    fi
-    exec uv run pytest {{ args }}
+    gate-cache --name "{{ trim('test ' + args) }}" -- uv run pytest {{ args }}
 ```
 
+`just test` runs under the name `test`, and `just test -k unit` runs under the name `test -k unit`. Each set of arguments has its own markers. `trim` removes the trailing space when there are no arguments, so `gate-cache status --name test` finds the bare form.
+
+### Inputs that git does not track
+
+A marker does not cover ignored files, environment variables, or tool versions. To make a gate depend on one of these inputs, put its value in the name. When the value changes, the name changes, so no marker matches and the command runs again:
+
+```just
+# Run again when the uv version changes
+test:
+    gate-cache --name "test-uv$(uv --version | cut -d' ' -f2)" -- uv run pytest
+
+# Run again when the ignored .env file changes
+integration:
+    gate-cache --name "integration-$(git hash-object .env)" -- uv run pytest tests/integration
+```
+
+`git hash-object` prints a hash of the file, so the name does not hold the secrets in `.env`. Do not put a secret value in a name, because the marker and `gate-cache status` record the name as plain text.
+
+The shell of the recipe expands `$(...)` before gate-cache starts. A hook `entry` or `args` list does not go through a shell, so put the command in a recipe and call the recipe from the hook. To look up the gate with `gate-cache status --name`, give the expanded name, such as `test-uv0.13.0`.
+
 ### Makefile
 
 ```make
@@ -205,7 +219,7 @@ $ gate-cache status --name ruff
 
 The `last run` line shows only when the name ran on this content without a pass.
 
-`status` runs nothing. With `--name`, it exits 0 when a marker matches and 1 when none does. Without `--name`, it exits 0 when at least one name passed. When gate-cache cannot match the content, for example because of uncommitted edits, the `tree` line gives the reason.
+`status` runs nothing. With `--name`, it exits 0 when a marker matches and 1 when none does. Without `--name`, it exits 0 when at least one name passed. When gate-cache cannot match the content, for example during a merge conflict, the `tree` line gives the reason.
 
 With `--json`, `status` writes one JSON object to stdout instead of text. The exit codes do not change:
 
@@ -222,11 +236,11 @@ $ gate-cache status --json
 }
 ```
 
-`reason` tells why gate-cache cannot match the content, and is `null` when it can. When caching is off, `enabled` is `false` and `reason` names the variable. With `--name`, the object has the `name`, `marker`, `result`, and `last_run` keys in place of `passed` and `missed`.
+`reason` tells why gate-cache cannot match the content, and is `null` when it can. When caching is off, `enabled` is `false`, `reason` names the variable, and the other keys are empty or `null`. With `--name`, the object has the `name`, `marker`, `result`, and `last_run` keys in place of `passed` and `missed`.
 
 ## Start fresh
 
-A marker records only the content of the tracked files. A new tool version, a changed environment variable, or a change to a service does not change that content, so the old markers still match.
+A marker records only the content of the files that git does not ignore. A new tool version, a changed environment variable, or a change to a service does not change that content, so the old markers still match. To make a gate run again when one of these inputs changes, put the input in the name, as [Inputs that git does not track](#inputs-that-git-does-not-track) shows.
 
 `gate-cache clear` deletes every marker for the repository, and the records of missed runs:
 

{gate_cache-0.3.0 → gate_cache-0.4.0}/README.md

@@ -4,7 +4,7 @@ Skip a command that already passed on the same code.
 
 A lint or test command often runs many times on code that did not change. It runs when you finish the work, in a pre-commit hook, after a rebase that changed nothing, and in a pre-push hook. When an agent works in the repository, it runs the same command again. If the command takes minutes, the repeat runs take most of the time between finished code and a pushed branch.
 
-gate-cache remembers which commands passed on which code. It wraps a command and gives it a name. When the command passes, gate-cache writes a marker for that name and for the exact content of the tracked files. The next time the same name runs on the same content, gate-cache finds the marker, skips the command, and exits 0 at once.
+gate-cache remembers which commands passed on which code. It wraps a command and gives it a name. When the command passes, gate-cache writes a marker for that name and for the exact content of the files in the working copy that git does not ignore. The next time the same name runs on the same content, gate-cache finds the marker, skips the command, and exits 0 at once.
 
 ```console
 $ gate-cache --name pytest -- uv run pytest
@@ -14,7 +14,7 @@ $ gate-cache --name pytest -- uv run pytest
 gate-cache: pytest already passed on tree aaff74984ccc, skipping
 ```
 
-A change to a tracked file changes the content, so no marker matches and the command runs again. A CI runner that starts from a fresh clone has no markers, so it runs every command.
+A change to any file that git does not ignore changes the content, so no marker matches and the command runs again. A CI runner that starts from a fresh clone has no markers, so it runs every command.
 
 ## Install
 
@@ -39,7 +39,7 @@ Each name has its own markers. A command run under one name never skips because
 
 gate-cache exits with the exit code of the command, or with 0 when it skips the command. It writes a marker only when the command passes, so a command that failed runs again the next time.
 
-When the working copy has uncommitted edits or files that git does not track, the content on disk is not a commit-ready snapshot. gate-cache then runs the command and writes no marker.
+The tree includes uncommitted edits and untracked files that git does not ignore. A gate that passes on uncommitted work therefore skips in the commit hook when you commit the same content. gate-cache computes the tree from a private copy of the index, so it never stages files or changes your index.
 
 ## Run it as a hook
 
@@ -83,20 +83,33 @@ After `just test` passes, the next `just test` on the same content skips pytest.
 
 ### Recipes with arguments
 
-gate-cache identifies a command by its name alone, not by its arguments. If `just test -k unit` and `just test` ran under the same name, a pass of the unit tests would also skip the full suite. Giving each set of arguments its own name, or caching only the bare form, keeps the two apart.
-
-This recipe caches `just test` and runs `just test -k unit` without the cache:
+gate-cache identifies a command by its name alone, not by its arguments. If `just test -k unit` and `just test` ran under the same name, a pass of the unit tests would also skip the full suite. To keep them apart, put the arguments in the name:
 
 ```just
 test *args:
-    #!/usr/bin/env bash
-    set -euo pipefail
-    if [[ -z "{{ args }}" ]]; then
-        exec gate-cache --name test -- uv run pytest
-    fi
-    exec uv run pytest {{ args }}
+    gate-cache --name "{{ trim('test ' + args) }}" -- uv run pytest {{ args }}
 ```
 
+`just test` runs under the name `test`, and `just test -k unit` runs under the name `test -k unit`. Each set of arguments has its own markers. `trim` removes the trailing space when there are no arguments, so `gate-cache status --name test` finds the bare form.
+
+### Inputs that git does not track
+
+A marker does not cover ignored files, environment variables, or tool versions. To make a gate depend on one of these inputs, put its value in the name. When the value changes, the name changes, so no marker matches and the command runs again:
+
+```just
+# Run again when the uv version changes
+test:
+    gate-cache --name "test-uv$(uv --version | cut -d' ' -f2)" -- uv run pytest
+
+# Run again when the ignored .env file changes
+integration:
+    gate-cache --name "integration-$(git hash-object .env)" -- uv run pytest tests/integration
+```
+
+`git hash-object` prints a hash of the file, so the name does not hold the secrets in `.env`. Do not put a secret value in a name, because the marker and `gate-cache status` record the name as plain text.
+
+The shell of the recipe expands `$(...)` before gate-cache starts. A hook `entry` or `args` list does not go through a shell, so put the command in a recipe and call the recipe from the hook. To look up the gate with `gate-cache status --name`, give the expanded name, such as `test-uv0.13.0`.
+
 ### Makefile
 
 ```make
@@ -169,7 +182,7 @@ $ gate-cache status --name ruff
 
 The `last run` line shows only when the name ran on this content without a pass.
 
-`status` runs nothing. With `--name`, it exits 0 when a marker matches and 1 when none does. Without `--name`, it exits 0 when at least one name passed. When gate-cache cannot match the content, for example because of uncommitted edits, the `tree` line gives the reason.
+`status` runs nothing. With `--name`, it exits 0 when a marker matches and 1 when none does. Without `--name`, it exits 0 when at least one name passed. When gate-cache cannot match the content, for example during a merge conflict, the `tree` line gives the reason.
 
 With `--json`, `status` writes one JSON object to stdout instead of text. The exit codes do not change:
 
@@ -186,11 +199,11 @@ $ gate-cache status --json
 }
 ```
 
-`reason` tells why gate-cache cannot match the content, and is `null` when it can. When caching is off, `enabled` is `false` and `reason` names the variable. With `--name`, the object has the `name`, `marker`, `result`, and `last_run` keys in place of `passed` and `missed`.
+`reason` tells why gate-cache cannot match the content, and is `null` when it can. When caching is off, `enabled` is `false`, `reason` names the variable, and the other keys are empty or `null`. With `--name`, the object has the `name`, `marker`, `result`, and `last_run` keys in place of `passed` and `missed`.
 
 ## Start fresh
 
-A marker records only the content of the tracked files. A new tool version, a changed environment variable, or a change to a service does not change that content, so the old markers still match.
+A marker records only the content of the files that git does not ignore. A new tool version, a changed environment variable, or a change to a service does not change that content, so the old markers still match. To make a gate run again when one of these inputs changes, put the input in the name, as [Inputs that git does not track](#inputs-that-git-does-not-track) shows.
 
 `gate-cache clear` deletes every marker for the repository, and the records of missed runs:
 

{gate_cache-0.3.0 → gate_cache-0.4.0}/pyproject.toml

@@ -8,12 +8,13 @@ classifiers = [
 dependencies = [
     "cyclopts>=5.2.0,<6.0.0",
     "nclutils>=3.4.4,<4.0.0",
+    "platformdirs>=4.13.0,<5.0.0",
 ]
 description = "Skip lint and test gates that already passed on the exact same git tree."
 name = "gate-cache"
 readme = "README.md"
 requires-python = ">=3.11"
-version = "0.3.0"
+version = "0.4.0"
 
 [[project.authors]]
 name = "Nate Landau"

{gate_cache-0.3.0 → gate_cache-0.4.0}/pyproject.toml.orig

@@ -6,13 +6,17 @@
         "Programming Language :: Python :: 3.13",
         "Programming Language :: Python :: 3.14",
     ]
-    dependencies = ["cyclopts>=5.2.0,<6.0.0", "nclutils>=3.4.4,<4.0.0"]
+    dependencies = [
+        "cyclopts>=5.2.0,<6.0.0",
+        "nclutils>=3.4.4,<4.0.0",
+        "platformdirs>=4.13.0,<5.0.0",
+    ]
     description = "Skip lint and test gates that already passed on the exact same git tree."
     license = { file = "LICENSE" }
     name = "gate-cache"
     readme = "README.md"
     requires-python = ">=3.11"
-    version = "0.3.0"
+    version = "0.4.0"
 
     [project.scripts]
         gate-cache = "gate_cache.cli:main"

{gate_cache-0.3.0 → gate_cache-0.4.0}/src/gate_cache/__init__.py

@@ -1,3 +1,3 @@
 """Skip lint and test gates that already passed on the exact same git tree."""
 
-__version__ = "0.3.0"
+__version__ = "0.4.0"

{gate_cache-0.3.0 → gate_cache-0.4.0}/src/gate_cache/cli.py

@@ -17,14 +17,14 @@ _SUBCOMMANDS = {"status", "clear"}
 _INFO_FLAGS = {"-h", "--help", "--version"}
 
 _HELP = """\
-Skip a lint or test command that already passed on the same git tree. gate-cache runs
-the command, records a pass keyed on the git tree of the tracked files (the index, when
-the working copy matches it), and skips the command next time the tree is identical.
-With unstaged changes or untracked files the command runs without caching and nothing
-is recorded.
+Skip a lint or test command that already passed on the same git tree. gate-cache
+runs the command, records a pass keyed on the git tree of the working copy
+(tracked and untracked, non-ignored files), and skips the command next time the
+tree is identical. A pass on uncommitted work therefore skips when the same
+content is committed.
 
-Both --name and -- are required, and only --name may come before --. Everything after
--- is the command, verbatim.
+Both --name and -- are required, and only --name may come before --. Everything
+after -- is the command, verbatim.
 
 Exit codes:
   run     the command's own exit code, or 0 when a recorded pass lets it skip.
@@ -40,11 +40,16 @@ same way.
 Set GATE_CACHE_MAX_AGE_DAYS to a positive whole number to change how many days a
 recorded pass lasts (default 30).
 
-Only file content is keyed. Never wrap checks that depend on anything else,
-such as commit-message or branch-name checks.
+Only the name and the file content are keyed. To also key on recipe arguments,
+an ignored file, an environment variable, or a tool version, put its value in
+--name. A changed value gives a new name, so the command runs again. Never put a
+secret in a name, because markers and status store it as plain text. Never wrap
+checks that depend on the commit message or the branch name.
 
 Typical use:
   gate-cache --name pytest -- uv run pytest -x
+  gate-cache --name "pytest -k unit" -- uv run pytest -k unit
+  gate-cache --name "test-$(git hash-object .env)" -- uv run pytest
 """
 
 app = App(
@@ -75,7 +80,7 @@ def run(
     On a skip, prints one line to stderr and exits 0 without running anything. Otherwise
     runs the command and exits with its exit code. The command runs without caching
     (and nothing is recorded) when GATE_CACHE_DISABLE or CI turns caching off, the tree
-    has unstaged changes or untracked files, or this is not a git repository. A pass is recorded only
+    is unknown (for example an unmerged index), or this is not a git repository. A pass is recorded only
     when the command exits 0 and the run leaves the tree unchanged.
 
     Parameters
@@ -84,6 +89,7 @@ def run(
         The command and its arguments, given after `--`. Example: -- uv run pytest -x
     name
         Gate name. Records match only when the name and the tree both match.
+        Include arguments or other inputs that the tree does not cover.
     """
     if not command:
         say("no command given after --")

{gate_cache-0.3.0 → gate_cache-0.4.0}/src/gate_cache/gate.py

@@ -12,8 +12,10 @@ from nclutils import pp
 from gate_cache.console import say
 from gate_cache.markers import (
     clear_markers,
+    marker_exists,
     marker_path,
     miss_path,
+    miss_record,
     missed_records,
     passed_names,
     prune_markers,
@@ -95,14 +97,6 @@ def marker_max_age_days(environ: Mapping[str, str] | None = None) -> int:
     return days
 
 
-def _marker_exists(path: Path) -> bool:
-    """Return True only when the marker is known to exist."""
-    try:
-        return path.exists()
-    except OSError:
-        return False
-
-
 def _run_uncached(reason: str, command: Sequence[str]) -> int:
     """Explain why caching is off, then run the command."""
     say(f"{reason}, running without cache")
@@ -170,7 +164,7 @@ def run_gate(name: str, command: Sequence[str]) -> int:
     if common_dir is None:
         return _run_uncached(_NO_COMMON_DIR_REASON, command)
 
-    if _marker_exists(marker_path(common_dir, tree, name)):
+    if marker_exists(marker_path(common_dir, tree, name)):
         say(f"{name} already passed on tree {tree[:12]}, skipping")
         return 0
 
@@ -210,7 +204,12 @@ def gate_status(name: str | None = None, *, as_json: bool = False) -> int:
     reason = disabled_reason()
     if reason is not None:
         if as_json:
-            _print_json({"enabled": False, "reason": reason, "tree": None})
+            empty = (
+                {"name": name, "marker": None, "result": "miss", "last_run": None}
+                if name is not None
+                else {"passed": [], "missed": []}
+            )
+            _print_json({"enabled": False, "tree": None, "reason": reason, **empty})
         else:
             say(reason)
         return 1
@@ -249,15 +248,13 @@ def _named_status(name: str, snapshot: Snapshot, common_dir: Path | None, *, as_
     """Print whether one gate would skip on the snapshot's tree."""
     tree = snapshot.tree
     path: Path | None = None
+    hit = False
     last_run: dict[str, Any] | None = None
     if tree is not None and common_dir is not None:
         path = marker_path(common_dir, tree, name)
-    hit = path is not None and _marker_exists(path)
-    if not hit and tree is not None and common_dir is not None:
-        last_run = next(
-            (_miss_summary(r) for r in missed_records(common_dir, tree) if r["name"] == name),
-            None,
-        )
+        hit = marker_exists(path)
+        record = None if hit else miss_record(common_dir, tree, name)
+        last_run = _miss_summary(record) if record is not None else None
     reason = _unknown_reason(snapshot, common_dir)
 
     if as_json:
@@ -274,12 +271,15 @@ def _named_status(name: str, snapshot: Snapshot, common_dir: Path | None, *, as_
         )
         return 0 if hit else 1
 
-    marker = str(path) if path is not None else "none"
-    if tree is not None and common_dir is None:
-        marker = f"none: {_NO_COMMON_DIR_REASON}"
+    if path is not None:
+        marker = str(path)
+    elif tree is not None:
+        marker = f"none: {reason}"
+    else:
+        marker = "none"
     rows = {
         "name": name,
-        "tree": tree if tree is not None else f"unknown: {snapshot.reason}",
+        "tree": tree if tree is not None else f"unknown: {reason}",
         "marker": marker,
         "result": "hit" if hit else "miss",
     }
@@ -325,7 +325,9 @@ def _list_gates(snapshot: Snapshot, common_dir: Path | None, *, as_json: bool) -
     if passed:
         rows["passed"] = ", ".join(passed)
     if missed:
-        rows["missed"] = ", ".join(f"{m['name']} ({m['reason']})" for m in missed)
+        rows["missed"] = ", ".join(
+            f"{m['name']} ({m['reason']})" if m["reason"] else m["name"] for m in missed
+        )
     pp.kv(rows)
     return code
 

{gate_cache-0.3.0 → gate_cache-0.4.0}/src/gate_cache/markers.py

@@ -162,8 +162,8 @@ def passed_names(common_dir: Path, tree: str) -> list[str]:
 def missed_records(common_dir: Path, tree: str) -> list[dict[str, Any]]:
     """List the gates that ran on a tree without passing, for reporting what would run.
 
-    A gate that also has a pass marker on the tree is left out, since a later pass
-    supersedes the miss.
+    A gate whose marker exists on the tree is left out, since a later pass supersedes the
+    miss. This holds for markers without JSON content too, because those still skip.
 
     Args:
         common_dir: Absolute path to the git common directory.
@@ -172,15 +172,50 @@ def missed_records(common_dir: Path, tree: str) -> list[dict[str, Any]]:
     Returns:
         Each miss record's JSON content, sorted by gate name.
     """
-    passed = set(passed_names(common_dir, tree))
     records = [
         record
         for record in _records(common_dir, tree, suffix=MISS_SUFFIX)
-        if record["name"] not in passed
+        if not marker_exists(marker_path(common_dir, tree, record["name"]))
     ]
     return sorted(records, key=lambda record: record["name"])
 
 
+def miss_record(common_dir: Path, tree: str, name: str) -> dict[str, Any] | None:
+    """Read one gate's miss record on a tree.
+
+    Args:
+        common_dir: Absolute path to the git common directory.
+        tree: Git tree hash.
+        name: Gate name.
+
+    Returns:
+        The record's JSON content, or None when it is missing, unreadable, or does not
+        match the tree and name.
+    """
+    path = miss_path(common_dir, tree, name)
+    data = _read_record(path)
+    if data is None or data.get("tree") != tree or data.get("name") != name:
+        return None
+    return data
+
+
+def marker_exists(path: Path) -> bool:
+    """Return True only when the path is known to exist."""
+    try:
+        return path.exists()
+    except OSError:
+        return False
+
+
+def _read_record(path: Path) -> dict[str, Any] | None:
+    """Parse a JSON object from a record file, or return None when it is not one."""
+    try:
+        data = json.loads(path.read_text(encoding="utf-8"))
+    except (OSError, UnicodeDecodeError, ValueError):
+        return None
+    return data if isinstance(data, dict) else None
+
+
 def _records(common_dir: Path, tree: str, *, suffix: str) -> list[dict[str, Any]]:
     """Read the records for a tree whose filename is the key of their own name plus suffix."""
     try:
@@ -189,13 +224,10 @@ def _records(common_dir: Path, tree: str, *, suffix: str) -> list[dict[str, Any]
         return []
     records: list[dict[str, Any]] = []
     for entry in entries:
-        if entry.name.startswith(TEMP_PREFIX):
-            continue
-        try:
-            data = json.loads(entry.read_text(encoding="utf-8"))
-        except (OSError, UnicodeDecodeError, ValueError):
+        if entry.name.startswith(TEMP_PREFIX) or not entry.name.endswith(suffix):
             continue
-        if not isinstance(data, dict):
+        data = _read_record(entry)
+        if data is None:
             continue
         name = data.get("name")
         if (

gate_cache-0.4.0/src/gate_cache/scratch.py

@@ -0,0 +1,67 @@
+"""Provide private, short-lived directories for gate-cache's own git probes."""
+
+from __future__ import annotations
+
+import contextlib
+import shutil
+import tempfile
+import time
+import warnings
+from contextlib import contextmanager
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+import platformdirs
+
+if TYPE_CHECKING:
+    from collections.abc import Generator
+
+SCRATCH_PREFIX = "snap-"
+STALE_AFTER_SECONDS = 3600
+
+
+def runtime_area() -> Path:
+    """Return the private directory that holds every scratch directory, creating it if needed.
+
+    Honors XDG_RUNTIME_DIR and falls back to the platform's per-user runtime location.
+
+    Returns:
+        The absolute path of the area, with mode 0700.
+    """
+    # An unset XDG_RUNTIME_DIR is normal in containers and SSH sessions, so the fallback
+    # warning would only add noise to gate output.
+    with warnings.catch_warnings():
+        warnings.simplefilter("ignore", platformdirs.RuntimeDirWarning)
+        area = platformdirs.user_runtime_path("gate-cache", appauthor=False)
+    area.mkdir(mode=0o700, parents=True, exist_ok=True)
+    return area
+
+
+def sweep_stale(area: Path, *, now: float | None = None) -> None:
+    """Delete scratch directories that a killed run left behind.
+
+    Only directories older than STALE_AFTER_SECONDS go, so a parallel run's directory survives.
+
+    Args:
+        area: Directory holding scratch directories.
+        now: Current time in seconds since the epoch; defaults to the clock.
+    """
+    cutoff = (time.time() if now is None else now) - STALE_AFTER_SECONDS
+    with contextlib.suppress(OSError):
+        for entry in area.glob(f"{SCRATCH_PREFIX}*"):
+            with contextlib.suppress(OSError):
+                if entry.stat().st_mtime < cutoff:
+                    shutil.rmtree(entry, ignore_errors=True)
+
+
+@contextmanager
+def scratch_dir() -> Generator[Path]:
+    """Yield a private directory that is deleted when the block exits, however it exits.
+
+    Yields:
+        The path of a new, empty directory inside the runtime area.
+    """
+    area = runtime_area()
+    sweep_stale(area)
+    with tempfile.TemporaryDirectory(prefix=SCRATCH_PREFIX, dir=area) as path:
+        yield Path(path)

gate_cache-0.4.0/src/gate_cache/snapshot.py

@@ -0,0 +1,366 @@
+"""Detect the git tree that a gate command is about to test."""
+
+from __future__ import annotations
+
+import os
+import shutil
+from dataclasses import dataclass
+from pathlib import Path
+
+from nclutils.git import run_git
+from nclutils.sh import ShellCommandError, ShellCommandNotFoundError
+
+from gate_cache import scratch
+
+
+@dataclass(frozen=True)
+class Snapshot:
+    """The tree a gate is about to test, or why that tree is unknown.
+
+    Attributes:
+        tree: Git tree hash describing the content under test, or None when unknown.
+        reason: Why the tree is unknown, or None when it is known.
+    """
+
+    tree: str | None
+    reason: str | None
+
+    @property
+    def known(self) -> bool:
+        """Return True when a tree hash identifies the content under test."""
+        return self.tree is not None
+
+
+def _hook_work_tree(cwd: Path | None) -> Path | None:
+    """Return the work tree that a GIT_DIR without GIT_WORK_TREE belongs to, if derivable.
+
+    Git hands hooks GIT_DIR alone, and then takes the current directory for the top of the
+    work tree. A hook that changes into a subdirectory would see every other file as changed.
+    """
+    git_dir = os.environ.get("GIT_DIR")
+    if not git_dir or os.environ.get("GIT_WORK_TREE"):
+        return None
+    if run_git("config", "--get", "core.worktree", cwd=cwd, check=False).stdout.strip():
+        return None
+    base = Path(cwd) if cwd is not None else Path.cwd()
+    path = (base / git_dir).resolve()
+    # A linked worktree's git dir holds a `gitdir` file naming `<work tree>/.git`.
+    gitdir_file = path / "gitdir"
+    if gitdir_file.is_file():
+        target = Path(gitdir_file.read_text(encoding="utf-8").strip())
+        return (path / target).resolve().parent
+    if path.name == ".git":
+        return path.parent
+    return None
+
+
+def _git_env(cwd: Path | None) -> dict[str, str] | None:
+    """Return an environment for gate-cache's own git probes, or None to inherit as is."""
+    try:
+        work_tree = _hook_work_tree(cwd)
+    except OSError:
+        return None
+    if work_tree is None:
+        return None
+    return {**os.environ, "GIT_WORK_TREE": str(work_tree)}
+
+
+def _work_tree_root(cwd: Path | None, env: dict[str, str] | None) -> Path | None:
+    """Return the absolute top of the work tree, or None when there is none."""
+    result = run_git("rev-parse", "--show-toplevel", cwd=cwd, env=env, check=False)
+    output = result.stdout.strip()
+    if result.returncode != 0 or not output or "\n" in output:
+        return None
+    return Path(output)
+
+
+_GITLINK_MODE = "160000"
+
+
+def _hidden_entry_reason(root: Path, env: dict[str, str] | None, label: str = "") -> str | None:
+    """Return a reason when an index entry tells git to ignore edits to its file."""
+    # `ls-files -v` tags assume-unchanged entries in lowercase and skip-worktree entries `S`.
+    # Sparse checkout marks every path outside the cone skip-worktree, so it lands here too.
+    output = run_git("ls-files", "-v", "-z", cwd=root, env=env).stdout
+    for record in output.split("\0"):
+        tag, _, path = record.partition(" ")
+        if path and (tag.islower() or tag == "S"):
+            return f"skip-worktree or assume-unchanged entry: {label}{path}"
+    return None
+
+
+@dataclass(frozen=True)
+class _State:
+    """Where the work tree is, or why no tree can describe it."""
+
+    root: Path | None
+    reason: str | None = None
+
+
+@dataclass(frozen=True)
+class _StatusRecord:
+    """One entry of `git status --porcelain=v2`."""
+
+    kind: str
+    submodule: str
+    path: str
+
+    @property
+    def submodule_moved(self) -> bool:
+        """Return True when a submodule's commit, tracked content, or untracked files differ."""
+        return self.submodule.startswith("S") and any(flag in self.submodule for flag in "CMU")
+
+
+def _status(directory: Path, env: dict[str, str] | None) -> list[_StatusRecord]:
+    """List every difference git status sees, regardless of status and submodule config."""
+    # Optional locks off: status would otherwise refresh and rewrite the index it reads,
+    # taking index.lock from a concurrent git command. `-unormal` and
+    # `--ignore-submodules=none` override config that would hide untracked files and
+    # submodule changes.
+    output = run_git(
+        "--no-optional-locks",
+        "status",
+        "--porcelain=v2",
+        "-z",
+        "--untracked-files=normal",
+        "--ignore-submodules=none",
+        cwd=directory,
+        env=env,
+    ).stdout
+    found: list[_StatusRecord] = []
+    records = iter(output.split("\0"))
+    for record in records:
+        kind = record[:1]
+        if kind in {"1", "2"}:
+            fields = record.split(" ", 9 if kind == "2" else 8)
+            if kind == "2":
+                next(records, None)  # a rename record is followed by its original path
+            found.append(_StatusRecord(kind=kind, submodule=fields[2], path=fields[-1]))
+        elif kind == "u":
+            fields = record.split(" ", 10)
+            found.append(_StatusRecord(kind=kind, submodule=fields[2], path=fields[-1]))
+        elif kind == "?":
+            found.append(_StatusRecord(kind=kind, submodule="N...", path=record[2:]))
+    return found
+
+
+def _gitlinks(directory: Path, env: dict[str, str] | None) -> list[str]:
+    """Return the paths of every gitlink (submodule or embedded repo) in the index."""
+    output = run_git("ls-files", "--stage", "-z", cwd=directory, env=env).stdout
+    links: list[str] = []
+    for record in output.split("\0"):
+        info, _, path = record.partition("\t")
+        if path and info.split(" ", 1)[0] == _GITLINK_MODE:
+            links.append(path)
+    return links
+
+
+def _gitlink_reason(directory: Path, label: str, env: dict[str, str]) -> str | None:
+    """Return why a submodule's checkout may hold content its recorded commit does not.
+
+    The superproject's status runs the submodule's own status, which obeys the submodule's
+    config and index flags, so each checkout is probed directly, recursively.
+    """
+    try:
+        # An uninitialized submodule is an empty directory. `git submodule update --init`
+        # later fills it without changing the tree, so an earlier pass proves nothing.
+        if not (directory.is_dir() and any(directory.iterdir())):
+            return f"submodule is not checked out: {label}"
+        checked_out = (directory / ".git").exists()
+    except OSError:
+        checked_out = False
+    if not checked_out:
+        return f"submodule is not a checked-out repository: {label}"
+    hidden = _hidden_entry_reason(directory, env, label=f"{label}/")
+    if hidden:
+        return hidden
+    if _status(directory, env):
+        return f"submodule not at its index state: {label}"
+    nested = (
+        _gitlink_reason(directory / path, f"{label}/{path}", env)
+        for path in _gitlinks(directory, env)
+    )
+    return next((reason for reason in nested if reason), None)
+
+
+def _inspect(cwd: Path | None, env: dict[str, str] | None) -> _State:
+    """Probe the repository for anything that keeps one tree from describing the working copy."""
+    if run_git("rev-parse", "--git-dir", cwd=cwd, env=env, check=False).returncode != 0:
+        return _State(root=None, reason="not a git repository")
+    # ls-files only lists paths under its cwd. Probing from the top avoids the `:/` pathspec,
+    # which GIT_LITERAL_PATHSPECS turns into a literal path that matches nothing.
+    root = _work_tree_root(cwd, env)
+    if root is None:
+        return _State(root=None, reason="could not locate work tree")
+    if run_git("ls-files", "--unmerged", cwd=root, env=env).stdout:
+        return _State(root=root, reason="unmerged index")
+    return _State(root=root, reason=_hidden_entry_reason(root, env))
+
+
+def _git_path(name: str, cwd: Path | None, env: dict[str, str] | None) -> Path:
+    """Return the absolute path git uses for NAME, honoring GIT_INDEX_FILE and friends."""
+    output = run_git("rev-parse", "--git-path", name, cwd=cwd, env=env).stdout.strip()
+    if not output or "\n" in output:
+        msg = f"unexpected `git rev-parse --git-path {name}` output: {output!r}"
+        raise OSError(msg)
+    path = Path(output)
+    if path.is_absolute():
+        return path
+    return (Path(cwd) if cwd is not None else Path.cwd()) / path
+
+
+def _require_private(env: dict[str, str], private: Path) -> None:
+    """Refuse to run git unless its index and object store sit inside PRIVATE.
+
+    The env is built a few lines earlier, so this never fires today. It stands between any
+    future change to that env and a `git add -A` that would stage into the real index.
+    """
+    for key in ("GIT_INDEX_FILE", "GIT_OBJECT_DIRECTORY"):
+        if not Path(env[key]).resolve().is_relative_to(private.resolve()):
+            msg = f"{key}={env[key]!r} is outside {private}"
+            raise OSError(msg)
+
+
+def _submodule_env(env: dict[str, str]) -> dict[str, str]:
+    """Return ENV without the variables that tie git to the superproject's repository."""
+    local = run_git("rev-parse", "--local-env-vars", env=env).stdout.split()
+    return {key: value for key, value in env.items() if key not in local}
+
+
+def _staged_reason(root: Path, env: dict[str, str]) -> str | None:
+    """Return why the staged private index may not describe what the command reads."""
+    links = _gitlinks(root, env)
+    # Only gitlinks produce submodule records, so a repo without them skips a full status scan.
+    if links:
+        for record in _status(root, env):
+            if record.submodule_moved:
+                return f"submodule not at its index state: {record.path}"
+        submodule_env = _submodule_env(env)
+        for path in links:
+            reason = _gitlink_reason(root / path, path, submodule_env)
+            if reason:
+                return reason
+    ignorecase = run_git("config", "--bool", "core.ignorecase", cwd=root, env=env, check=False)
+    if ignorecase.stdout.strip() == "true":
+        return _case_mismatch_reason(root, env)
+    return None
+
+
+def _case_mismatch_reason(root: Path, env: dict[str, str]) -> str | None:
+    """Return a reason when an indexed path is spelled differently on disk than in the index.
+
+    On a case-insensitive filesystem a case-only rename keeps the old index entry, so the tree
+    would name a path the command does not see. Comparing each path component with the real
+    directory entries is exact and does not depend on ignore rules.
+    """
+    output = run_git("ls-files", "-z", cwd=root, env=env).stdout
+    listings: dict[Path, set[str]] = {}
+    for path in (entry for entry in output.split("\0") if entry):
+        parent = root
+        for part in path.split("/"):
+            if parent not in listings:
+                listings[parent] = {child.name for child in parent.iterdir()}
+            if part not in listings[parent]:
+                return f"path differs from the index only by case: {path}"
+            parent = parent / part
+    return None
+
+
+def _private_snapshot(cwd: Path | None, root: Path, env: dict[str, str] | None) -> Snapshot:
+    """Stage the working copy into a private index and return its tree, writing nothing real.
+
+    A copy of the index and a throwaway object store take every write, so the real index,
+    its lock, and `.git/objects` stay untouched even by write-tree's cache-tree update.
+    """
+    # Resolved from cwd, as git itself resolves an inherited relative GIT_INDEX_FILE.
+    index = _git_path("index", cwd, env)
+    objects = _git_path("objects", cwd, env)
+    base = dict(os.environ if env is None else env)
+    with scratch.scratch_dir() as private:
+        private_index = private / "index"
+        private_objects = private / "objects"
+        # copy2 keeps the mtime git compares against to spot racily clean entries. A fresh
+        # mtime would make git trust stat data for a file edited in the same second.
+        if index.exists():
+            shutil.copy2(index, private_index)
+        private_objects.mkdir()
+        alternates = [str(objects), base.get("GIT_ALTERNATE_OBJECT_DIRECTORIES", "")]
+        private_env = {
+            **base,
+            "GIT_INDEX_FILE": str(private_index),
+            "GIT_OBJECT_DIRECTORY": str(private_objects),
+            "GIT_ALTERNATE_OBJECT_DIRECTORIES": os.pathsep.join(filter(None, alternates)),
+        }
+        _require_private(private_env, private)
+        # Split index mode would write a shared index file into the real git directory.
+        no_split = ("-c", "core.splitIndex=false")
+        added = run_git(*no_split, "add", "-A", cwd=root, env=private_env, check=False)
+        if added.returncode != 0:
+            # Name git's complaint, such as an embedded repository with no commit, so the
+            # user can see what keeps the cache off.
+            lines = [line for line in added.stderr.splitlines() if line.strip()]
+            errors = [line for line in lines if line.startswith(("fatal:", "error:"))]
+            detail = (errors or lines or [None])[0]
+            reason = "could not stage working copy"
+            return Snapshot(tree=None, reason=f"{reason}: {detail}" if detail else reason)
+        reason = _staged_reason(root, private_env)
+        if reason:
+            return Snapshot(tree=None, reason=reason)
+        tree = run_git(*no_split, "write-tree", cwd=root, env=private_env).stdout
+    return Snapshot(tree=tree, reason=None)
+
+
+def take_snapshot(cwd: Path | None = None) -> Snapshot:
+    """Identify the tree a gate command is about to test.
+
+    The tree is the one a commit of the working copy would get, so a pass on uncommitted work
+    matches the later commit. Every situation where the tree might not describe what the
+    command reads, including any git failure, resolves to unknown so the caller runs the
+    command uncached.
+
+    Args:
+        cwd: Directory to run git in. Defaults to the process working directory.
+
+    Returns:
+        A Snapshot holding the tree hash, or the reason it is unknown.
+    """
+    try:
+        env = _git_env(cwd)
+        state = _inspect(cwd, env)
+    except ShellCommandNotFoundError:
+        return Snapshot(tree=None, reason="git not found")
+    except (ShellCommandError, UnicodeDecodeError):
+        return Snapshot(tree=None, reason="could not inspect git state")
+    if state.reason or state.root is None:
+        return Snapshot(tree=None, reason=state.reason or "could not locate work tree")
+
+    try:
+        return _private_snapshot(cwd, state.root, env)
+    except (ShellCommandError, OSError, UnicodeDecodeError):
+        return Snapshot(tree=None, reason="could not hash working copy")
+
+
+def git_common_dir(cwd: Path | None = None) -> Path | None:
+    """Return the absolute git common directory shared by all linked worktrees.
+
+    Args:
+        cwd: Directory to run git in. Defaults to the process working directory.
+
+    Returns:
+        The absolute path, or None when git fails or its output is not a single path.
+    """
+    try:
+        result = run_git("rev-parse", "--git-common-dir", cwd=cwd)
+    except (ShellCommandError, UnicodeDecodeError):
+        return None
+    output = result.stdout.strip()
+    if not output or "\n" in output:
+        return None
+    path = Path(output)
+    if path.is_absolute():
+        return path
+    try:
+        base = Path(cwd) if cwd is not None else Path.cwd()
+        return (base / path).resolve()
+    except (OSError, RuntimeError):
+        return None

gate_cache-0.3.0/src/gate_cache/snapshot.py

@@ -1,148 +0,0 @@
-"""Detect the git tree that a gate command is about to test."""
-
-from __future__ import annotations
-
-import os
-from dataclasses import dataclass
-from pathlib import Path
-
-from nclutils.git import run_git
-from nclutils.sh import ShellCommandError, ShellCommandNotFoundError
-
-
-@dataclass(frozen=True)
-class Snapshot:
-    """The tree a gate is about to test, or why that tree is unknown.
-
-    Attributes:
-        tree: Git tree hash describing the content under test, or None when unknown.
-        reason: Why the tree is unknown, or None when it is known.
-    """
-
-    tree: str | None
-    reason: str | None
-
-    @property
-    def known(self) -> bool:
-        """Return True when a tree hash identifies the content under test."""
-        return self.tree is not None
-
-
-def _hook_work_tree(cwd: Path | None) -> Path | None:
-    """Return the work tree that a GIT_DIR without GIT_WORK_TREE belongs to, if derivable.
-
-    Git hands hooks GIT_DIR alone, and then takes the current directory for the top of the
-    work tree. A hook that changes into a subdirectory would see every other file as changed.
-    """
-    git_dir = os.environ.get("GIT_DIR")
-    if not git_dir or os.environ.get("GIT_WORK_TREE"):
-        return None
-    if run_git("config", "--get", "core.worktree", cwd=cwd, check=False).stdout.strip():
-        return None
-    base = Path(cwd) if cwd is not None else Path.cwd()
-    path = (base / git_dir).resolve()
-    # A linked worktree's git dir holds a `gitdir` file naming `<work tree>/.git`.
-    gitdir_file = path / "gitdir"
-    if gitdir_file.is_file():
-        target = Path(gitdir_file.read_text(encoding="utf-8").strip())
-        return (path / target).resolve().parent
-    if path.name == ".git":
-        return path.parent
-    return None
-
-
-def _git_env(cwd: Path | None) -> dict[str, str] | None:
-    """Return an environment for gate-cache's own git probes, or None to inherit as is."""
-    try:
-        work_tree = _hook_work_tree(cwd)
-    except OSError:
-        return None
-    if work_tree is None:
-        return None
-    return {**os.environ, "GIT_WORK_TREE": str(work_tree)}
-
-
-def _work_tree_root(cwd: Path | None, env: dict[str, str] | None) -> Path | None:
-    """Return the absolute top of the work tree, or None when there is none."""
-    result = run_git("rev-parse", "--show-toplevel", cwd=cwd, env=env, check=False)
-    output = result.stdout.strip()
-    if result.returncode != 0 or not output or "\n" in output:
-        return None
-    return Path(output)
-
-
-def _unknown_reason(cwd: Path | None, env: dict[str, str] | None) -> str | None:
-    """Return why the working copy cannot be described by one tree, or None if it can."""
-    if run_git("rev-parse", "--git-dir", cwd=cwd, env=env, check=False).returncode != 0:
-        return "not a git repository"
-    # ls-files only lists paths under its cwd. Probing from the top avoids the `:/` pathspec,
-    # which GIT_LITERAL_PATHSPECS turns into a literal path that matches nothing.
-    root = _work_tree_root(cwd, env)
-    if root is None:
-        return "could not locate work tree"
-    if run_git("ls-files", "--unmerged", cwd=root, env=env).stdout:
-        return "unmerged index"
-    if run_git("diff", "--quiet", cwd=root, env=env, okay_codes=(0, 1)).returncode != 0:
-        return "unstaged changes to tracked files"
-    untracked = run_git(
-        "ls-files", "--others", "--exclude-standard", "-z", cwd=root, env=env
-    ).stdout
-    if untracked:
-        return f"untracked file: {untracked.split(chr(0))[0]}"
-    return None
-
-
-def take_snapshot(cwd: Path | None = None) -> Snapshot:
-    """Identify the tree a gate command is about to test.
-
-    The snapshot is known only when the working copy matches the index for tracked files and
-    no untracked, non-ignored files exist. Every other situation, including any git failure,
-    resolves to unknown so the caller runs the command uncached.
-
-    Args:
-        cwd: Directory to run git in. Defaults to the process working directory.
-
-    Returns:
-        A Snapshot holding the tree hash, or the reason it is unknown.
-    """
-    try:
-        env = _git_env(cwd)
-        reason = _unknown_reason(cwd, env)
-    except ShellCommandNotFoundError:
-        return Snapshot(tree=None, reason="git not found")
-    except (ShellCommandError, UnicodeDecodeError):
-        return Snapshot(tree=None, reason="could not inspect git state")
-    if reason:
-        return Snapshot(tree=None, reason=reason)
-
-    try:
-        tree = run_git("write-tree", cwd=cwd, env=env).stdout
-    except (ShellCommandError, UnicodeDecodeError):
-        return Snapshot(tree=None, reason="could not compute tree")
-    return Snapshot(tree=tree, reason=None)
-
-
-def git_common_dir(cwd: Path | None = None) -> Path | None:
-    """Return the absolute git common directory shared by all linked worktrees.
-
-    Args:
-        cwd: Directory to run git in. Defaults to the process working directory.
-
-    Returns:
-        The absolute path, or None when git fails or its output is not a single path.
-    """
-    try:
-        result = run_git("rev-parse", "--git-common-dir", cwd=cwd)
-    except (ShellCommandError, UnicodeDecodeError):
-        return None
-    output = result.stdout.strip()
-    if not output or "\n" in output:
-        return None
-    path = Path(output)
-    if path.is_absolute():
-        return path
-    try:
-        base = Path(cwd) if cwd is not None else Path.cwd()
-        return (base / path).resolve()
-    except (OSError, RuntimeError):
-        return None