judgetap 0.0.2.dev26__tar.gz → 0.0.2.dev28__tar.gz

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: judgetap
-Version: 0.0.2.dev26
+Version: 0.0.2.dev28
 Summary: Fast typed decisions (choice, score, yes/no) across Jev-style engines, plus a guard for coding agents. Early development.
 Project-URL: Homepage, https://github.com/mergesafe-ai/judgetap
 Author-email: Omer Bar-Ness <omer@zsquared.io>
@@ -80,7 +80,7 @@ On a CPU-only Linux box, install the CPU PyTorch wheel before `judgetap[laya]` (
 
 ## Guard details
 
-**Loop detection (Claude Code).** A `PostToolUse` hook (`judgetap guard post`, added by `guard install --for claude-code`) notices when the same command or edit fails the same way three times within eight actions and adds a note asking the agent to re-plan. It never blocks, uses no model, and stores only a redacted action and an error hash per session. Cursor and Codex: not yet.
+**Loop detection (Claude Code).** `PostToolUse` and `PostToolUseFailure` hooks (`judgetap guard post`, added by `guard install --for claude-code`) notices when the same command or edit fails the same way three times within eight actions and adds a note asking the agent to re-plan. It never blocks, uses no model, and stores only a redacted action and an error hash per session. Cursor and Codex: not yet.
 
 - **Agents:** Claude Code (shell, writes and edits), Cursor and Codex (shell only; their hooks don't expose writes and edits).
 - **Engine:** `judgetap guard install` uses one you already have (`$JUDGETAP_ENGINE`, a `TYPESAFE_API_KEY`, or a local AgentJev) and saves it in `~/.judgetap/guard.toml`, because agents often run hooks without your shell's environment.

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/README.md

@@ -63,7 +63,7 @@ On a CPU-only Linux box, install the CPU PyTorch wheel before `judgetap[laya]` (
 
 ## Guard details
 
-**Loop detection (Claude Code).** A `PostToolUse` hook (`judgetap guard post`, added by `guard install --for claude-code`) notices when the same command or edit fails the same way three times within eight actions and adds a note asking the agent to re-plan. It never blocks, uses no model, and stores only a redacted action and an error hash per session. Cursor and Codex: not yet.
+**Loop detection (Claude Code).** `PostToolUse` and `PostToolUseFailure` hooks (`judgetap guard post`, added by `guard install --for claude-code`) notices when the same command or edit fails the same way three times within eight actions and adds a note asking the agent to re-plan. It never blocks, uses no model, and stores only a redacted action and an error hash per session. Cursor and Codex: not yet.
 
 - **Agents:** Claude Code (shell, writes and edits), Cursor and Codex (shell only; their hooks don't expose writes and edits).
 - **Engine:** `judgetap guard install` uses one you already have (`$JUDGETAP_ENGINE`, a `TYPESAFE_API_KEY`, or a local AgentJev) and saves it in `~/.judgetap/guard.toml`, because agents often run hooks without your shell's environment.

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/docs/SPEC.md

@@ -67,7 +67,7 @@ A pre-action hook for coding agents, built on the core.
 - **Outcomes**: allow (silent), hold (block with a reason the agent reads and re-plans from), ask (escalate to the user). Holds should be rare; the target is under 5 per 1,000 calls.
 - **Fails safe and visibly**: Claude Code treats a crashing hook as non-blocking, so the guard catches its own errors, applies the rules layer alone, and says so.
 - **Log**: every decision to a local JSONL, so `judgetap guard stats` can report holds and cost. Marking a hold as a false alarm comes with the dashboard (#10).
-- **Loop detection** (Claude Code `PostToolUse`, no model): the same action failing with the same error (numbers ignored) 3 times in the last 8 actions adds `additionalContext` telling the agent to re-plan; a success of that action resets the count. Never blocks. Per-session ring buffer of 20 redacted actions and error hashes in `~/.judgetap/sessions/`, 0600. Logged as layer `loop`, outcome `note`.
+- **Loop detection** (Claude Code `PostToolUseFailure` for failures, `PostToolUse` for successes that reset a streak; no model): the same action failing with the same error (numbers ignored) 3 times in the last 8 actions adds `additionalContext` telling the agent to re-plan; a success of that action resets the count. Never blocks. Per-session ring buffer of 20 redacted actions and error hashes in `~/.judgetap/sessions/`, 0600. Logged as layer `loop`, outcome `note`.
 
 ## Decided: the guard's default engine (#8)
 

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "judgetap"
-version = "0.0.2.dev26"
+version = "0.0.2.dev28"
 description = "Fast typed decisions (choice, score, yes/no) across Jev-style engines, plus a guard for coding agents. Early development."
 readme = "README.md"
 license = "Apache-2.0"

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/__init__.py

@@ -23,7 +23,7 @@ from judgetap.errors import (
 )
 from judgetap.types import Decision, Question
 
-__version__ = "0.0.2.dev26"  # x-release-please-version
+__version__ = "0.0.2.dev28"  # x-release-please-version
 
 __all__ = [
     "Cascade",

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/_compat.py

@@ -14,8 +14,11 @@ OLD_PREFIX, NEW_PREFIX = "SNAPJUDGE_", "JUDGETAP_"
 
 
 def env(name: str) -> str | None:
-    """$JUDGETAP_<name>, else the old $SNAPJUDGE_<name>."""
-    return os.environ.get(NEW_PREFIX + name) or os.environ.get(OLD_PREFIX + name)
+    """$JUDGETAP_<name>, else the old $SNAPJUDGE_<name>. The new name wins
+    whenever it is set, even to an empty string (which reads as unset)."""
+    if NEW_PREFIX + name in os.environ:
+        return os.environ[NEW_PREFIX + name] or None
+    return os.environ.get(OLD_PREFIX + name) or None
 
 
 def default_home() -> Path:
@@ -26,7 +29,6 @@ def default_home() -> Path:
 
 def env_source(name: str) -> str | None:
     """Which variable `env(name)` read: '$JUDGETAP_<name>' or '$SNAPJUDGE_<name>'."""
-    for prefix in (NEW_PREFIX, OLD_PREFIX):
-        if os.environ.get(prefix + name):
-            return f"${prefix}{name}"
-    return None
+    if NEW_PREFIX + name in os.environ:  # set, even empty: the old name is ignored
+        return f"${NEW_PREFIX}{name}" if os.environ[NEW_PREFIX + name] else None
+    return f"${OLD_PREFIX}{name}" if os.environ.get(OLD_PREFIX + name) else None

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/cli.py

@@ -190,7 +190,7 @@ def _dashboard(args) -> int:
     from judgetap.guard.hook import home
 
     server = serve(home(), args.port)
-    url = f"http://127.0.0.1:{args.port}/"
+    url = f"http://127.0.0.1:{server.server_address[1]}/"  # the bound port (--port 0)
     print(f"judgetap dashboard on {url} (Ctrl+C to stop); reading {home()}")
     if not args.no_browser:
         webbrowser.open(url)
@@ -238,8 +238,14 @@ def _offer_keychain(stdin=None) -> None:
 def _keys_set(args) -> int:
     import getpass
 
-    from judgetap.secrets import set_key
+    from judgetap.secrets import KNOWN_KEYS, set_key
 
+    if args.name not in KNOWN_KEYS:
+        print(
+            f"{args.name} isn't read by any engine; known keys: {', '.join(KNOWN_KEYS)}."
+        )
+        print("The llm engine reads its provider's key from the environment only.")
+        return 2
     value = getpass.getpass(f"{args.name}: ")  # never echoed, never in argv
     if not value:
         print("Nothing entered; not saved.")
@@ -254,9 +260,15 @@ def _keys_set(args) -> int:
 
 
 def _keys_status(args) -> int:
-    from judgetap.secrets import key_source
+    from judgetap.secrets import KNOWN_KEYS, key_source
 
-    for name in args.names or ["TYPESAFE_API_KEY"]:
+    unknown = [n for n in args.names if n not in KNOWN_KEYS]
+    if unknown:
+        print(
+            f"Not read by any engine: {', '.join(unknown)}; known keys: {', '.join(KNOWN_KEYS)}."
+        )
+        return 2
+    for name in args.names or KNOWN_KEYS:
         print(f"{name}: {key_source(name) or 'not set'}")
     return 0
 

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/dashboard/data.py

@@ -85,8 +85,12 @@ def _load(home: Path, today: date) -> dict[str, Any]:
     calls: Counter = Counter()  # true totals; the latency deques are capped
     recent: deque = deque(maxlen=MAX_RECENT)
     cost, total, false_holds, library, loops, stops = 0.0, 0, 0, 0, 0, 0
-    for n, r in _iter_jsonl(home / "guard.jsonl"):
-        r["id"] = record_id(r, n)
+    for n, raw in _iter_jsonl(home / "guard.jsonl"):
+        try:
+            r = _clean(raw)
+            r["id"] = record_id(r, n)
+        except Exception:  # noqa: BLE001, S112 -- one unreadable record never breaks the page
+            continue
         r["source"] = r.get("source") or "guard"
         r["false_alarm"] = r["id"] in false_alarms
         cost += r.get("cost_usd") or 0
@@ -118,7 +122,8 @@ def _load(home: Path, today: date) -> dict[str, Any]:
         "outcomes": dict(outcomes),
         "holds_per_1000": round(1000 * outcomes["hold"] / total, 1) if total else None,
         "false_alarms": false_holds,
-        "cost_usd": round(cost, 6),
+        # Individually finite costs can still sum past float range.
+        "cost_usd": round(cost, 6) if math.isfinite(cost) else None,
         "engines": {
             name: {
                 "calls": calls[name],
@@ -132,6 +137,89 @@ def _load(home: Path, today: date) -> dict[str, Any]:
     return {"summary": summary, "recent": list(recent)[::-1]}
 
 
+STR_FIELDS = (
+    "id",
+    "ts",
+    "session",
+    "source",
+    "tool",
+    "subject",
+    "outcome",
+    "layer",
+    "rule",
+    "reason",
+    "engine",
+    "error",
+    "call",
+    "batch",
+)
+NUM_FIELDS = ("cost_usd", "latency_ms")
+
+
+def _finite(value: Any) -> float | None:
+    """A finite int/float as float; anything else (str, bool, NaN, inf) is None."""
+    if isinstance(value, bool) or not isinstance(value, int | float):
+        return None
+    try:
+        number = float(value)  # huge JSON ints overflow here: an invalid field
+    except OverflowError:
+        return None
+    return number if math.isfinite(number) else None
+
+
+MAX_DEPTH = 8  # deeper than any field the page reads
+
+
+def _scrub(value: Any, depth: int = 0) -> Any:
+    """NaN/Infinity anywhere (even in fields the page doesn't use) become None.
+    Nesting past MAX_DEPTH is cut to None, so a pathological unrelated field
+    can't exhaust the stack and cost the record."""
+    if isinstance(value, float) and not math.isfinite(value):
+        return None
+    if isinstance(value, dict | list) and depth >= MAX_DEPTH:
+        return None
+    if isinstance(value, dict):
+        return {k: _scrub(v, depth + 1) for k, v in value.items()}
+    if isinstance(value, list):
+        return [_scrub(v, depth + 1) for v in value]
+    return value
+
+
+def _clean(r: dict[str, Any]) -> dict[str, Any]:
+    """The log is ours but may be edited, truncated or written by an older
+    version: coerce every field the page uses to its expected type, so no
+    value can crash the summary or make the JSON invalid (NaN)."""
+    out = _scrub(dict(r))
+    for key in STR_FIELDS:
+        if key in out and not isinstance(out[key], str):
+            out[key] = None
+    for key in NUM_FIELDS:
+        if key in out:
+            out[key] = _finite(out[key])
+    if not isinstance(out.get("p"), dict):
+        out["p"] = {}
+    else:
+        out["p"] = {
+            k: v
+            for k, v in out["p"].items()
+            if isinstance(k, str) and _finite(v) is not None
+        }
+    if "calls" in out and not isinstance(out["calls"], list):
+        # Not a calls list at all: treat the record as having none, so the
+        # legacy fallback still counts its engine.
+        del out["calls"]
+    if "calls" in out:
+        # Calls are echoed back in "recent" too: keep only well-formed ones
+        # with a finite (or absent) latency.
+        calls = out["calls"]
+        out["calls"] = [
+            {**c, "latency_ms": _finite(c.get("latency_ms"))}
+            for c in calls
+            if isinstance(c, dict) and isinstance(c.get("engine"), str)
+        ]
+    return out
+
+
 def _count_calls(r: dict[str, Any], calls: Counter, by_engine) -> None:
     """Engine metrics come from the calls each record reports, each with its
     own latency: nothing is inferred. A batch writes its calls on one record
@@ -143,14 +231,17 @@ def _count_calls(r: dict[str, Any], calls: Counter, by_engine) -> None:
         for c in reported:
             if isinstance(c, dict) and isinstance(c.get("engine"), str):
                 calls[c["engine"]] += 1
-                if isinstance(c.get("latency_ms"), int | float):
-                    by_engine[c["engine"]].append(float(c["latency_ms"]))
+                latency = _finite(c.get("latency_ms"))
+                if latency is not None:
+                    by_engine[c["engine"]].append(latency)
         return
     if not r.get("engine") or r.get("error"):
         return
     if r.get("layer") == "judge":
         calls[r["engine"]] += 1
-        by_engine[r["engine"]].append(float(r.get("latency_ms") or 0))
+        latency = _finite(r.get("latency_ms"))
+        if latency is not None:  # no sample for a missing or invalid latency
+            by_engine[r["engine"]].append(latency)
     elif r.get("layer") == "stop" and r.get("outcome") in ("allow", "block"):
         # A Stop check from before calls were logged: it asked its engine
         # once, but recorded no latency, so only the call is counted.

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/dashboard/server.py

@@ -44,7 +44,10 @@ def make_handler(home: Path, token: str, port: int) -> type[BaseHTTPRequestHandl
             self.wfile.write(body)
 
         def _json(self, status: int, obj: object) -> None:
-            self._send(status, json.dumps(obj).encode(), "application/json")
+            # allow_nan=False: bare NaN isn't JSON and would break the page.
+            self._send(
+                status, json.dumps(obj, allow_nan=False).encode(), "application/json"
+            )
 
         def do_GET(self) -> None:
             if not self._host_ok():
@@ -94,4 +97,9 @@ def make_handler(home: Path, token: str, port: int) -> type[BaseHTTPRequestHandl
 def serve(home: Path, port: int = 8765) -> ThreadingHTTPServer:
     token = secrets.token_urlsafe(24)
     server = ThreadingHTTPServer(("127.0.0.1", port), make_handler(home, token, port))
+    # Port 0 means "any free port": the Host allow-list must use the port
+    # the OS actually bound, or every request is refused.
+    bound = server.server_address[1]
+    if bound != port:
+        server.RequestHandlerClass = make_handler(home, token, bound)
     return server

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/guard/install.py

@@ -89,7 +89,14 @@ def _event(agent: str) -> str:
 
 def _events(agent: str, with_stop: bool = False) -> list[str]:
     if agent == "claude-code":
-        return ["PreToolUse", "PostToolUse", *(["Stop"] if with_stop else [])]
+        # Loop detection needs both: failures arrive only on
+        # PostToolUseFailure, successes (which end a streak) on PostToolUse.
+        return [
+            "PreToolUse",
+            "PostToolUse",
+            "PostToolUseFailure",
+            *(["Stop"] if with_stop else []),
+        ]
     return [_event(agent)]
 
 
@@ -98,7 +105,11 @@ def _entry(agent: str, event: str) -> dict:
         return {"command": hook_command(agent)}
     if event == "Stop":  # Stop takes no matcher
         return {"hooks": [{"type": "command", "command": STOP_COMMAND}]}
-    command = POST_COMMAND if event == "PostToolUse" else hook_command(agent)
+    command = (
+        POST_COMMAND
+        if event in ("PostToolUse", "PostToolUseFailure")
+        else hook_command(agent)
+    )
     # Codex's PreToolUse fires for shell only today; the matcher says so.
     matcher = "^(exec_command|shell|Bash)$" if agent == "codex" else MATCHER
     return {"matcher": matcher, "hooks": [{"type": "command", "command": command}]}

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/guard/loop.py

@@ -13,6 +13,7 @@ import json
 import os
 import re
 import sys
+import time
 import uuid
 from contextlib import contextmanager
 from datetime import UTC, datetime
@@ -50,10 +51,20 @@ EXIT_PREFIX = re.compile(r"^\s*exit code[: ]\s*(-?\d+)", re.IGNORECASE)
 
 
 def failure(payload: dict[str, Any]) -> str | None:
-    """A short hash of the error, or None when the call succeeded."""
+    """A short hash of the error, or None when the call succeeded.
+
+    Claude Code sends failures on PostToolUseFailure with the error as a
+    top-level `error` string (for Bash, first line "Exit code N"); successes
+    arrive on PostToolUse. The tool_response checks cover other agents and
+    older shapes."""
     resp = payload.get("tool_response")
     error = payload.get("error")
-    text, failed, code = "", bool(error), None
+    failure_event = payload.get("hook_event_name") == "PostToolUseFailure"
+    text, failed, code = "", bool(error) or failure_event, None
+    if isinstance(error, str):
+        m = EXIT_PREFIX.match(error)
+        if m:
+            code = int(m.group(1))
     if isinstance(resp, dict):
         code = resp.get("exit_code", resp.get("exitCode", resp.get("returncode")))
         if isinstance(code, int) and code != 0:
@@ -80,6 +91,72 @@ def failure(payload: dict[str, Any]) -> str | None:
     return hashlib.sha256(stable.encode()).hexdigest()[:12]
 
 
+SESSION_TTL_SECONDS = 7 * 24 * 3600
+PRUNE_EVERY_SECONDS = 3600
+
+
+def prune_sessions(directory: Path, now: float | None = None) -> None:
+    """Delete session state (json, stop, tmp) untouched for SESSION_TTL_SECONDS.
+
+    Lock files are never deleted: unlinking a lock someone holds would let a
+    second hook lock a fresh inode and break mutual exclusion. They are empty,
+    so keeping them costs an inode, not space. A session's data is only
+    removed while holding its lock without waiting; a busy session is skipped.
+    Runs at most once per PRUNE_EVERY_SECONDS and never raises."""
+    try:
+        now = time.time() if now is None else now
+        marker = directory / ".pruned"
+        if marker.exists() and now - marker.stat().st_mtime < PRUNE_EVERY_SECONDS:
+            return
+        directory.mkdir(mode=0o700, parents=True, exist_ok=True)
+        marker.touch()
+        os.utime(marker, (now, now))
+        for f in directory.iterdir():
+            if f.name == ".pruned" or f.suffix not in (".json", ".stop", ".tmp"):
+                continue
+            try:
+                if now - f.stat().st_mtime <= SESSION_TTL_SECONDS:
+                    continue
+                _unlink_if_unlocked(f)
+            except OSError:
+                continue
+    except OSError:
+        return
+
+
+TMP_NAME = re.compile(r"^(?P<stem>.+)\.[0-9a-f]{32}\.tmp$")
+
+
+def _lock_for(f: Path) -> Path:
+    """The lock _session_lock takes for this file. State files are
+    `<id>.json` / `<id>.stop` and lock `<id>.lock` (with_suffix, so a
+    session id may itself contain dots); their temp files are
+    `<id>.<uuid>.tmp` (with_suffix replaces .json/.stop), so the stem before
+    the uuid is the session id."""
+    m = TMP_NAME.match(f.name)
+    if m:
+        return f.with_name(m.group("stem") + ".lock")
+    return f.with_suffix(".lock")
+
+
+def _unlink_if_unlocked(f: Path) -> None:
+    """Remove f only while holding its session lock (non-blocking)."""
+    lock = _lock_for(f)
+    fd = os.open(lock, os.O_WRONLY | os.O_CREAT, 0o600)
+    try:
+        try:
+            import fcntl
+
+            fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        except ImportError:
+            pass  # no flock (Windows): best effort, as for the hooks themselves
+        except OSError:
+            return  # a hook holds it: this session is in use, keep its data
+        f.unlink(missing_ok=True)
+    finally:
+        os.close(fd)
+
+
 def _state_path(session: str | None) -> Path | None:
     if not session or not SESSION_ID.fullmatch(session):
         return None
@@ -167,6 +244,9 @@ def handle(payload: dict[str, Any]) -> dict[str, Any] | None:
     path = _state_path(payload.get("session_id"))
     if act is None or path is None:
         return None
+    if payload.get("is_interrupt"):
+        return None  # an abort, not an error the tool reported: not a loop signal
+    prune_sessions(path.parent)
     # Only a hash and the (redacted) action are stored: never file contents.
     # Hooks for one session can finish together: serialise the
     # read-append-write so no action is lost.
@@ -187,9 +267,13 @@ def handle(payload: dict[str, Any]) -> dict[str, Any] | None:
         return None
     _log(payload.get("session_id"), actions[-1]["act"], count)
     shown = actions[-1]["act"].split(":", 1)[1][:200]
+    event = payload.get("hook_event_name")
+    if event not in ("PostToolUse", "PostToolUseFailure"):
+        event = "PostToolUseFailure"
     return {
         "hookSpecificOutput": {
-            "hookEventName": "PostToolUse",
+            # Answer on the event that fired (PostToolUseFailure for failures).
+            "hookEventName": event,
             "additionalContext": (
                 f"judgetap: `{shown}` has now failed the same way {count} times; "
                 "stop and re-plan (read the error, try a different approach)."

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/guard/stop.py

@@ -113,6 +113,9 @@ def _count_path(session: str) -> Path:
 def _take_block(session: str) -> bool:
     """Count one block for this session; False once the cap is reached."""
     path = _count_path(session)
+    from judgetap.guard.loop import prune_sessions
+
+    prune_sessions(path.parent)
     with _session_lock(path):
         try:
             count = int(path.read_text().strip() or 0)
@@ -177,6 +180,11 @@ def handle(payload: dict[str, Any], engine=None) -> dict[str, Any] | None:
         return None  # needs a judge: no rules-only behaviour here
     transcript = payload.get("transcript_path")
     task, said = task_and_reply(transcript)
+    # Claude Code passes the final reply directly: the transcript is written
+    # asynchronously and may not contain it yet at Stop time.
+    last = payload.get("last_assistant_message")
+    if isinstance(last, str) and last.strip():
+        said = last[-ASSISTANT_TAIL_CHARS:]
     if not task or not said:
         _log(
             session,

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/src/judgetap/secrets.py

@@ -10,6 +10,9 @@ from __future__ import annotations
 
 import os
 
+# Keys an engine reads through get_key. Saving any other name to the keychain
+# would store something nothing reads.
+KNOWN_KEYS = ("TYPESAFE_API_KEY",)
 SERVICE = "judgetap"
 OLD_SERVICE = "snapjudge"  # keys saved before the rename (#38)
 

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/tests/test_guard_loop.py

@@ -16,16 +16,33 @@ def _home(tmp_path, monkeypatch):
 def call(
     command, *, fail=True, err="npm ERR! missing script: build", session="s1", code=1
 ):
-    payload = {
+    """A documented Claude Code payload: failures on PostToolUseFailure with a
+    top-level `error` ("Exit code N" first line for Bash), successes on
+    PostToolUse with a tool_response."""
+    base = {
         "session_id": session,
+        "transcript_path": "/tmp/t.jsonl",
+        "cwd": "/tmp",
+        "permission_mode": "default",
         "tool_name": "Bash",
-        "tool_input": {"command": command},
-        "tool_response": {
-            "stdout": "",
-            "stderr": err if fail else "",
-            "exit_code": code if fail else 0,
-        },
+        "tool_input": {"command": command, "description": "run"},
+        "tool_use_id": "toolu_01ABC",
     }
+    if fail:
+        payload = {
+            **base,
+            "hook_event_name": "PostToolUseFailure",
+            "error": f"Exit code {code}\n{err}",
+            "is_interrupt": False,
+            "duration_ms": 10,
+        }
+    else:
+        payload = {
+            **base,
+            "hook_event_name": "PostToolUse",
+            "tool_response": {"stdout": "ok", "stderr": "", "interrupted": False},
+            "duration_ms": 10,
+        }
     out = io.StringIO()
     assert loop.run(io.StringIO(json.dumps(payload)), out) == 0
     return json.loads(out.getvalue()) if out.getvalue() else None
@@ -36,7 +53,7 @@ def test_third_identical_failure_adds_a_note():
     assert call("npm run build") is None
     out = call("npm run build")
     ctx = out["hookSpecificOutput"]
-    assert ctx["hookEventName"] == "PostToolUse"
+    assert ctx["hookEventName"] == "PostToolUseFailure"
     assert (
         "`npm run build` has now failed the same way 3 times"
         in ctx["additionalContext"]
@@ -225,3 +242,141 @@ def test_concurrent_hooks_do_not_lose_actions(tmp_path, monkeypatch):
     for t in threads:
         t.join()
     assert len(real_load(tmp_path / "sessions" / "race.json")) == 6
+
+
+DOC_FAILURE = {
+    # Verbatim from code.claude.com/docs/en/hooks#posttoolusefailure-input
+    "session_id": "abc123",
+    "transcript_path": "/Users/.../.claude/projects/.../00893aaf-19fa-41d2-8238-13269b9b3ca0.jsonl",
+    "cwd": "/Users/...",
+    "permission_mode": "default",
+    "hook_event_name": "PostToolUseFailure",
+    "tool_name": "Bash",
+    "tool_input": {"command": "npm test", "description": "Run test suite"},
+    "tool_use_id": "toolu_01ABC123...",
+    "error": "Exit code 1\nError: Cannot find module 'express'",
+    "is_interrupt": False,
+    "duration_ms": 4187,
+}
+
+
+def test_documented_failure_payload_triggers_on_the_third_repeat():
+    outs = [loop.handle(dict(DOC_FAILURE)) for _ in range(3)]
+    assert outs[:2] == [None, None]
+    assert outs[2]["hookSpecificOutput"]["hookEventName"] == "PostToolUseFailure"
+
+
+def test_interrupts_are_not_loop_signals():
+    for _ in range(4):
+        assert loop.handle({**DOC_FAILURE, "is_interrupt": True}) is None
+
+
+def test_install_adds_post_tool_use_failure_and_upgrades_old_installs(tmp_path):
+    path = tmp_path / "settings.json"
+    old = {
+        "hooks": {
+            "PostToolUse": [
+                {
+                    "matcher": "Bash",
+                    "hooks": [{"type": "command", "command": POST_COMMAND}],
+                }
+            ]
+        }
+    }
+    path.write_text(json.dumps(old))
+    assert install(path) is True
+    hooks = json.loads(path.read_text())["hooks"]
+    assert POST_COMMAND in json.dumps(hooks["PostToolUseFailure"])
+    assert install(path) is False
+    assert uninstall(path) is True
+    assert "PostToolUseFailure" not in json.loads(path.read_text()).get("hooks", {})
+
+
+def test_prune_sessions_removes_old_state_once_an_hour(tmp_path):
+    import os
+
+    d = tmp_path / "sessions"
+    d.mkdir()
+    old, fresh = d / "a.json", d / "b.json"
+    old.write_text("[]")
+    fresh.write_text("[]")
+    now = 10_000_000.0
+    os.utime(old, (now - 8 * 86400, now - 8 * 86400))
+    os.utime(fresh, (now - 60, now - 60))
+    loop.prune_sessions(d, now=now)
+    assert not old.exists() and fresh.exists()
+    stale = d / "c.json"
+    stale.write_text("[]")
+    os.utime(stale, (now - 9 * 86400, now - 9 * 86400))
+    loop.prune_sessions(d, now=now + 60)  # within the hour: no sweep
+    assert stale.exists()
+    loop.prune_sessions(d, now=now + 3700)
+    assert not stale.exists()
+
+
+def test_prune_keeps_locks_and_skips_busy_sessions(tmp_path):
+    import fcntl
+    import os
+    import time
+
+    from judgetap.guard.loop import SESSION_TTL_SECONDS, prune_sessions
+
+    old = time.time() - SESSION_TTL_SECONDS - 100
+    for name in (
+        "idle.json",
+        "idle.lock",
+        "busy.json",
+        "busy.lock",
+        "idle.stop",
+        "x." + "ab" * 16 + ".tmp",
+    ):
+        p = tmp_path / name
+        p.write_text("{}")
+        os.utime(p, (old, old))
+    fd = os.open(tmp_path / "busy.lock", os.O_WRONLY)
+    fcntl.flock(fd, fcntl.LOCK_EX)  # another hook is working on "busy"
+    try:
+        prune_sessions(tmp_path)
+    finally:
+        os.close(fd)
+    left = sorted(p.name for p in tmp_path.iterdir() if p.name != ".pruned")
+    # x.lock: created to take the orphan temp file's session lock before deleting it.
+    assert left == ["busy.json", "busy.lock", "idle.lock", "x.lock"]
+
+
+def test_prune_leaves_a_temp_file_whose_session_is_locked(tmp_path):
+    import fcntl
+    import os
+    import time
+
+    from judgetap.guard.loop import SESSION_TTL_SECONDS, prune_sessions
+
+    old = time.time() - SESSION_TTL_SECONDS - 100
+    tmp = tmp_path / (
+        "busy." + "0123abcd" * 4 + ".tmp"
+    )  # <id>.<uuid hex>.tmp, as _save names it
+    lock = tmp_path / "busy.lock"
+    for p in (tmp, lock):
+        p.write_text("x")
+        os.utime(p, (old, old))
+    fd = os.open(lock, os.O_WRONLY)
+    fcntl.flock(fd, fcntl.LOCK_EX)
+    try:
+        prune_sessions(tmp_path)
+    finally:
+        os.close(fd)
+    assert tmp.exists()
+    prune_sessions(tmp_path, now=time.time() + 7200)  # next sweep, lock free
+    assert not tmp.exists()
+
+
+def test_lock_path_matches_session_lock_for_dotted_ids(tmp_path):
+    import uuid
+
+    from judgetap.guard.loop import _lock_for
+
+    d = tmp_path
+    for data in ("a.b.json", "a.b.stop"):
+        assert _lock_for(d / data) == (d / data).with_suffix(".lock") == d / "a.b.lock"
+        tmp = (d / data).with_suffix(f".{uuid.uuid4().hex}.tmp")  # as _save writes it
+        assert _lock_for(tmp) == d / "a.b.lock"

{judgetap-0.0.2.dev26 → judgetap-0.0.2.dev28}/tests/test_guard_stop.py

@@ -212,3 +212,40 @@ def test_failed_judge_logs_its_calls_and_lets_the_agent_stop(tmp_path, monkeypat
         is None
     )
     assert load(tmp_path)["summary"]["engines"]["low"]["calls"] == 1
+
+
+def test_stop_uses_last_assistant_message_when_the_transcript_lags(
+    tmp_path, monkeypatch
+):
+    import json
+
+    from judgetap.guard import stop
+    from judgetap.testing import StaticEngine
+
+    monkeypatch.setenv("JUDGETAP_HOME", str(tmp_path))
+    t = tmp_path / "t.jsonl"
+    t.write_text(
+        json.dumps({"type": "user", "message": {"content": "Fix all 3 failing tests"}})
+        + "\n"
+    )
+    seen = []
+
+    def judge(q, ctx):
+        seen.append(ctx["assistant_last_message"])
+        return {"yes": 0.05, "no": 0.95}
+
+    # The documented Stop input: the final reply comes in last_assistant_message.
+    payload = {
+        "session_id": "abc123",
+        "transcript_path": str(t),
+        "cwd": "/tmp",
+        "permission_mode": "default",
+        "hook_event_name": "Stop",
+        "stop_hook_active": False,
+        "last_assistant_message": "I fixed one test; two remain, stopping.",
+        "background_tasks": [],
+        "session_crons": [],
+    }
+    out = stop.handle(payload, engine=StaticEngine(judge, name="j"))
+    assert seen == ["I fixed one test; two remain, stopping."]
+    assert out["decision"] == "block"

judgetap-0.0.2.dev28/tests/test_robustness_68.py

@@ -0,0 +1,177 @@
+import json
+import math
+import threading
+import urllib.request
+
+import pytest
+
+from judgetap.dashboard.data import load
+
+
+def write(tmp_path, rows):
+    (tmp_path / "guard.jsonl").write_text("\n".join(rows) + "\n")
+
+
+@pytest.mark.parametrize(
+    "line",
+    [
+        '{"outcome":"allow","cost_usd":"0.1"}',
+        '{"outcome":["hold"]}',
+        '{"outcome":"allow","cost_usd":NaN}',
+        '{"outcome":"allow","layer":"judge","calls":[{"engine":"e","latency_ms":NaN}]}',
+        '{"outcome":"allow","layer":{"x":1},"engine":7,"source":[1],"p":"nope","calls":"x"}',
+        '{"outcome":"allow","latency_ms":Infinity,"layer":"judge","engine":"e"}',
+    ],
+)
+def test_odd_log_values_never_break_load(tmp_path, line):
+    write(tmp_path, [json.dumps({"outcome": "hold", "layer": "rules"}), line])
+    data = load(tmp_path)
+    json.dumps(data, allow_nan=False)  # strict JSON: no NaN/Infinity left
+    assert data["summary"]["outcomes"].get("hold") == 1
+    assert all(isinstance(k, str) or k is None for k in data["summary"]["outcomes"])
+
+
+def test_nan_latency_is_not_sampled(tmp_path):
+    write(
+        tmp_path,
+        [
+            '{"outcome":"allow","layer":"judge","calls":[{"engine":"e","latency_ms":NaN},{"engine":"e","latency_ms":5}]}'
+        ],
+    )
+    e = load(tmp_path)["summary"]["engines"]["e"]
+    assert e["calls"] == 2 and e["p50_ms"] == 5 and not math.isnan(e["p50_ms"])
+
+
+def test_api_data_answers_with_a_bad_line(tmp_path):
+    from judgetap.dashboard.server import serve
+
+    write(tmp_path, ['{"outcome":"allow","cost_usd":"0.1"}', '{"outcome":["hold"]}'])
+    srv = serve(tmp_path, 0)
+    port = srv.server_address[1]
+    threading.Thread(target=srv.serve_forever, daemon=True).start()
+    try:
+        with urllib.request.urlopen(f"http://127.0.0.1:{port}/api/data") as res:
+            assert res.status == 200 and json.loads(res.read())["summary"]["total"] == 2
+    finally:
+        srv.shutdown()
+        srv.server_close()
+
+
+def test_port_zero_uses_the_bound_port(tmp_path):
+    from judgetap.dashboard.server import serve
+
+    srv = serve(tmp_path, 0)
+    port = srv.server_address[1]
+    assert port != 0
+    threading.Thread(target=srv.serve_forever, daemon=True).start()
+    try:
+        with urllib.request.urlopen(f"http://127.0.0.1:{port}/") as res:
+            assert res.status == 200
+    finally:
+        srv.shutdown()
+        srv.server_close()
+
+
+def test_keys_set_rejects_names_no_engine_reads(monkeypatch, capsys):
+    from judgetap.cli import main
+
+    asked = []
+    monkeypatch.setattr("getpass.getpass", lambda prompt: asked.append(prompt) or "v")
+    assert main(["keys", "set", "OPENAI_API_KEY"]) == 2
+    assert not asked and "isn't read by any engine" in capsys.readouterr().out
+    assert main(["keys", "status", "OPENAI_API_KEY"]) == 2
+
+
+def test_empty_new_env_wins_over_old(monkeypatch):
+    from judgetap._compat import env, env_source
+
+    monkeypatch.setenv("JUDGETAP_ENGINE", "")
+    monkeypatch.setenv("SNAPJUDGE_ENGINE", "llm")
+    assert env("ENGINE") is None and env_source("ENGINE") is None
+    monkeypatch.delenv("JUDGETAP_ENGINE")
+    assert env("ENGINE") == "llm" and env_source("ENGINE") == "$SNAPJUDGE_ENGINE"
+
+
+def test_nan_in_unknown_fields_is_scrubbed(tmp_path):
+    write(tmp_path, ['{"outcome":"allow","extra":{"x":[NaN, -Infinity]}}'])
+    json.dumps(load(tmp_path), allow_nan=False)
+
+
+def test_huge_int_cost_keeps_the_record(tmp_path):
+    import json
+
+    from judgetap.dashboard.data import load
+
+    (tmp_path / "guard.jsonl").write_text(
+        json.dumps({"outcome": "hold", "cost_usd": 10**400}) + "\n"
+    )
+    s = load(tmp_path)["summary"]
+    assert s["total"] == 1 and s["outcomes"] == {"hold": 1}
+
+
+def test_invalid_legacy_latency_adds_no_sample(tmp_path):
+    from judgetap.dashboard.data import load
+
+    (tmp_path / "guard.jsonl").write_text(
+        '{"outcome":"allow","layer":"judge","engine":"e","latency_ms":NaN}\n'
+    )
+    e = load(tmp_path)["summary"]["engines"]["e"]
+    assert e["calls"] == 1 and e["p50_ms"] is None
+
+
+def test_overflowing_cost_total_still_serves(tmp_path):
+    import json
+    import socket
+    import threading
+    import urllib.request
+    from http.server import ThreadingHTTPServer
+
+    from judgetap.dashboard.server import make_handler
+
+    (tmp_path / "guard.jsonl").write_text(
+        "\n".join(json.dumps({"outcome": "allow", "cost_usd": 1e308}) for _ in range(2))
+        + "\n"
+    )
+    with socket.socket() as sock:
+        sock.bind(("127.0.0.1", 0))
+        port = sock.getsockname()[1]
+    srv = ThreadingHTTPServer(("127.0.0.1", port), make_handler(tmp_path, "tok", port))
+    threading.Thread(target=srv.serve_forever, daemon=True).start()
+    try:
+        with urllib.request.urlopen(f"http://127.0.0.1:{port}/api/data") as res:
+            assert res.status == 200
+            assert json.loads(res.read())["summary"]["cost_usd"] is None
+    finally:
+        srv.shutdown()
+        srv.server_close()
+
+
+def test_malformed_calls_field_keeps_legacy_fallback(tmp_path):
+    import json
+
+    from judgetap.dashboard.data import load
+
+    (tmp_path / "guard.jsonl").write_text(
+        json.dumps(
+            {
+                "id": "a",
+                "outcome": "allow",
+                "layer": "judge",
+                "engine": "e",
+                "latency_ms": 12,
+                "calls": "x",
+            }
+        )
+        + "\n"
+    )
+    e = load(tmp_path)["summary"]["engines"]["e"]
+    assert e["calls"] == 1 and e["p50_ms"] == 12
+
+
+def test_deeply_nested_extra_field_does_not_drop_the_record(tmp_path):
+    from judgetap.dashboard.data import load
+
+    deep = "[" * 5000 + "]" * 5000
+    line = '{"id": "a", "outcome": "hold", "layer": "rules", "extra": ' + deep + "}"
+    (tmp_path / "guard.jsonl").write_text(line + "\n")
+    assert load(tmp_path)["summary"]["total"] == 1