@miller-tech/uap 1.165.0 → 1.168.0

package/src/policies/schemas/policies/branch-freshness.md

@@ -0,0 +1,81 @@
+# Policy: Branch Freshness
+
+**ID**: `policy-branch-freshness`
+**Name**: Work Against the Latest Integration Branch
+**Category**: workflow
+**Level**: REQUIRED
+**Enforcement Stage**: pre-exec
+**Version**: 1.0
+
+## Purpose
+
+Multiple agents working the same codebase in parallel lose each other's work in two
+distinct ways. Only one of them is a "merge conflict".
+
+1. **Concurrent collision** — two agents hold the same file at the same moment.
+   Git surfaces this loudly at merge time. Handled by `coordinate-file.sh`'s
+   live-agent lock.
+2. **Sequential drift** — agent A lands a change; agent B's branch was cut before
+   that and never re-synced, so B edits a stale copy. B's merge silently reverts
+   A's work, or forces a conflict resolution that a model resolves by picking its
+   own version. Git does NOT warn about this. This is the expensive one.
+
+This policy addresses (2): work must be based on, and kept close to, the current
+integration branch.
+
+## Rules
+
+```rules
+- title: "Fresh Base"
+  keywords: ["worktree", "branch", "create", "start work", "new feature"]
+  antiPatterns: ["branch from stale local", "no fetch before branching", "based on local master", "detached stale base"]
+
+- title: "Stale File Guard"
+  keywords: ["edit", "write", "modify", "change file"]
+  antiPatterns: ["edit file changed upstream", "ignore upstream change", "overwrite landed work", "revert merged change"]
+
+- title: "Drift Ceiling"
+  keywords: ["edit", "write", "worktree", "long-running branch"]
+  antiPatterns: ["hundreds of commits behind", "never synced", "stale worktree", "abandoned branch still edited"]
+```
+
+## Enforcement Behavior
+
+### When Triggered
+
+- **Worktree creation** — `uap worktree create` fetches and bases the new branch on
+  `origin/<default>` rather than whatever the local checkout has at HEAD.
+- **Every worktree file edit** — two independent checks:
+  - *file-precise* (`coordinate-file.sh`): blocks when the specific file being
+    edited has changed on the integration branch since this branch's merge-base.
+  - *branch-coarse* (`branch_freshness.py`): warns at 50 commits behind, blocks at 200.
+
+### Required Actions
+
+1. Cut branches from the fetched remote tip, never from a stale local ref.
+2. Before editing a file that moved upstream, run `uap worktree sync` and re-apply
+   the change on top of the landed version.
+3. Keep a branch within the drift ceiling for its whole life; sync routinely rather
+   than once at `finish`, when resolution is most expensive.
+4. Land PRs through `uap merge queue` so each merge re-syncs the PRs it impacts.
+
+### Overrides
+
+| Variable | Effect |
+|---|---|
+| `UAP_COORD_DRIFT=warn` | file-precise check warns instead of blocking |
+| `UAP_COORD_DRIFT=off` | disable the file-precise check |
+| `UAP_NO_FRESHNESS=1` | disable the branch-coarse check |
+| `UAP_FRESHNESS_WARN` / `UAP_FRESHNESS_BLOCK` | tune the drift thresholds |
+| `uap worktree create --no-fetch` | branch offline, accepting a possibly stale base |
+
+All checks fail **open**: no remote, no git, or an unreadable coordination DB
+allows the edit. Coordination being unavailable must never stop work.
+
+## Rationale
+
+Measured on this repository before these gates existed: 151 worktrees, the worst
+1241 commits behind `origin/master`, 23 holding unmerged commits and 15 with
+uncommitted files. A worktree created during that audit was born 1 commit behind
+because creation read local `HEAD` and never fetched. None of that drift was
+visible to any gate, and none of it produced a warning at edit time.

package/templates/hooks/coordinate-file.sh

@@ -11,7 +11,9 @@
 # different worktrees collides (that IS the future merge conflict).
 #
 # Exit 0 = allow (no conflict, or only a stale/self-healed announcement → warn).
-# Exit 2 = BLOCK: another LIVE agent (heartbeat < THRESHOLD) holds this file.
+# Exit 2 = BLOCK: another LIVE agent (heartbeat < THRESHOLD) holds this file, OR
+#          the file has MOVED on the integration branch since this branch's
+#          merge-base (editing it now would silently revert landed work).
 #
 # Always fails OPEN: any missing dependency or DB error allows the edit. This
 # hook must never break editing because coordination is unavailable.
@@ -27,10 +29,158 @@ ABS="${6:-}"
 # Seconds since another agent's last heartbeat for it to count as "live".
 THRESHOLD="${UAP_COORD_LIVE_SECONDS:-120}"
 
-# Fail open on any missing prerequisite.
+# ---------------------------------------------------------------------------
+# Merge-base drift check (sequential-overwrite protection).
+#
+# The live-lock below only covers CONCURRENT edits — two agents holding the same
+# file at the same moment. The far more common loss is SEQUENTIAL: agent A landed
+# a change to foo.ts an hour ago; agent B's worktree was cut before that and never
+# re-synced, so B edits a stale copy and its merge quietly reverts A's work.
+#
+# So: if THIS exact file changed on the integration branch since our merge-base,
+# block until the agent syncs. Scoped to the single file being edited, so a stale
+# worktree touching untouched files keeps working — no blanket freeze.
+#
+#   UAP_COORD_DRIFT=block|warn|off   (default block)
+#   UAP_COORD_FETCH_SECONDS=<n>      throttle for the background fetch (default 600)
+# ---------------------------------------------------------------------------
+DRIFT_MODE="${UAP_COORD_DRIFT:-block}"
+FETCH_THROTTLE="${UAP_COORD_FETCH_SECONDS:-600}"
+
+# Validate every numeric knob before it reaches SQL or arithmetic. THRESHOLD is
+# interpolated into the liveness SQL below; a non-numeric value there produced a
+# silent SQL error, which fails the query open and disables the lock entirely.
+case "$THRESHOLD" in ''|*[!0-9]*) THRESHOLD=120 ;; esac
+case "$FETCH_THROTTLE" in ''|*[!0-9]*) FETCH_THROTTLE=600 ;; esac
+
+# Bound the git calls in check_drift. Only the fetch was bounded before, so a
+# slow merge-base/diff on a huge repo or a stalled network FS could hang the
+# agent's Edit tool with no diagnostic. macOS has no `timeout` by default.
+if command -v timeout >/dev/null 2>&1; then
+  DRIFT_TIMEOUT="timeout -k 1 10"
+elif command -v gtimeout >/dev/null 2>&1; then
+  DRIFT_TIMEOUT="gtimeout -k 1 10"
+else
+  DRIFT_TIMEOUT=""
+fi
+
+[ -n "$ME" ] && [ -n "$REL" ] || exit 0
+
+# Resolve the integration ref (origin/HEAD → origin/master → origin/main).
+integration_ref() {
+  local d="$1" r
+  r=$(git -C "$d" symbolic-ref --quiet refs/remotes/origin/HEAD 2>/dev/null) && {
+    printf '%s' "${r#refs/remotes/}"; return 0; }
+  for c in master main; do
+    git -C "$d" rev-parse --verify --quiet "refs/remotes/origin/$c" >/dev/null 2>&1 && {
+      printf 'origin/%s' "$c"; return 0; }
+  done
+  return 1
+}
+
+check_drift() {
+  [ "$DRIFT_MODE" = "off" ] && return 0
+  command -v git >/dev/null 2>&1 || return 0
+  [ -n "$ABS" ] || return 0
+
+  local dir; dir="${ABS%/*}"
+  [ -d "$dir" ] || return 0
+
+  # Anchor EVERY git call to the working tree that actually contains the file.
+  # Running them from the file's own directory was a latent no-op: git resolves a
+  # pathspec against the process prefix, so `-C <wt>/src/cli ... -- src/cli/x.ts`
+  # looked for <wt>/src/cli/src/cli/x.ts and matched nothing. Drift then "passed"
+  # for every file below the repo root — i.e. all real source — silently and
+  # fail-open. It only appeared to work for files sitting at the root, which is
+  # exactly what the first round of tests used.
+  local top; top=$(git -C "$dir" rev-parse --show-toplevel 2>/dev/null) || return 0
+  [ -n "$top" ] || return 0
+
+  # Never fight an in-progress merge/rebase: the conflict markers the agent has to
+  # edit are, by definition, in files that moved upstream. Blocking there deadlocks
+  # the very resolution `uap worktree sync` just asked for.
+  local gitdir; gitdir=$(git -C "$top" rev-parse --absolute-git-dir 2>/dev/null) || return 0
+  for marker in MERGE_HEAD REBASE_HEAD CHERRY_PICK_HEAD REVERT_HEAD; do
+    [ -e "$gitdir/$marker" ] && return 0
+  done
+  [ -d "$gitdir/rebase-merge" ] || [ -d "$gitdir/rebase-apply" ] && return 0
+
+  # Path relative to THIS working tree, derived here rather than trusting the
+  # caller's REL — which is relative to the agent's cwd and is wrong whenever the
+  # agent stands in the main checkout while editing into a worktree.
+  local rel="${ABS#"$top"/}"
+  [ "$rel" != "$ABS" ] || return 0
+
+  local ref; ref=$(integration_ref "$top") || return 0
+
+  # Throttled fetch: an edit-time network call on EVERY keystroke-scale edit would
+  # be intolerable, so refresh at most once per FETCH_THROTTLE seconds. Between
+  # refreshes we compare against the last known remote tip — still catches the
+  # overwhelming majority of drift, and `uap worktree sync` refreshes on demand.
+  local common; common=$(git -C "$top" rev-parse --path-format=absolute --git-common-dir 2>/dev/null) \
+    || common=$(git -C "$top" rev-parse --git-common-dir 2>/dev/null) || return 0
+  case "$common" in /*) ;; *) common="$top/$common" ;; esac
+  # If the stamp dir is not real, DISABLE the fetch rather than the throttle —
+  # otherwise a bad path silently means "fetch on every single edit".
+  if [ -d "$common" ]; then
+    local stamp="$common/.uap-drift-fetch"
+    local now; now=$(date +%s)
+    local last=0
+    [ -f "$stamp" ] && read -r last < "$stamp" 2>/dev/null
+    case "$last" in ''|*[!0-9]*) last=0 ;; esac
+    # A future-dated stamp (clock skew, or a stray write) would otherwise wedge
+    # fetching off indefinitely.
+    [ "$last" -gt "$now" ] 2>/dev/null && last=0
+    # mkdir is atomic on POSIX and NFS: with many agents sharing one common dir,
+    # a read-compare-write throttle lets every agent in the window fetch at once.
+    if [ $((now - last)) -ge "$FETCH_THROTTLE" ] && mkdir "$stamp.lock" 2>/dev/null; then
+      printf '%s' "$now" > "$stamp.tmp.$$" 2>/dev/null &&
+        mv -f "$stamp.tmp.$$" "$stamp" 2>/dev/null || true
+      # Detached: nothing in THIS edit depends on the fetch landing — we compare
+      # against the last known tip either way. Redirections are load-bearing; a
+      # background job holding the hook's stdout makes the harness wait on it.
+      (
+        GIT_TERMINAL_PROMPT=0 GIT_SSH_COMMAND='ssh -oBatchMode=yes -oConnectTimeout=5' \
+          $DRIFT_TIMEOUT git -C "$top" fetch -q origin "${ref#origin/}" >/dev/null 2>&1
+        rmdir "$stamp.lock" 2>/dev/null
+      ) </dev/null >/dev/null 2>&1 &
+    fi
+    # Reap a lock orphaned by a killed agent (older than 10 min).
+    if [ -d "$stamp.lock" ]; then
+      local lockage; lockage=$(find "$stamp.lock" -maxdepth 0 -mmin +10 2>/dev/null || true)
+      [ -n "$lockage" ] && rmdir "$stamp.lock" 2>/dev/null
+    fi
+  fi
+
+  local mb; mb=$($DRIFT_TIMEOUT git -C "$top" merge-base HEAD "$ref" 2>/dev/null) || return 0
+  [ -n "$mb" ] || return 0
+
+  # Did this specific path change on the integration branch since we branched?
+  # diff-tree (plumbing) skips the work-tree setup and index read that `git diff`
+  # performs, and ":(top)" pins the pathspec to the repo root regardless of cwd.
+  local moved; moved=$($DRIFT_TIMEOUT git -C "$top" diff-tree -r --name-only --no-renames \
+    "$mb" "$ref" -- ":(top)$rel" 2>/dev/null)
+  [ -n "$moved" ] || return 0
+
+  local n; n=$($DRIFT_TIMEOUT git -C "$top" rev-list --count "$mb..$ref" 2>/dev/null || echo '?')
+
+  if [ "$DRIFT_MODE" = "warn" ]; then
+    echo "COORDINATION WARNING: ${rel} changed on ${ref} since your branch point (${n} commits behind). Run 'uap worktree sync' before editing or your merge may revert landed work." >&2
+    return 0
+  fi
+
+  echo "{\"decision\":\"block\",\"reason\":\"STALE FILE: ${rel} has changed on ${ref} since your branch point (you are ${n} commits behind). Editing this copy risks silently reverting work that already landed. Run 'uap worktree sync' first, then re-apply your change on top. Override: UAP_COORD_DRIFT=warn.\"}" >&2
+  return 2
+}
+
+# 0) Sequential-drift check FIRST. It depends only on git, never on the
+#    coordination DB — a missing/unwritable DB must not silently disable
+#    overwrite protection (it did: the DB prerequisite used to exit 0 above it).
+check_drift || exit 2
+
+# The announcement half needs sqlite3 and a real DB file; fail open without them.
 command -v sqlite3 >/dev/null 2>&1 || exit 0
 [ -n "$DB" ] && [ -f "$DB" ] || exit 0
-[ -n "$ME" ] && [ -n "$REL" ] || exit 0
 
 # SQL-escape single quotes.
 q() { printf '%s' "${1:-}" | sed "s/'/''/g"; }

package/templates/hooks/session-end.sh

@@ -50,6 +50,21 @@ if [ -f "$COORD_DB" ]; then
            OR (strftime('%s','now') - strftime('%s', last_heartbeat)) >= $STALE_SECS
       );
   " 2>/dev/null || true
+
+  # Release THIS agent's own holds immediately. The stale sweep above only frees
+  # an agent once its heartbeat has been dead for STALE_SECS — so a session that
+  # ended cleanly seconds after an edit kept its files locked against every peer
+  # for the full window, and peers saw a "live agent" that had already exited.
+  # We know our own id, so there is no reason to wait for it to look dead.
+  if [ -n "${UAP_AGENT_ID:-}" ]; then
+    ME_Q=$(printf '%s' "$UAP_AGENT_ID" | sed "s/'/''/g")
+    sqlite3 "$COORD_DB" "
+      UPDATE work_announcements SET completed_at = '$TIMESTAMP'
+      WHERE completed_at IS NULL AND agent_id = '$ME_Q';
+      UPDATE agent_registry SET status = 'completed'
+      WHERE id = '$ME_Q';
+    " 2>/dev/null || true
+  fi
 fi
 
 # Clean up backup files older than 7 days (retention policy)

package/templates/hooks/session-start.sh

@@ -146,9 +146,27 @@ fi
 GIT_BRANCH=$(git -C "$PROJECT_DIR" branch --show-current 2>/dev/null || echo "?")
 GIT_DIRTY=$(git -C "$PROJECT_DIR" status --porcelain 2>/dev/null | wc -l | tr -d ' ')
 
+# .worktrees/ lives in the MAIN checkout, not in a linked worktree — so when the
+# session starts INSIDE a worktree (the normal case under the worktree policy),
+# PROJECT_DIR has no .worktrees/ and the count silently read 0. COORD_ROOT is
+# already resolved from git-common-dir, i.e. the main checkout; reuse it.
 WORKTREE_COUNT=0
-if [ -d "${PROJECT_DIR}/.worktrees" ]; then
-  WORKTREE_COUNT=$(find "${PROJECT_DIR}/.worktrees" -maxdepth 1 -mindepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
+if [ -d "${COORD_ROOT}/.worktrees" ]; then
+  WORKTREE_COUNT=$(find "${COORD_ROOT}/.worktrees" -maxdepth 1 -mindepth 1 -type d 2>/dev/null | wc -l | tr -d ' ')
+fi
+
+# Worktree drift advisory. A count of worktrees says nothing about whether any of
+# them still hold work — silent accumulation is exactly how parallel-agent work
+# gets lost. Measured here before this line existed: 151 worktrees, worst 1241
+# commits behind origin/master, 23 holding unmerged commits. Purely advisory and
+# time-boxed, so a slow git call can never delay session start.
+WORKTREE_DRIFT=""
+if [ "$WORKTREE_COUNT" -gt 0 ] && command -v uap >/dev/null 2>&1; then
+  if command -v timeout >/dev/null 2>&1; then
+    WORKTREE_DRIFT=$(cd "$COORD_ROOT" 2>/dev/null && timeout 15 uap worktree hygiene --brief 2>/dev/null || true)
+  else
+    WORKTREE_DRIFT=$(cd "$COORD_ROOT" 2>/dev/null && uap worktree hygiene --brief 2>/dev/null || true)
+  fi
 fi
 
 PATTERN_COUNT=0
@@ -200,6 +218,10 @@ output+="- Worktree gate: run uap worktree ensure --strict (or uap worktree crea
 output+="- Backup gate: copy files to .uap-backups/$(date +%Y-%m-%d)/ before modification."$'\n'
 output+="- Coordination: agent=${AGENT_ID}; announce work, check overlaps, complete announcement when done."$'\n'
 output+="- Validation: run relevant build/tests for changed code before finalizing."$'\n'
+if [ -n "$WORKTREE_DRIFT" ]; then
+  output+="- ${WORKTREE_DRIFT}"$'\n'
+  output+="- Freshness: branch from the fetched remote tip; \`uap worktree sync\` before editing a file that moved upstream."$'\n'
+fi
 output+=""$'\n'
 output+="</system-reminder>"$'\n\n'