autonomous-sdlc-harness 0.4.0 → 0.4.2

package/templates/scripts/lib/harness-run-lib.sh

@@ -58,21 +58,27 @@
 #      and are written only by the `hr_lane_*` functions.
 #   2. THE RUN REGISTRY writes the registry file its caller names. Fence: that
 #      file is `<root>/<state_dir>/autonomous_logs/registry.json`, resolved
-#      through `hr_state_path`, and it is written only by `hr_registry_init`
-#      and `hr_registry_set`.
+#      through `hr_state_path`, plus the lock directory `<file>.lock` (and its
+#      `.stale.*` move-aside and `.break` mutex while a stale one is broken)
+#      and the temp files
+#      `.registry.*` in the registry's own directory. Written only by
+#      `hr_registry_init`, `hr_registry_set` and the `hr_registry_lock` /
+#      `hr_registry_unlock` pair `hr_registry_set` calls.
 #   3. THE REMOTE STATE BUNDLE writes the files its format lists. Fence: inside
 #      `<root>/<state_dir>/` (resolved through `hr_state_dir`), only
 #      `autonomous_logs/remote_status.json`, `clarifications/<branch>/`,
-#      `PAUSE_PROGRESS.md`, `.flow_walker_state` and the move-aside directory
-#      `autonomous_logs/remote_superseded/`; outside it, only the caller-named
-#      `<out_dir>` of `hr_remote_bundle_write` and the caller-named `<out_json>`
+#      `PAUSE_PROGRESS.md`, `.flow_walker_state`, the move-aside directory
+#      `autonomous_logs/remote_superseded/` and, only where nothing exists yet,
+#      files under the eight planning paths `hr_remote_planning_paths` assigns;
+#      outside it, only the caller-named `<out_dir>` (its `planning/` included)
+#      of `hr_remote_bundle_write` and the caller-named `<out_json>`
 #      of `hr_remote_status_write`. Written only by `hr_remote_status_write`,
 #      `hr_remote_bundle_write` and `hr_remote_bundle_restore`, and nothing
 #      there but a writer's own failed temp file is ever removed.
 #
 # A caller that calls no `hr_lane_*`, `hr_registry_init`, `hr_registry_set`,
-# `hr_remote_status_write`, `hr_remote_bundle_write` or
-# `hr_remote_bundle_restore` function still gets a library that only reads. The
+# `hr_registry_lock`, `hr_registry_unlock`, `hr_remote_status_write`,
+# `hr_remote_bundle_write` or `hr_remote_bundle_restore` function still gets a library that only reads. The
 # lane's ceilings are the only environment values here that carry policy, because
 # the lane is machine-scoped and has no configuration key to carry them; each is
 # named where it is used. `XDG_STATE_HOME`, `XDG_CONFIG_HOME`, `XDG_CACHE_HOME`,
@@ -165,7 +171,9 @@
 # because callers capture stdout. The one pass-through is the registry's two
 # writers, `hr_registry_init` and `hr_registry_set`, which leave the shell's,
 # `mktemp`'s and `jq`'s own stderr on a failed write to the caller, as the
-# watcher's bodies they replaced did — that stream is the watcher's log. That silence is why the lane reports a lock it BROKE through a
+# watcher's bodies they replaced did — that stream is the watcher's log — and
+# add one line of their own, naming the lock, when `hr_registry_set` cannot
+# take the registry lock. That silence is why the lane reports a lock it BROKE through a
 # variable instead of a log line — the caller owns the log.
 #
 # NAMING. Every function is prefixed `hr_`; every variable this file touches
@@ -177,7 +185,9 @@
 # `HR_LANE_RANK`, `HR_LANE_STATE`, `HR_LANE_RESUME_AT`, `HR_LANE_OBSERVED_AT`,
 # `HR_LANE_OBSERVED_REPO`, `HR_LANE_OWNER_SLUG`, `HR_LANE_OWNER_PID`,
 # `HR_LANE_OWNER_AT` and `HR_LANE_BROKEN_OWNER`, and the remote state bundle's
-# names, which `hr_remote_names_var` assigns. Every one of them is assigned
+# names, which `hr_remote_names_var` assigns, with `HR_REMOTE_PLANNING_PATHS`
+# (`hr_remote_planning_paths`) and `HR_REMOTE_PLANNING_PLACED` /
+# `HR_REMOTE_PLANNING_KEPT` (`hr_remote_bundle_restore`). Every one of them is assigned
 # before it is read by the function that owns it, so an inherited value from a
 # parent process is overwritten rather than believed.
 #
@@ -1110,31 +1120,187 @@ hr_push_env_files() {
 # Each function takes the registry file as its first argument — the caller
 # resolves it as `hr_state_path <root> autonomous_logs/registry.json` — and is
 # write exception 2 in the header. Every value is written as a JSON string;
-# every write stamps `branch` and `updated_at` on the record and replaces the
-# file through a `mktemp` + `mv`. No shell option is assumed: the caller may set
-# `-e`, `-u` or neither.
+# every write stamps `branch` and `updated_at` on the record. No shell option is
+# assumed: the caller may set `-e`, `-u` or neither.
+#
+# EVERY WRITE IS A LOCKED READ-MODIFY-WRITE. Every script sharing the file
+# writes it, and so does the watcher's own `( … ) &` engine subshell, so two
+# unserialized writers each read the same record and the second `mv` silently
+# drops the first one's key. The lock is a `mkdir` of `<file>.lock` holding an
+# `owner` file with a per-call token — the `mktemp` name of that call's temp
+# file, because `$$` inside a subshell is the parent's pid and bash 3.2 has no
+# per-subshell pid variable — and only a call whose token is still in `owner`
+# releases it.
+#
+# STALENESS IS THE LOCK DIRECTORY'S AGE, NEVER A PID. The stall watchdog kills
+# the engine subshell mid-write, so a dead holder is expected, and a pid says
+# nothing about a lock the killed subshell took under its parent's `$$`. A write
+# is one `jq` over a file of a few records, well under a second, so a lock 10 s
+# old is a crashed holder; a waiter polls every 0.05 s for up to 12 s — past the
+# stale age, so a waiter that arrives just after a crash breaks the lock rather
+# than giving up. Both are constants here, not environment values: the header
+# allows policy-carrying environment values for the lane only.
+#
+# BREAKERS ARE SERIALIZED, AND NONE RENAMES A LIVE LOCK. A waiter that judges
+# the lock stale takes a second `mkdir` mutex, `<file>.lock.break`, judges the
+# lock's age again under it, and only then renames the lock aside and empties
+# it, as `hr_lane_acquire` does. A lock re-taken since the first judgement is
+# fresh, so the second leaves it alone. Between the second judgement and the
+# rename, nothing but a breaker or the lock's own holder frees the path:
+# breakers are serialized, and the holder of a lock that old is dead by the
+# stale rule. The watchdog can kill a breaker inside the mutex, so a mutex 10 s
+# old is removed by the next waiter. That removal is the one window left open.
+# Two waiters that judge one abandoned mutex old together can both remove it,
+# and the second can remove the fresh mutex the first has just taken. Two
+# breakers then judge the lock at once, so reaching it takes a breaker killed
+# inside the mutex, a stale lock and three concurrent writers.
+#
+# A write that cannot take the lock prints one stderr line naming it, writes
+# nothing and returns 1.
+#
+# ONE CALL, SEVERAL KEYS. `hr_registry_set <file> <branch> <key> <value> [<key>
+# <value> …]` writes every pair in one `jq` pass under one lock and one `mv`.
+# A caller writing two keys that a concurrent reader must never see apart — a
+# status and the reason for it — MUST pass them in one call.
+#
+# READERS TAKE NO LOCK. The temp file sits beside the registry, so the `mv` is a
+# same-filesystem rename and `hr_registry_get`, `hr_registry_branches` and any
+# other `jq` over the file see the old record or the new one, never a partial
+# one. `hr_registry_init` creates the file by `ln`-ing a complete temp file to
+# the registry name, which fails when the name exists, so a reader's create
+# never truncates a registry a writer has just created.
 # ---------------------------------------------------------------------------
 
-# Create an empty registry at <file> when none exists.
+# The directory holding <file> — where its temp files go, so every `mv` over it
+# is a same-filesystem rename.
+hr_registry_dir() {
+  local file="${1-}" dir
+  case "$file" in
+    */*)
+      dir="${file%/*}"
+      [ -n "$dir" ] || dir=/
+      ;;
+    *) dir=. ;;
+  esac
+  printf '%s\n' "$dir"
+}
+
+# Create an empty registry at <file> when none exists. Atomic: the name is
+# either absent or a complete `{"runs":{}}` — never truncated, never partial.
 hr_registry_init() {
-  local file="${1-}"
+  local file="${1-}" dir tmp
   [ -n "$file" ] || return 1
-  [ -f "$file" ] || printf '{"runs":{}}\n' >"$file"
+  [ -f "$file" ] && return 0
+  dir=$(hr_registry_dir "$file")
+  tmp=$(mktemp "$dir/.registry.XXXXXX") || return 1
+  if printf '{"runs":{}}\n' >"$tmp"; then
+    ln "$tmp" "$file" 2>/dev/null || :
+  fi
+  rm -f "$tmp"
+  [ -f "$file" ]
+}
+
+# hr_registry_lock <file> <token> — take `<file>.lock` for <token>, breaking a
+# stale one (see the section comment). 0 when held; 1, with one stderr line,
+# when the wait ceiling passed.
+hr_registry_lock() {
+  local lock="${1-}.lock" token="${2-}" polls=0 start="" now m stale bm
+  # 12 — the wait ceiling and 10 — the stale age, both in seconds. The wait is
+  # judged by the clock, because each poll's forks cost more than its sleep;
+  # 240 polls is the ceiling only when `date` gives no epoch.
+  while :; do
+    if mkdir "$lock" 2>/dev/null; then
+      if printf '%s\n' "$token" >"$lock/owner" 2>/dev/null; then
+        return 0
+      fi
+      rm -f "$lock/owner" 2>/dev/null || :
+      rmdir "$lock" 2>/dev/null || :
+      break
+    fi
+    m=$(hr_lane_mtime "$lock")
+    now=$(date +%s 2>/dev/null) || now=0
+    case "$now" in '' | *[!0-9]*) now=0 ;; esac
+    [ -n "$start" ] || start="$now"
+    if [ "$now" -gt 0 ]; then
+      [ $((now - start)) -lt 12 ] || break
+    else
+      [ "$polls" -lt 240 ] || break
+    fi
+    if [ "$m" -gt 0 ] && [ "$now" -gt 0 ] && [ $((now - m)) -ge 10 ]; then
+      if mkdir "$lock.break" 2>/dev/null; then
+        # One breaker at a time: judge the age again under the mutex, so a
+        # lock re-taken since the judgement above is fresh and left alone.
+        m=$(hr_lane_mtime "$lock")
+        now=$(date +%s 2>/dev/null) || now=0
+        case "$now" in '' | *[!0-9]*) now=0 ;; esac
+        if [ "$m" -gt 0 ] && [ "$now" -gt 0 ] && [ $((now - m)) -ge 10 ]; then
+          stale="$lock.stale.${token##*.}"
+          if mv "$lock" "$stale" 2>/dev/null; then
+            rm -f "$stale/owner" 2>/dev/null || :
+            rmdir "$stale" 2>/dev/null || :
+          fi
+        fi
+        rmdir "$lock.break" 2>/dev/null || :
+        polls=$((polls + 1))
+        continue
+      fi
+      # Another breaker holds the mutex. One the watchdog killed inside it
+      # left it behind: a mutex 10 s old is removed here.
+      bm=$(hr_lane_mtime "$lock.break")
+      if [ "$bm" -gt 0 ] && [ $((now - bm)) -ge 10 ]; then
+        rmdir "$lock.break" 2>/dev/null || :
+      fi
+    fi
+    sleep 0.05
+    polls=$((polls + 1))
+  done
+  printf 'hr_registry_set: could not take the registry lock %s\n' "$lock" >&2
+  return 1
 }
 
-# hr_registry_set <file> <branch> <key> <value>
+# hr_registry_unlock <file> <token> — release `<file>.lock` only while its
+# `owner` still holds <token>; a lock broken and re-taken is someone else's.
+hr_registry_unlock() {
+  local lock="${1-}.lock" token="${2-}" owner=""
+  [ -r "$lock/owner" ] && { IFS= read -r owner <"$lock/owner" || :; } 2>/dev/null
+  [ -n "$token" ] && [ "$owner" = "$token" ] || return 0
+  rm -f "$lock/owner" 2>/dev/null || :
+  rmdir "$lock" 2>/dev/null || :
+}
+
+# hr_registry_set <file> <branch> <key> <value> [<key> <value> …]
+# 1, writing nothing, on no key/value pair or an odd count of them. Each pair
+# reaches `jq` as `--arg kN` / `--arg vN` — jq 1.5 has no `$ARGS` — and only
+# those generated names enter the program text, never a value.
 hr_registry_set() {
-  local file="${1-}" branch="${2-}" key="${3-}" value="${4-}" tmp
-  hr_registry_init "$file" || :
-  tmp="$(mktemp)" || return 1
-  if jq --arg b "$branch" --arg k "$key" --arg v "$value" --arg now "$(date '+%Y-%m-%dT%H:%M:%S')" '
-    .runs[$b] = ((.runs[$b] // {}) + {($k): $v, "branch": $b, "updated_at": $now})
-  ' "$file" >"$tmp"; then
-    mv "$tmp" "$file"
-  else
+  local file="${1-}" branch="${2-}" dir tmp fields="" i=0 status=0
+  local -a args
+  [ -n "$file" ] && [ "$#" -ge 4 ] && [ $(($# % 2)) -eq 0 ] || return 1
+  shift 2
+  args=(--arg b "$branch" --arg now "$(date '+%Y-%m-%dT%H:%M:%S')")
+  while [ "$#" -gt 0 ]; do
+    args+=(--arg "k$i" "$1" --arg "v$i" "$2")
+    fields="$fields(\$k$i): \$v$i, "
+    i=$((i + 1))
+    shift 2
+  done
+  dir=$(hr_registry_dir "$file")
+  tmp=$(mktemp "$dir/.registry.XXXXXX") || return 1
+  if ! hr_registry_lock "$file" "$tmp"; then
     rm -f "$tmp"
     return 1
   fi
+  hr_registry_init "$file" || :
+  if jq "${args[@]}" "
+    .runs[\$b] = ((.runs[\$b] // {}) + {$fields\"branch\": \$b, \"updated_at\": \$now})
+  " "$file" >"$tmp"; then
+    mv "$tmp" "$file" || status=1
+  else
+    status=1
+  fi
+  [ "$status" -eq 0 ] || rm -f "$tmp"
+  hr_registry_unlock "$file" "$tmp"
+  return "$status"
 }
 
 # hr_registry_get <file> <branch> <key>   -> the value, or nothing
@@ -1165,18 +1331,43 @@ hr_registry_branches() {
 #   <bundle>/PAUSE_PROGRESS.md           when present
 #   <bundle>/flow_walker_state           <state_dir>/.flow_walker_state, WITHOUT its dot
 #   <bundle>/run.log                     <state_dir>/autonomous_logs/<branch>.log; never restored
+#   <bundle>/planning/<path>             each of these under <state_dir>, when present:
+#                                          story_plans/<branch>_story_plan.md
+#                                          task_plans/<branch>
+#                                          ui_test_plans/<branch>_ui_test_plan.md
+#                                          ui_test_plans/<branch>
+#                                          task_plan_reviews/<branch>
+#                                          business_parity_reviews/<branch>
+#                                          architecture_reviews/<branch>
+#                                          ui_test_plan_reviews/<branch>
 #
 # The walker state loses its dot because `actions/upload-artifact` skips hidden
-# files by default. NOTHING IN THE BUNDLE IS EVER COMMITTED: every file in it is
-# gitignored machine-local state, and the remote-status and move-aside paths sit
-# under `autonomous_logs/`, whose ignore rule already covers them.
+# files by default. THE BUNDLE ITSELF IS NEVER COMMITTED. Every file outside
+# `planning/` is gitignored machine-local state, and the remote-status and
+# move-aside paths sit under `autonomous_logs/`, whose ignore rule already
+# covers them. The files under `planning/` are untracked drafts that the flow
+# commits itself at its P1/P3 convergence; the bundle only carries them.
+#
+# THE PLANNING PATHS ARE A MIRROR of the contracts that write and stage them:
+# `plugin/instructions/task_plan_writing_instructions_autonomous.md` →
+# `## Override 3` and `## Override 4` staging lists, and
+# `<scripts_dir>/flows/task_plan_writing.graph.json` → each node's `findingsFolder`. A path
+# added there is an edit to `hr_remote_planning_paths`. Only planning is
+# carried because the walker's one graph is `task_plan_writing.graph.json`;
+# implementation-phase per-unit review folders never span a pause, which waits
+# for a clean tracked tree. `HR_REMOTE_STATE_SCHEMA` stays `'1'`: `planning/`
+# is additive, an older reader ignores it, and a newer reader of an older
+# bundle finds none.
 #
 # WHO READS EACH FILE. `status.json`: `remote-run.sh sync` / `status` (into the
 # local registry), `continue` / `poll` (the decision, `chain`, the reset time)
 # and the next job's seed. The clarification directory and `PAUSE_PROGRESS.md`:
 # the next job, and the user's local mirror. The walker state: the next job
-# only. `run.log`: the user only — `sync` copies it to the main checkout's logs
-# directory itself, and no restore places it.
+# only. `planning/`: the next job only, never a mirror — an untracked draft left
+# in the mirror would make its later fast-forward to `origin/<branch>` refuse,
+# because the draft's own convergence commit adds the same path. `run.log`: the
+# user only — `sync` copies it to the main checkout's logs directory itself,
+# and no restore places it.
 #
 # `status.json` — schema `HR_REMOTE_STATE_SCHEMA`; every value a JSON string:
 #   schema                  a reader that does not recognise it treats the bundle as absent
@@ -1210,6 +1401,27 @@ hr_remote_names_var() {
   HR_REMOTE_LOGS_DIR='autonomous_logs'
   HR_REMOTE_STATUS_SOURCE="$HR_REMOTE_LOGS_DIR/remote_status.json"
   HR_REMOTE_SUPERSEDED_DIR="$HR_REMOTE_LOGS_DIR/remote_superseded"
+  HR_REMOTE_PLANNING_DIR='planning'
+}
+
+# hr_remote_planning_paths <branch>
+#
+# Assigns `HR_REMOTE_PLANNING_PATHS`: the eight planning paths of the format
+# above, relative to <state_dir>, newline-separated, no trailing slash. 1 with
+# an empty value for an empty <branch>.
+hr_remote_planning_paths() {
+  local branch="${1-}"
+  HR_REMOTE_PLANNING_PATHS=''
+  [ -n "$branch" ] || return 1
+  HR_REMOTE_PLANNING_PATHS="story_plans/${branch}_story_plan.md
+task_plans/$branch
+ui_test_plans/${branch}_ui_test_plan.md
+ui_test_plans/$branch
+task_plan_reviews/$branch
+business_parity_reviews/$branch
+architecture_reviews/$branch
+ui_test_plan_reviews/$branch"
+  return 0
 }
 
 # hr_remote_status_write <registry_file> <branch> <out_json> <decision> <detail>
@@ -1313,10 +1525,12 @@ hr_remote_status_get() {
 # <root>'s configured state directory. `status.json` is the job's own
 # `autonomous_logs/remote_status.json` when present; otherwise it is written
 # from <branch>'s registry record with decision `stop`, because a job that never
-# wrote its status never decided to continue. 0 written; 1 a missing argument,
+# wrote its status never decided to continue. Each planning path present is
+# copied under `planning/`, whether or not the branch tracks it: the restore
+# never overwrites, so a tracked copy is inert. 0 written; 1 a missing argument,
 # a non-empty <out_dir> or a failed copy; 2 <root>'s configuration unresolvable.
 hr_remote_bundle_write() {
-  local root="${1-}" branch="${2-}" registry="${3-}" out="${4-}" state base clarify
+  local root="${1-}" branch="${2-}" registry="${3-}" out="${4-}" state base clarify rel dst
   [ -n "$root" ] && [ -n "$branch" ] && [ -n "$registry" ] && [ -n "$out" ] || return 1
   state=$(hr_state_dir "$root") || return 2
   hr_remote_names_var
@@ -1349,15 +1563,36 @@ hr_remote_bundle_write() {
   if [ -f "$base/$HR_REMOTE_LOGS_DIR/$branch.log" ]; then
     cp "$base/$HR_REMOTE_LOGS_DIR/$branch.log" "$out/$HR_REMOTE_LOG_FILE" 2>/dev/null || return 1
   fi
+  hr_remote_planning_paths "$branch" || return 1
+  while IFS= read -r rel; do
+    dst="$out/$HR_REMOTE_PLANNING_DIR/$rel"
+    if [ -f "$base/$rel" ]; then
+      mkdir -p "${dst%/*}" 2>/dev/null || return 1
+      cp "$base/$rel" "$dst" 2>/dev/null || return 1
+    elif [ -d "$base/$rel" ]; then
+      mkdir -p "${dst%/*}" 2>/dev/null || return 1
+      cp -R "$base/$rel" "$dst" 2>/dev/null || return 1
+    fi
+  done <<EOF
+$HR_REMOTE_PLANNING_PATHS
+EOF
   return 0
 }
 
 # hr_remote_bundle_restore <bundle_dir> <root> <branch> <mode>
 #
 # <mode> `job` places the clarification directory, `PAUSE_PROGRESS.md`, the
-# walker state (back under its dotted name) and `status.json` (as
-# `autonomous_logs/remote_status.json`); `mirror` places the first two only. The
-# run log is placed by neither.
+# walker state (back under its dotted name), the planning drafts and
+# `status.json` (as `autonomous_logs/remote_status.json`); `mirror` places the
+# first two only. The run log is placed by neither.
+#
+# A PLANNING DRAFT NEVER OVERWRITES. A regular file under `planning/` whose
+# path lies in `HR_REMOTE_PLANNING_PATHS` and has no `..` segment is placed only
+# where nothing exists, counted in `HR_REMOTE_PLANNING_PLACED`; one whose target
+# exists is left byte-identical — the checkout's copy is the branch's committed
+# record — and counted in `HR_REMOTE_PLANNING_KEPT`. A symlink, a non-regular
+# entry or a path outside the set counts in neither. Both are `0` at entry and
+# stay `0` in `mirror` mode.
 #
 # THE CLARIFICATION DIRECTORY IS REPLACED WHOLESALE, AND NOTHING IS DELETED. An
 # existing target is moved aside with one `mv` into
@@ -1373,7 +1608,9 @@ hr_remote_bundle_write() {
 # <branch>) or <root>'s configuration is unresolvable.
 hr_remote_bundle_restore() {
   local bundle="${1-}" root="${2-}" branch="${3-}" mode="${4-}"
-  local state base named target epoch aside n tmp
+  local state base named target epoch aside n tmp pdir file rel p inset
+  HR_REMOTE_PLANNING_PLACED=0
+  HR_REMOTE_PLANNING_KEPT=0
   [ -n "$bundle" ] && [ -n "$root" ] && [ -n "$branch" ] || return 1
   case "$mode" in
     job|mirror) ;;
@@ -1413,6 +1650,37 @@ hr_remote_bundle_restore() {
     mkdir -p "$base" 2>/dev/null || return 1
     cp "$bundle/$HR_REMOTE_WALKER_FILE" "$base/$HR_REMOTE_WALKER_SOURCE" 2>/dev/null || return 1
   fi
+  pdir="$bundle/$HR_REMOTE_PLANNING_DIR"
+  if [ -d "$pdir" ] && [ ! -L "$pdir" ]; then
+    hr_remote_planning_paths "$branch" || return 1
+    # Process substitution, not a pipe: the loop must run in this shell so the
+    # counters and `return 1` reach the caller.
+    while IFS= read -r file; do
+      # Re-tested per line: a name holding a newline arrives split and fails here.
+      [ -f "$file" ] && [ ! -L "$file" ] || continue
+      rel=${file#"$pdir"/}
+      case "/$rel/" in
+        */../*) continue ;;
+      esac
+      inset=0
+      while IFS= read -r p; do
+        case "$rel" in
+          "$p"|"$p"/*) inset=1; break ;;
+        esac
+      done <<EOF
+$HR_REMOTE_PLANNING_PATHS
+EOF
+      [ "$inset" = 1 ] || continue
+      if [ -e "$base/$rel" ] || [ -L "$base/$rel" ]; then
+        HR_REMOTE_PLANNING_KEPT=$((HR_REMOTE_PLANNING_KEPT + 1))
+        continue
+      fi
+      target="$base/$rel"
+      mkdir -p "${target%/*}" 2>/dev/null || return 1
+      cp "$file" "$target" 2>/dev/null || return 1
+      HR_REMOTE_PLANNING_PLACED=$((HR_REMOTE_PLANNING_PLACED + 1))
+    done < <(find "$pdir" -type f 2>/dev/null)
+  fi
   mkdir -p "$base/$HR_REMOTE_LOGS_DIR" 2>/dev/null || return 1
   tmp=$(mktemp "$base/$HR_REMOTE_STATUS_SOURCE.tmp.XXXXXX" 2>/dev/null) || return 1
   if cp "$bundle/$HR_REMOTE_STATUS_FILE" "$tmp" 2>/dev/null \

package/templates/scripts/remote-run.sh

@@ -71,11 +71,15 @@
 # past a run with none, and stopping at one whose artifact has expired, since
 # an older copy would be staler state. An expired one restores nothing: under
 # --resume answer it exits 2; otherwise it prints a `::warning::` line naming
-# the run, the expiry and the lost counts and clarification history, and the
-# job continues from the committed ledger. An unexpired one it downloads to `<state_dir>/autonomous_logs/remote_download/<branch>/<id>/`
+# the run, the expiry and the lost counts, clarification history and
+# uncommitted planning drafts, and the job continues from the committed ledger. An unexpired one it downloads to `<state_dir>/autonomous_logs/remote_download/<branch>/<id>/`
 # (skipped when that directory already holds its status.json); and restores it
 # in `job` mode — on every --resume kind, `none` included, because a reused
-# branch keeps its clarification history. Then, under --resume answer, it
+# branch keeps its clarification history. A job-mode restore also places the
+# bundle's `planning/` drafts at their paths under the state directory, never
+# over a file the checkout already has, and when it placed or kept any prints
+# `placed <n> planning file(s) for <branch>; kept <n> the checkout already
+# carries`. Then, under --resume answer, it
 # writes each `"<n>": "<text>"` entry to `clarifications/<branch>/answer_<n>.md`
 # with the exact bytes, after checking every entry first; and with
 # `HARNESS_INPUT_PARK_LOOP_CLEAR` exactly `true` it sets `park_loop_cycles` to
@@ -230,7 +234,8 @@
 #      `mirror` mode into the record's `worktree`, `run.log` copied to the main
 #      checkout's `autonomous_logs/<branch>.remote.log`, and `status`,
 #      `pause_reason`, `usage_resume_at`, `park_loop_cycles`, `remote_run_id`,
-#      `remote_run_url`, `remote_detail` and `remote_synced_at` written
+#      `remote_run_url`, `remote_detail` and `remote_synced_at` written. A
+#      `mirror` restore places no planning draft
 #   4. no artifact, while some bundle exists (`remote_run_id` is set, or an
 #      older finished run carries one): a job that died before its upload.
 #      `paused` / `killed`, `remote_run_id` / `remote_run_url` re-pointed at
@@ -276,7 +281,7 @@
 # no job to stop and may complete before its cancel lands — trying every one even
 # after a failure. (3) Only when (1) and (2) all succeeded, and only when a
 # local registry record exists, it writes `remote_stopped_at` and sets `status`
-# to `failed` through `hr_registry_set`; a partial stop leaves the record alone
+# to `failed` in one `hr_registry_set` call; a partial stop leaves the record alone
 # and exits 3, so running `stop` again is the remedy.
 #
 # `warm` dispatches action=warm on GitHub's OWN default branch (`gh repo view
@@ -294,7 +299,8 @@
 # `<state_dir>/autonomous_logs/remote_download/<branch>/<id>/` and
 # `<branch>.remote.log` in the main checkout, plus the mirror restore
 # `hr_remote_bundle_restore` performs in the record's `worktree`; for
-# `restore`, that download directory, the job restore, `answer_<n>.md` and the
+# `restore`, that download directory, the job restore (the planning drafts
+# among it), `answer_<n>.md` and the
 # `park_loop_cycles` rewrite of `remote_status.json`, all in the job's
 # checkout; for `save`, <out_dir> and the step summary; for `poll`, its
 # download directories and `<state_dir>/autonomous_logs/poll_state/previous/`
@@ -363,7 +369,8 @@
 #   flow_walker_state) and GITHUB_RUN_ID set to another id:
 #   restore    bash scripts/remote-run.sh restore feat_x --resume none   -> 0; the
 #              checkout carries clarifications/feat_x/question_1.md,
-#              .flow_walker_state and autonomous_logs/remote_status.json
+#              .flow_walker_state and autonomous_logs/remote_status.json, and
+#              places the bundle's planning/ drafts where the checkout has none
 #   answer     HARNESS_INPUT_ANSWERS='{"1":"Use B.\n"}' ... --resume answer -> 0;
 #              clarifications/feat_x/answer_1.md holds exactly `Use B.` + newline
 #   no question  HARNESS_INPUT_ANSWERS='{"2":"x"}' ... --resume answer
@@ -381,7 +388,7 @@
 #              -> prints the expired line, "$r" byte-identical
 #   save       bash scripts/remote-run.sh save feat_x /tmp/b -> 0; /tmp/b holds
 #              status.json, clarifications/feat_x/, flow_walker_state (and
-#              PAUSE_PROGRESS.md, run.log when present); with
+#              PAUSE_PROGRESS.md, run.log, planning/ when present); with
 #              GITHUB_STEP_SUMMARY=/tmp/s, /tmp/s gains the status table
 #   never started  no remote_status.json and no registry: save -> 0, /tmp/b
 #              empty
@@ -811,8 +818,7 @@ verb_stop() {
 
   registry=$(hr_state_path "$root" autonomous_logs/registry.json) || registry=""
   if [ -n "$registry" ] && [ -f "$registry" ] && [ -n "$(hr_registry_get "$registry" "$branch" branch)" ]; then
-    hr_registry_set "$registry" "$branch" remote_stopped_at "$stopped_at" \
-      && hr_registry_set "$registry" "$branch" status failed \
+    hr_registry_set "$registry" "$branch" remote_stopped_at "$stopped_at" status failed \
       || echo "remote-run.sh: stopped on GitHub, but the local record of $branch could not be updated" >&2
   fi
   echo "remote-run.sh: stopped $branch"
@@ -904,6 +910,17 @@ set_or_fail() {
   }
 }
 
+# set_many_or_fail <key> <value> [<key> <value> …] — every pair in one write, so
+# a concurrent reader never sees an outcome half-applied.
+set_many_or_fail() {
+  local keys="" i
+  hr_registry_set "$registry" "$branch" "$@" || {
+    for ((i = 1; i <= $#; i += 2)); do keys="$keys${keys:+, }${!i}"; done
+    echo "remote-run.sh: writing $keys of $branch to '$registry' failed" >&2
+    exit "$EXIT_USAGE"
+  }
+}
+
 verb_status() {
   local runs finished_id synced_id field
   list_runs
@@ -935,18 +952,14 @@ verb_status() {
 sync_expired() {
   local line
   line=$(expired_line "$1")
-  set_or_fail status paused
-  set_or_fail pause_reason expired
-  set_or_fail remote_run_id "$1"
-  set_or_fail remote_run_url "$2"
-  set_or_fail remote_detail "$line"
-  set_or_fail remote_synced_at "$3"
+  set_many_or_fail status paused pause_reason expired remote_run_id "$1" \
+    remote_run_url "$2" remote_detail "$line" remote_synced_at "$3"
   echo "remote-run.sh: $line"
 }
 
 verb_sync() {
   local worktree runs newest id state url synced_id now older download status_file
-  local status reason detail
+  local status reason detail resume_at cycles
   worktree=$(hr_registry_get "$registry" "$branch" worktree)
   if [ -z "$worktree" ] || [ ! -d "$worktree" ]; then
     echo "remote-run.sh: refused, nothing written: the mirror working copy '$worktree' of $branch is missing" >&2
@@ -965,8 +978,7 @@ verb_sync() {
   now=$(date +%s)
 
   if [ "$state" != completed ]; then
-    set_or_fail status running
-    set_or_fail remote_synced_at "$now"
+    set_many_or_fail status running remote_synced_at "$now"
     echo "remote-run.sh: run $id of $branch is $state; the record is running, nothing downloaded"
     return 0
   fi
@@ -1039,14 +1051,11 @@ verb_sync() {
       detail="the job ended mid-run (its bundle still says running): $url"
     fi
     [ -n "$detail" ] || detail="synced from $url"
-    set_or_fail status "$status"
-    set_or_fail pause_reason "$reason"
-    set_or_fail usage_resume_at "$(hr_remote_status_get "$status_file" usage_resume_at || :)"
-    set_or_fail park_loop_cycles "$(hr_remote_status_get "$status_file" park_loop_cycles || :)"
-    set_or_fail remote_run_id "$id"
-    set_or_fail remote_run_url "$url"
-    set_or_fail remote_detail "$detail"
-    set_or_fail remote_synced_at "$now"
+    resume_at=$(hr_remote_status_get "$status_file" usage_resume_at) || resume_at=""
+    cycles=$(hr_remote_status_get "$status_file" park_loop_cycles) || cycles=""
+    set_many_or_fail status "$status" pause_reason "$reason" usage_resume_at "$resume_at" \
+      park_loop_cycles "$cycles" remote_run_id "$id" remote_run_url "$url" \
+      remote_detail "$detail" remote_synced_at "$now"
     echo "remote-run.sh: synced run $id of $branch: $status${reason:+ ($reason)}"
     return 0
   fi
@@ -1061,20 +1070,17 @@ verb_sync() {
     done
   fi
   if [ "$bundle_exists" -eq 1 ]; then
-    set_or_fail status paused
-    set_or_fail pause_reason killed
-    set_or_fail remote_run_id "$id"
-    set_or_fail remote_run_url "$url"
-    set_or_fail remote_detail "run $id ended with no state bundle (killed, cancelled or replaced): $url"
-    set_or_fail remote_synced_at "$now"
+    set_many_or_fail status paused pause_reason killed remote_run_id "$id" remote_run_url "$url" \
+      remote_detail "run $id ended with no state bundle (killed, cancelled or replaced): $url" \
+      remote_synced_at "$now"
     echo "remote-run.sh: run $id of $branch left no bundle; the record is paused (killed), nothing restored"
     return 0
   fi
 
   # Case 5 — no bundle in any run.
-  set_or_fail status failed
-  set_or_fail remote_detail "no run of $branch ever uploaded a state bundle; newest: $url"
-  set_or_fail remote_synced_at "$now"
+  set_many_or_fail status failed \
+    remote_detail "no run of $branch ever uploaded a state bundle; newest: $url" \
+    remote_synced_at "$now"
   echo "remote-run.sh: no run of $branch carries a state bundle; the record is failed"
 }
 
@@ -1141,7 +1147,7 @@ verb_restore() {
   if [ "$PREV_RUN_STATE" = expired ]; then
     [ "$resume" != answer ] \
       || restore_refuse "the state bundle of run $id expired on $BUNDLE_EXPIRES_AT, so its questions can no longer be answered here: resume from the committed ledger with $RESUME_HINT $branch, or re-drop the task; nothing written"
-    echo "::warning::remote-run.sh: the state bundle of run $id expired on $BUNDLE_EXPIRES_AT: the park-loop, auto-resume and stall counts and the clarification history it carried are lost; this job continues from the committed ledger"
+    echo "::warning::remote-run.sh: the state bundle of run $id expired on $BUNDLE_EXPIRES_AT: the park-loop, auto-resume and stall counts, the clarification history and any planning drafts not yet committed that it carried are lost; this job continues from the committed ledger"
   elif [ -z "$id" ]; then
     [ "$resume" != answer ] \
       || restore_refuse "--resume answer, but no finished run of $branch carries a state bundle; nothing written"
@@ -1164,7 +1170,11 @@ verb_restore() {
     fi
     hr_remote_bundle_restore "$download" "$root" "$branch" job
     case $? in
-      0) echo "remote-run.sh: restored the bundle of run $id into $root" ;;
+      0)
+        echo "remote-run.sh: restored the bundle of run $id into $root"
+        [ $((HR_REMOTE_PLANNING_PLACED + HR_REMOTE_PLANNING_KEPT)) -eq 0 ] \
+          || echo "remote-run.sh: placed $HR_REMOTE_PLANNING_PLACED planning file(s) for $branch; kept $HR_REMOTE_PLANNING_KEPT the checkout already carries"
+        ;;
       2) restore_refuse "the bundle in '$download' is unrecognised for $branch; nothing restored" ;;
       *) restore_fail "restoring '$download' into '$root' failed" ;;
     esac

package/templates/state-dir/autonomous_logs/README.md

@@ -1,6 +1,6 @@
 # autonomous_logs/
 
-One readable transcript `<branch>.log` and one raw event log `<branch>.stream.jsonl` per dispatched run, plus the daemon's own `watcher.log`, the service manager's capture of its standard output and error as `watcher.out.log` and `watcher.err.log`, and `registry.json` — the run registry that indexes every run's status. All of it resolves to `<state_dir>/autonomous_logs/` **in the main checkout**, whichever sibling working copy a run actually executes in, so one repository has one set of these files however many runs it has in flight.
+One readable transcript `<branch>.log` and one raw event log `<branch>.stream.jsonl` per dispatched run, plus the daemon's own `watcher.log`, the service manager's capture of its standard output and error as `watcher.out.log` and `watcher.err.log`, and `registry.json` — the run registry that indexes every run's status. All of it resolves to `<state_dir>/autonomous_logs/` **in the main checkout**, whichever sibling working copy a run actually executes in, so one repository has one set of these files however many runs it has in flight. A `registry.json.lock` directory beside the registry exists only for the width of one registry write; one left over after a crash is broken automatically by the next writer, and is never removed by hand while a watcher runs.
 
 Everything here is written by the run daemon and read by the operator watching or diagnosing a run: the transcript is formatted live and tailable while the run is going, and the raw stream beside it is the deep-debug copy and the one thing the usage gate can parse, since a run cannot read the stream it is itself producing. The registry has readers beyond the operator, and that is what makes it different in kind from its neighbours: it is the single record of which runs exist and what state each is in, and the harness's status command, its pause and resume commands, `<scripts_dir>/restart-watcher.sh` and `<scripts_dir>/cleanup-merged-worktrees.sh` all read it before they act, and `<scripts_dir>/remote-run.sh` reads and writes it for a run executing on GitHub Actions.