muse-crew 0.4.5 → 0.4.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/lib/AGENTS.md CHANGED
@@ -4,12 +4,12 @@ Shell scripts for the crew's infrastructure. Called by workflow scripts, cron, a
4
4
 
5
5
  - `build-registry.js` — deterministic extractor that generates `workflows/registry.json` (workflow step registry) from the workflow files' `meta` blocks at release time; invoked by `crew-release.sh` deploy
6
6
  - `crew-release.sh` — immutable release manager: deploy, rollback, prune
7
- - `merge-lock.sh` — serialized merge lock for concurrent agents; records owner PID
7
+ - `merge-lock.sh` — serialized merge lock for concurrent agents: time-based holder lease (bug 2fc8f52f — an unexpired lease is held regardless of process liveness; only an expired lease may be broken). Lock file is key=value: task_id, opaque holder identity (never a PID), acquired_at epoch, lease_seconds (default 600, override via MERGE_LOCK_LEASE_SECONDS). acquire/refresh/release/status/force-release; holder-only refresh and release; every op appends to $CREW_HOME/.merge-lock.log
8
8
  - `worktree-lifecycle.sh` — the worktree lifecycle seam: prepare/cleanup/inspect/integrate/status/post-deploy over git worktrees (`.worktrees/<id>`, branch `task/<id>`). The crew registry (`<repo>/.worktrees/.registry/<id>`) is the source of truth for task→branch/path — never reconstruct it from git state. Prepare fails closed on dirty `main`; cleanup is forgiving.
9
9
  - `test-worktree-backend.sh` — regression tests for the lifecycle script (validate, prepare/reuse, inspect, status, cleanup, idempotent cleanup, dirty-main preflight) on a scratch repo
10
10
  - `test-version-write.sh` — regression tests for the escape-preserving step-8 version write in publish-npm.sh (fixture: current package.json with the \u2014 escape; extracts the shipped block by anchor)
11
11
  - `test-publish-verify.sh` — regression tests for the retry-tolerant step-12 verification in publish-npm.sh (canary 5a027278): extracts the shipped block by anchor and runs it against a fake npm whose read replica lags (non-zero exits, then the old version, then the target) — requires convergence on success, fail-closed `PUBLISH_FAILED=verify` on exhaustion, and `--prefer-online` on every read
12
- - `orphan-sweep.sh` — find and clean stale worktrees and merge locks
12
+ - `orphan-sweep.sh` — find and clean stale worktrees and merge locks: parses `merge-lock.sh status` key=value fields (never a PID); a lock whose task has a running session is untouched regardless of lease age; a lock with no running session is broken only when its lease is expired (via merge-lock.sh release so the break lands in the audit log)
13
13
  - `publish-npm.sh` — deterministic npm publish: lock refresh, release install, version write/commit, pack, registry publish, verify, push, post-deploy. Takes TARGET_VERSION as input; idempotent on retry/resume.
14
14
  - `compose-evidence.py` — deterministic visual-evidence compositor (Pillow): pairs identical PNG stems from baseline/ and postchange/ dirs, emits `<stem>-sidebyside.png` and amplified-difference `<stem>-overlay.png` into composites/, prints `PAIR`/`SKIP` manifests. Byte-deterministic; nonzero exit on errors.
15
15
  - `test-orphan-sweep.sh` — regression tests for orphan-sweep.sh (active-run guard, verified removal, fail-closed)
package/lib/merge-lock.sh CHANGED
@@ -1,16 +1,89 @@
1
1
  #!/usr/bin/env bash
2
- # Serialized merge lock for the orchestra-dashboard repo.
2
+ # merge-lock.sh — serialized merge lock for the crew's repos.
3
+ #
4
+ # Time-based holder lease (bug 2fc8f52f): the old PID-liveness staleness
5
+ # check never excluded anyone. The recorded PID was the short-lived shell
6
+ # that ran the acquire, so for essentially the whole Integrate→Publish→
7
+ # post-deploy hold window the holder's PID was dead, every concurrent
8
+ # acquire took the STALE path, and two Publish phases both believed they
9
+ # owned version assignment. PID is the wrong liveness signal (orphan-sweep
10
+ # already treats it as such) — the lease is time-based and refreshed by
11
+ # the workflow while the hold is live.
12
+ #
13
+ # The lock file ($CREW_REPO/.worktrees/.merge-lock) is key=value:
14
+ # task_id=<task id that holds the lock>
15
+ # holder=<opaque task+run identity, never a PID>
16
+ # acquired_at=<epoch seconds; refresh rewrites it to extend the lease>
17
+ # lease_seconds=<lease window; MERGE_LOCK_LEASE_SECONDS, default 600>
18
+ #
19
+ # The lock is HELD while now - acquired_at < lease_seconds. An expired
20
+ # (or malformed) lease is broken with logging and re-acquired. Every op
21
+ # appends one line to $CREW_HOME/.merge-lock.log (the audit trail).
22
+ #
3
23
  # Usage:
4
- # merge-lock.sh acquire <task_id> [pid]acquires lock, exits 0 on success, 1 if held
5
- # merge-lock.sh release <task_id> releases lock if held by this task
6
- # merge-lock.sh refresh <task_id> [pid] refresh lock timestamp (resets staleness clock)
7
- # merge-lock.sh status prints current lock holder or "unlocked"
8
- # merge-lock.sh force-release breaks the lock (admin use only)
24
+ # merge-lock.sh acquire <task_id> <holder>acquire the lock (0 ok, 1 held)
25
+ # merge-lock.sh refresh <task_id> extend the lease (holder task only)
26
+ # merge-lock.sh release <task_id> release the lock (holder task only)
27
+ # merge-lock.sh status key=value holder/age/remaining, or UNLOCKED
28
+ # merge-lock.sh force-release break the lock (admin use only)
9
29
 
10
30
  set -euo pipefail
11
31
 
12
32
  LOCK_DIR="${CREW_REPO:-$HOME/workspace/ts-spaces/orchestra-dashboard}/.worktrees"
13
33
  LOCK_FILE="$LOCK_DIR/.merge-lock"
34
+ LOG_FILE="${CREW_HOME:-$HOME/workspace/.jarvis}/.merge-lock.log"
35
+ LEASE_SECONDS="${MERGE_LOCK_LEASE_SECONDS:-600}"
36
+
37
+ log_op() { # log_op <op> <detail> — audit trail; logging never breaks a lock op
38
+ mkdir -p "$(dirname "$LOG_FILE")" 2>/dev/null || true
39
+ printf '%s op=%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$1" "$2" >> "$LOG_FILE" 2>/dev/null || true
40
+ }
41
+
42
+ # read_lock — parse the lock file into lock_task_id / lock_holder /
43
+ # lock_acquired_at / lock_lease_seconds. Returns 1 when there is no lock.
44
+ read_lock() {
45
+ lock_task_id=""; lock_holder=""; lock_acquired_at=""; lock_lease_seconds=""
46
+ [ -f "$LOCK_FILE" ] || return 1
47
+ local key val
48
+ while IFS='=' read -r key val; do
49
+ case "$key" in
50
+ task_id) lock_task_id="$val" ;;
51
+ holder) lock_holder="$val" ;;
52
+ acquired_at) lock_acquired_at="$val" ;;
53
+ lease_seconds) lock_lease_seconds="$val" ;;
54
+ esac
55
+ done < "$LOCK_FILE"
56
+ return 0
57
+ }
58
+
59
+ lock_age() { # seconds since acquired_at, or "unknown" when unparsable
60
+ if [[ "${lock_acquired_at:-}" =~ ^[0-9]+$ ]]; then
61
+ echo $(( $(date +%s) - lock_acquired_at ))
62
+ else
63
+ echo "unknown"
64
+ fi
65
+ }
66
+
67
+ lock_remaining() { # lease seconds remaining (<=0 means expired), or "unknown"
68
+ local age="${1:-$(lock_age)}" lease="$lock_lease_seconds"
69
+ [[ "$lease" =~ ^[0-9]+$ ]] || lease="$LEASE_SECONDS"
70
+ if [ "$age" = "unknown" ]; then
71
+ echo "unknown"
72
+ else
73
+ echo $(( lease - age ))
74
+ fi
75
+ }
76
+
77
+ # write_lock <task_id> <holder> — atomic create via noclobber; 0 on success.
78
+ # A concurrent writer wins exactly one create; the loser retries the read.
79
+ write_lock() {
80
+ local task_id="$1" holder="$2"
81
+ mkdir -p "$LOCK_DIR"
82
+ ( set -C
83
+ printf 'task_id=%s\nholder=%s\nacquired_at=%s\nlease_seconds=%s\n' \
84
+ "$task_id" "$holder" "$(date +%s)" "$LEASE_SECONDS" > "$LOCK_FILE"
85
+ ) 2>/dev/null
86
+ }
14
87
 
15
88
  cmd="${1:-}"
16
89
  task_id="${2:-}"
@@ -18,90 +91,98 @@ task_id="${2:-}"
18
91
  case "$cmd" in
19
92
  acquire)
20
93
  [ -z "$task_id" ] && { echo "ERROR: task_id required"; exit 1; }
21
- owner_pid="${3:-$PPID}"
22
- mkdir -p "$LOCK_DIR"
23
- if (set -C; echo "$task_id $(date -u +%Y-%m-%dT%H:%M:%SZ) $owner_pid" > "$LOCK_FILE") 2>/dev/null; then
24
- echo "ACQUIRED by $task_id"
94
+ holder="${3:-$task_id}"
95
+ if write_lock "$task_id" "$holder"; then
96
+ log_op acquire "task_id=$task_id holder=$holder result=ACQUIRED"
97
+ echo "ACQUIRED by $task_id (holder $holder)"
25
98
  exit 0
26
- else
27
- holder=$(cut -d' ' -f1 "$LOCK_FILE" 2>/dev/null || echo "unknown")
28
- acquired_at=$(cut -d' ' -f2 "$LOCK_FILE" 2>/dev/null || echo "unknown")
29
- lock_pid=$(awk '{print $3}' "$LOCK_FILE" 2>/dev/null || echo "-")
30
- # Self-healing: if the holder's PID is dead, missing, or invalid, the lock
31
- # is stale (the workflow that acquired it terminated without releasing
32
- # via post-deploy). Break the stale lock and retry the acquire. A live
33
- # PID means the lock is genuinely held — report HELD. This makes terminal
34
- # cleanup mechanical: a stopped/failed workflow's lock is reclaimed on
35
- # the next acquire, not via manual release or a background sweeper.
36
- if [ -z "$lock_pid" ] || [ "$lock_pid" = "-" ] || ! kill -0 "$lock_pid" 2>/dev/null; then
37
- echo "STALE: lock held by $holder since $acquired_at (pid $lock_pid dead) — breaking" >&2
38
- rm -f "$LOCK_FILE"
39
- if (set -C; echo "$task_id $(date -u +%Y-%m-%dT%H:%M:%SZ) $owner_pid" > "$LOCK_FILE") 2>/dev/null; then
40
- echo "ACQUIRED by $task_id (reclaimed stale lock from $holder)"
41
- exit 0
42
- fi
43
- # Lost the race: another task acquired it after we broke the stale lock.
44
- holder=$(cut -d' ' -f1 "$LOCK_FILE" 2>/dev/null || echo "unknown")
45
- acquired_at=$(cut -d' ' -f2 "$LOCK_FILE" 2>/dev/null || echo "unknown")
46
- lock_pid=$(awk '{print $3}' "$LOCK_FILE" 2>/dev/null || echo "-")
99
+ fi
100
+ # A lock file exists (or raced into existence) — read it.
101
+ if ! read_lock; then
102
+ # Raced with a release between create and read: retry the create once.
103
+ if write_lock "$task_id" "$holder"; then
104
+ log_op acquire "task_id=$task_id holder=$holder result=ACQUIRED"
105
+ echo "ACQUIRED by $task_id (holder $holder)"
106
+ exit 0
47
107
  fi
48
- echo "HELD by $holder since $acquired_at (pid $lock_pid)"
108
+ read_lock || { echo "ERROR: lock state unreadable"; log_op acquire "task_id=$task_id holder=$holder result=ERROR"; exit 1; }
109
+ fi
110
+ remaining="$(lock_remaining)"
111
+ if [ "$remaining" != "unknown" ] && [ "$remaining" -gt 0 ]; then
112
+ log_op acquire "task_id=$task_id holder=$holder result=HELD holder_task=$lock_task_id holder_id=$lock_holder remaining=${remaining}s"
113
+ echo "HELD by $lock_task_id (holder $lock_holder, ${remaining}s of lease remaining)"
49
114
  exit 1
50
115
  fi
116
+ # Expired or malformed lease — break it (logged) and re-acquire.
117
+ echo "STALE: lease for ${lock_task_id:-unknown} (holder ${lock_holder:-unknown}) expired — breaking" >&2
118
+ log_op acquire "task_id=$task_id holder=$holder result=STALE-BREAK previous_task=${lock_task_id:-unknown} previous_holder=${lock_holder:-unknown}"
119
+ rm -f "$LOCK_FILE"
120
+ if write_lock "$task_id" "$holder"; then
121
+ log_op acquire "task_id=$task_id holder=$holder result=ACQUIRED-RECLAIMED previous_task=${lock_task_id:-unknown}"
122
+ echo "ACQUIRED by $task_id (holder $holder; reclaimed expired lease from ${lock_task_id:-unknown})"
123
+ exit 0
124
+ fi
125
+ # Lost the race: whoever won the create holds it now.
126
+ read_lock 2>/dev/null || true
127
+ log_op acquire "task_id=$task_id holder=$holder result=HELD-RACE holder_task=${lock_task_id:-unknown}"
128
+ echo "HELD by ${lock_task_id:-unknown} (holder ${lock_holder:-unknown}) — lost reclaim race"
129
+ exit 1
130
+ ;;
131
+ refresh)
132
+ [ -z "$task_id" ] && { echo "ERROR: task_id required"; exit 1; }
133
+ if ! read_lock; then
134
+ log_op refresh "task_id=$task_id result=NO-LOCK"
135
+ echo "ERROR: no lock to refresh"
136
+ exit 1
137
+ fi
138
+ if [ "$lock_task_id" != "$task_id" ]; then
139
+ log_op refresh "task_id=$task_id result=NOT-HOLDER holder_task=$lock_task_id"
140
+ echo "ERROR: lock held by $lock_task_id, not $task_id"
141
+ exit 1
142
+ fi
143
+ # Extend the lease: acquired_at=now, holder and lease window unchanged.
144
+ printf 'task_id=%s\nholder=%s\nacquired_at=%s\nlease_seconds=%s\n' \
145
+ "$lock_task_id" "$lock_holder" "$(date +%s)" "${lock_lease_seconds:-$LEASE_SECONDS}" > "$LOCK_FILE"
146
+ log_op refresh "task_id=$task_id holder=$lock_holder result=REFRESHED"
147
+ echo "REFRESHED by $task_id (holder $lock_holder)"
148
+ exit 0
51
149
  ;;
52
150
  release)
53
151
  [ -z "$task_id" ] && { echo "ERROR: task_id required"; exit 1; }
54
- if [ -f "$LOCK_FILE" ]; then
55
- holder=$(cut -d' ' -f1 "$LOCK_FILE" 2>/dev/null || echo "")
56
- if [ "$holder" = "$task_id" ]; then
57
- rm -f "$LOCK_FILE"
58
- echo "RELEASED by $task_id"
59
- exit 0
60
- else
61
- echo "ERROR: lock held by $holder, not $task_id"
62
- exit 1
63
- fi
64
- else
152
+ if ! read_lock; then
153
+ log_op release "task_id=$task_id result=NOT-LOCKED"
65
154
  echo "RELEASED (was not locked)"
66
155
  exit 0
67
156
  fi
157
+ if [ "$lock_task_id" = "$task_id" ]; then
158
+ rm -f "$LOCK_FILE"
159
+ log_op release "task_id=$task_id holder=$lock_holder result=RELEASED"
160
+ echo "RELEASED by $task_id"
161
+ exit 0
162
+ fi
163
+ log_op release "task_id=$task_id result=NOT-HOLDER holder_task=$lock_task_id"
164
+ echo "ERROR: lock held by $lock_task_id, not $task_id"
165
+ exit 1
68
166
  ;;
69
167
  status)
70
- if [ -f "$LOCK_FILE" ]; then
71
- holder=$(cut -d' ' -f1 "$LOCK_FILE" 2>/dev/null || echo "unknown")
72
- acquired_at=$(cut -d' ' -f2 "$LOCK_FILE" 2>/dev/null || echo "unknown")
73
- lock_pid=$(awk '{print $3}' "$LOCK_FILE" 2>/dev/null || echo "-")
74
- echo "LOCKED by $holder since $acquired_at (pid $lock_pid)"
75
- else
168
+ if ! read_lock; then
76
169
  echo "UNLOCKED"
170
+ exit 0
77
171
  fi
172
+ printf 'locked=true\ntask_id=%s\nholder=%s\nacquired_at=%s\nlease_seconds=%s\nage_seconds=%s\nremaining_seconds=%s\n' \
173
+ "$lock_task_id" "$lock_holder" "$lock_acquired_at" "${lock_lease_seconds:-$LEASE_SECONDS}" \
174
+ "$(lock_age)" "$(lock_remaining)"
78
175
  exit 0
79
176
  ;;
80
177
  force-release)
178
+ read_lock 2>/dev/null || true
179
+ log_op force-release "result=FORCE-RELEASED previous_task=${lock_task_id:-unknown}"
81
180
  rm -f "$LOCK_FILE"
82
181
  echo "FORCE RELEASED"
83
182
  exit 0
84
183
  ;;
85
- refresh)
86
- [ -z "$task_id" ] && { echo "ERROR: task_id required"; exit 1; }
87
- new_pid="${3:-$PPID}"
88
- if [ -f "$LOCK_FILE" ]; then
89
- holder=$(cut -d' ' -f1 "$LOCK_FILE" 2>/dev/null || echo "")
90
- if [ "$holder" = "$task_id" ]; then
91
- echo "$task_id $(date -u +%Y-%m-%dT%H:%M:%SZ) $new_pid" > "$LOCK_FILE"
92
- echo "REFRESHED by $task_id"
93
- exit 0
94
- else
95
- echo "ERROR: lock held by $holder, not $task_id"
96
- exit 1
97
- fi
98
- else
99
- echo "ERROR: no lock to refresh"
100
- exit 1
101
- fi
102
- ;;
103
184
  *)
104
- echo "Usage: merge-lock.sh {acquire|release|refresh|status|force-release} [task_id] [pid]"
185
+ echo "Usage: merge-lock.sh {acquire|release|refresh|status|force-release} <task_id> [holder]"
105
186
  exit 1
106
187
  ;;
107
188
  esac
@@ -4,9 +4,9 @@
4
4
  #
5
5
  # Usage:
6
6
  # orphan-sweep.sh report — list orphans (read-only for worktrees);
7
- # stale locks with dead PIDs are released
7
+ # expired-lease locks are released
8
8
  # orphan-sweep.sh clean — remove safe-to-clean orphans (merged branches only),
9
- # release stale merge locks, and remove pins whose
9
+ # release expired-lease merge locks, and remove pins whose
10
10
  # task has no running session
11
11
  #
12
12
  # Active-run knowledge is injected by the caller, never fetched here:
@@ -44,7 +44,7 @@ set -euo pipefail
44
44
  REPO="${CREW_REPO:-$HOME/workspace/ts-spaces/orchestra-dashboard}"
45
45
  WORKTREE_DIR="$REPO/.worktrees"
46
46
  LOCK_FILE="$WORKTREE_DIR/.merge-lock"
47
- STALE_LOCK_MIN=30
47
+ MERGE_LOCK_SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/merge-lock.sh"
48
48
 
49
49
  cmd="${1:-report}"
50
50
  found=0
@@ -100,40 +100,44 @@ if [ "$cmd" != "clean" ] && [ "$have_active_data" -eq 0 ]; then
100
100
  echo "NOTE: no active-run data provided; ACTIVE checks skipped"
101
101
  fi
102
102
 
103
- # --- Stale merge lock ---
103
+ # --- Stale merge lock (lease-based; bug 2fc8f52f) ---
104
+ # Owner identity comes from merge-lock.sh status key=value fields — never
105
+ # from a PID. Fail-closed: a lock whose task has a running session is
106
+ # untouched regardless of lease age; a lock whose task has no running
107
+ # session is broken only when its lease is expired. The break goes through
108
+ # merge-lock.sh release (or force-release for an unparseable file with no
109
+ # identifiable holder) so it lands in the merge-lock audit log.
104
110
  if [ -f "$LOCK_FILE" ]; then
105
- lock_holder=$(cut -d' ' -f1 "$LOCK_FILE" 2>/dev/null || echo "unknown")
106
- if task_is_active "$lock_holder"; then
107
- echo "ACTIVE_LOCK: held by $lock_holder (active run on dashboard)skipping"
108
- found=1
111
+ lock_status="$("$MERGE_LOCK_SCRIPT" status 2>/dev/null || echo UNLOCKED)"
112
+ if [ "$lock_status" = "UNLOCKED" ]; then
113
+ : # lock file vanished between the check and the read nothing held
109
114
  else
110
- lock_time=$(cut -d' ' -f2 "$LOCK_FILE" 2>/dev/null || echo "")
111
- lock_pid=$(awk '{print $3}' "$LOCK_FILE" 2>/dev/null || echo "")
112
- if [ -n "$lock_time" ]; then
113
- lock_epoch=$(date -d "$lock_time" +%s 2>/dev/null || echo 0)
114
- now_epoch=$(date -u +%s)
115
- age_min=$(( (now_epoch - lock_epoch) / 60 ))
116
- if [ "$age_min" -gt "$STALE_LOCK_MIN" ]; then
117
- # Check if owner process is still alive
118
- pid_alive=0
119
- if [ -n "$lock_pid" ] && [ "$lock_pid" != "-" ] && kill -0 "$lock_pid" 2>/dev/null; then
120
- pid_alive=1
121
- fi
122
-
123
- if [ "$pid_alive" -eq 1 ]; then
124
- echo "ACTIVE_LOCK: held ${age_min}m by $lock_holder (pid $lock_pid alive) — skipping"
125
- else
126
- echo "STALE_LOCK: held ${age_min}m by $lock_holder (pid ${lock_pid:-none} dead) — threshold ${STALE_LOCK_MIN}m"
127
- found=1
128
- rm -f "$LOCK_FILE"
129
- if [ ! -f "$LOCK_FILE" ]; then
130
- echo " → released (dead PID)"
131
- else
132
- echo " → FAILED: lock file still present after rm"
133
- any_failed=1
134
- fi
135
- fi
115
+ lock_holder=$(printf '%s\n' "$lock_status" | sed -n 's/^task_id=//p' | head -1)
116
+ lock_remaining=$(printf '%s\n' "$lock_status" | sed -n 's/^remaining_seconds=//p' | head -1)
117
+ lock_age=$(printf '%s\n' "$lock_status" | sed -n 's/^age_seconds=//p' | head -1)
118
+ if task_is_active "$lock_holder"; then
119
+ echo "ACTIVE_LOCK: held by ${lock_holder:-unknown} (active run on dashboard) — skipping"
120
+ found=1
121
+ elif [ "${lock_remaining:-unknown}" != "unknown" ] && [ "$lock_remaining" -gt 0 ]; then
122
+ echo "HELD_LOCK: held by ${lock_holder:-unknown} (lease ${lock_remaining}s remaining) — skipping"
123
+ found=1
124
+ else
125
+ echo "STALE_LOCK: held by ${lock_holder:-unknown} (age ${lock_age:-unknown}s, lease expired, no running session)"
126
+ found=1
127
+ if [ -n "$lock_holder" ]; then
128
+ rel_out=$("$MERGE_LOCK_SCRIPT" release "$lock_holder" 2>&1) || true
129
+ else
130
+ rel_out=$("$MERGE_LOCK_SCRIPT" force-release 2>&1) || true
136
131
  fi
132
+ case "$rel_out" in
133
+ *RELEASED*)
134
+ echo " → released (lease expired)"
135
+ ;;
136
+ *)
137
+ echo " → FAILED: could not release lock: $rel_out"
138
+ any_failed=1
139
+ ;;
140
+ esac
137
141
  fi
138
142
  fi
139
143
  fi
@@ -174,6 +174,14 @@ if [ "$ALREADY_PUBLISHED" = "0" ]; then
174
174
  # STEP-12-END
175
175
  fi
176
176
 
177
+ # 12b. Refresh the merge lock again — release-deploy, pack, publish, and the
178
+ # verify retry loop can together consume more than half the lease on a slow
179
+ # registry. The push and the post-deploy below must happen under our lock;
180
+ # if the refresh fails (expired lease, someone else holds it), stop — never
181
+ # push a version-bump commit or release a lock we no longer own.
182
+ LOCK_OUT2="$(CREW_REPO="$REPO_PATH" bash "${LIFECYCLE:?LIFECYCLE is required}" refresh-lock "$TASK_ID" 2>&1)" \
183
+ || fail "lock-refresh" "$LOCK_OUT2"
184
+
177
185
  # 13. Push the version-bump commit (idempotent: no-op if already pushed).
178
186
  PUSH_OUT="$(git -C "$REPO_PATH" push origin main 2>&1)" \
179
187
  || fail "push" "$PUSH_OUT"
@@ -2,9 +2,12 @@
2
2
  # test-merge-lock.sh — regression tests for merge-lock.sh
3
3
  #
4
4
  # Self-contained: uses a temp CREW_REPO and drives lib/merge-lock.sh through
5
- # acquire/release/status. Tests the self-healing acquire: a lock held by a
6
- # dead PID is reclaimed automatically; a lock held by a live PID reports HELD.
7
- # Exits 0 only if every case passes.
5
+ # acquire/refresh/release/status. Tests the time-based lease model (bug
6
+ # 2fc8f52f): an unexpired lease blocks contenders even when no holder
7
+ # process is alive (PID liveness is never consulted); an expired lease is
8
+ # reclaimed with logging; refresh extends the lease; release is
9
+ # holder-only; status emits parseable holder/age/remaining fields; every
10
+ # op appends to the merge-lock audit log. Exits 0 only if every case passes.
8
11
 
9
12
  set -uo pipefail
10
13
 
@@ -20,49 +23,89 @@ TMPBASE="$(mktemp -d)"
20
23
  trap 'rm -rf "$TMPBASE"' EXIT
21
24
 
22
25
  export CREW_REPO="$TMPBASE/repo"
26
+ export CREW_HOME="$TMPBASE/home"
27
+ export MERGE_LOCK_LEASE_SECONDS=600
23
28
  mkdir -p "$CREW_REPO/.worktrees"
24
29
 
25
- # --- Case 1: acquire on unlocked succeeds ---
26
- # Use $$ (this shell's PID) which is definitely alive.
27
- out=$("$LOCK" acquire task-1 $$ 2>&1); code=$?
28
- [ "$code" -eq 0 ] && [[ "$out" == *"ACQUIRED by task-1"* ]] && ok "1: acquire on unlocked succeeds" || no "1: exit $code, out: $out"
30
+ # --- Case 1: acquire on unlocked succeeds (opaque holder, not a PID) ---
31
+ out=$("$LOCK" acquire task-1 "task-1/run-aaa" 2>&1); code=$?
32
+ [ "$code" -eq 0 ] && [[ "$out" == *"ACQUIRED by task-1"* ]] && [[ "$out" == *"task-1/run-aaa"* ]] \
33
+ && ok "1: acquire on unlocked succeeds with opaque holder" || no "1: exit $code, out: $out"
29
34
 
30
- # --- Case 2: second acquire with live PID reports HELD ---
31
- out=$("$LOCK" acquire task-2 99999 2>&1); code=$?
32
- [ "$code" -eq 1 ] && [[ "$out" == *"HELD by task-1"* ]] && ok "2: acquire with live holder PID reports HELD" || no "2: exit $code, out: $out"
35
+ # --- Case 2: contender blocked while lease unexpired (no PID involved) ---
36
+ out=$("$LOCK" acquire task-2 "task-2/run-bbb" 2>&1); code=$?
37
+ [ "$code" -eq 1 ] && [[ "$out" == *"HELD by task-1"* ]] \
38
+ && ok "2: unexpired lease blocks contender" || no "2: exit $code, out: $out"
33
39
 
34
- # --- Case 3: release by owner succeeds ---
35
- out=$("$LOCK" release task-1 2>&1); code=$?
36
- [ "$code" -eq 0 ] && [[ "$out" == *"RELEASED by task-1"* ]] && ok "3: release by owner succeeds" || no "3: exit $code, out: $out"
37
-
38
- # --- Case 4: status reports UNLOCKED after release ---
40
+ # --- Case 3: status emits parseable holder/age/remaining fields ---
39
41
  out=$("$LOCK" status 2>&1); code=$?
40
- [ "$code" -eq 0 ] && [[ "$out" == "UNLOCKED" ]] && ok "4: status UNLOCKED after release" || no "4: exit $code, out: $out"
42
+ [ "$code" -eq 0 ] \
43
+ && [[ "$out" == *"locked=true"* ]] \
44
+ && [[ "$out" == *"task_id=task-1"* ]] \
45
+ && [[ "$out" == *"holder=task-1/run-aaa"* ]] \
46
+ && [[ "$out" == *"age_seconds="* ]] \
47
+ && [[ "$out" == *"remaining_seconds="* ]] \
48
+ && ok "3: status has holder/age/remaining fields" || no "3: exit $code, out: $out"
49
+
50
+ # --- Case 4: refresh extends the lease (holder-only) ---
51
+ # Sleep long enough that integer-second lease math must show decay; a
52
+ # refresh then resets the clock and the remaining lease grows again.
53
+ sleep 3
54
+ just_before=$("$LOCK" status | sed -n 's/^remaining_seconds=//p')
55
+ "$LOCK" refresh task-1 >/dev/null 2>&1; code=$?
56
+ just_after=$("$LOCK" status | sed -n 's/^remaining_seconds=//p')
57
+ [ "$code" -eq 0 ] && [ "$just_after" -gt "$just_before" ] \
58
+ && ok "4: refresh extends the lease" || no "4: code $code, before $just_before, after $just_after"
59
+
60
+ # --- Case 5: refresh by non-holder fails ---
61
+ out=$("$LOCK" refresh task-2 2>&1); code=$?
62
+ [ "$code" -eq 1 ] && [[ "$out" == *"held by task-1"* ]] \
63
+ && ok "5: refresh by non-holder fails" || no "5: exit $code, out: $out"
41
64
 
42
- # --- Case 5: stale lock (dead PID) is reclaimed on acquire ---
43
- # Write a lock file with a PID that is certainly dead (99999999).
44
- echo "stale-task 2026-09-10T06:21:15Z 99999999" > "$CREW_REPO/.worktrees/.merge-lock"
45
- out=$("$LOCK" acquire task-3 22222 2>&1); code=$?
46
- [ "$code" -eq 0 ] && [[ "$out" == *"reclaimed stale lock from stale-task"* ]] && ok "5: stale lock (dead PID) reclaimed on acquire" || no "5: exit $code, out: $out"
65
+ # --- Case 6: release by non-holder fails ---
66
+ out=$("$LOCK" release task-2 2>&1); code=$?
67
+ [ "$code" -eq 1 ] && [[ "$out" == *"held by task-1"* ]] \
68
+ && ok "6: release by non-holder fails" || no "6: exit $code, out: $out"
47
69
 
48
- # --- Case 6: after reclaim, the new holder owns the lock ---
70
+ # --- Case 7: release by holder succeeds and removes the lock ---
71
+ out=$("$LOCK" release task-1 2>&1); code=$?
72
+ [ "$code" -eq 0 ] && [[ "$out" == *"RELEASED by task-1"* ]] && [ ! -f "$CREW_REPO/.worktrees/.merge-lock" ] \
73
+ && ok "7: release by holder removes lock file" || no "7: exit $code, out: $out"
74
+
75
+ # --- Case 8: status reports UNLOCKED after release ---
49
76
  out=$("$LOCK" status 2>&1); code=$?
50
- [ "$code" -eq 0 ] && [[ "$out" == *"LOCKED by task-3"* ]] && ok "6: reclaimed lock owned by new task" || no "6: exit $code, out: $out"
77
+ [ "$code" -eq 0 ] && [ "$out" = "UNLOCKED" ] \
78
+ && ok "8: status UNLOCKED after release" || no "8: exit $code, out: $out"
79
+
80
+ # --- Case 9: expired lease is reclaimed (logged) by a contender ---
81
+ # Write a lock file whose lease expired long ago (epoch 2020, lease 600s).
82
+ printf 'task_id=old-task\nholder=old-task/run-old\nacquired_at=1577836800\nlease_seconds=600\n' \
83
+ > "$CREW_REPO/.worktrees/.merge-lock"
84
+ out=$("$LOCK" acquire task-3 "task-3/run-ccc" 2>&1); code=$?
85
+ [ "$code" -eq 0 ] && [[ "$out" == *"ACQUIRED by task-3"* ]] && [[ "$out" == *"reclaimed expired lease from old-task"* ]] \
86
+ && ok "9: expired lease reclaimed by contender" || no "9: exit $code, out: $out"
87
+ [ -f "$TMPBASE/home/.merge-lock.log" ] && grep -q "op=acquire.*result=STALE-BREAK.*previous_task=old-task" "$TMPBASE/home/.merge-lock.log" \
88
+ && ok "9b: reclaim logged to the audit log" || no "9b: no STALE-BREAK line in audit log"
51
89
 
52
- # --- Case 7: release by non-owner fails ---
53
- out=$("$LOCK" release wrong-task 2>&1); code=$?
54
- [ "$code" -eq 1 ] && [[ "$out" == *"ERROR: lock held by task-3"* ]] && ok "7: release by non-owner fails" || no "7: exit $code, out: $out"
90
+ # --- Case 10: malformed lock file is treated as expired and reclaimed ---
91
+ printf 'garbage without any key=value\n' > "$CREW_REPO/.worktrees/.merge-lock"
92
+ out=$("$LOCK" acquire task-4 "task-4/run-ddd" 2>&1); code=$?
93
+ [ "$code" -eq 0 ] && [[ "$out" == *"ACQUIRED by task-4"* ]] \
94
+ && ok "10: malformed lock reclaimed" || no "10: exit $code, out: $out"
55
95
 
56
- # --- Case 8: release by owner cleans up ---
57
- out=$("$LOCK" release task-3 2>&1); code=$?
58
- [ "$code" -eq 0 ] && [ ! -f "$CREW_REPO/.worktrees/.merge-lock" ] && ok "8: release removes lock file" || no "8: exit $code, out: $out"
96
+ # --- Case 11: every op appends to the audit log ---
97
+ lines=$(wc -l < "$TMPBASE/home/.merge-lock.log")
98
+ [ "$lines" -ge 8 ] && ok "11: audit log has $lines lines" || no "11: audit log has only $lines lines"
59
99
 
60
- # --- Case 9: lock with missing PID field is treated as stale ---
61
- # A lock file with "-" as PID (or empty) should be reclaimable.
62
- echo "old-task 2026-09-10T06:21:15Z -" > "$CREW_REPO/.worktrees/.merge-lock"
63
- out=$("$LOCK" acquire task-4 33333 2>&1); code=$?
64
- [ "$code" -eq 0 ] && [[ "$out" == *"reclaimed stale lock from old-task"* ]] && ok "9: lock with '-' PID reclaimed as stale" || no "9: exit $code, out: $out"
100
+ # --- Case 12: concurrent acquirers serialize exactly one wins ---
65
101
  "$LOCK" release task-4 >/dev/null 2>&1
102
+ for n in $(seq 1 20); do
103
+ ( "$LOCK" acquire "racer-$n" "racer-$n/run" >/dev/null 2>&1 ) &
104
+ done
105
+ wait
106
+ holder_count=$("$LOCK" status | grep -c '^task_id=racer-' || true)
107
+ [ "$holder_count" -eq 1 ] && ok "12: exactly one concurrent acquirer holds the lock" || no "12: holder_count=$holder_count"
108
+ grep -q "result=ACQUIRED" "$TMPBASE/home/.merge-lock.log" && ok "12b: at least one ACQUIRED logged" || no "12b: no ACQUIRED in audit log"
66
109
 
67
110
  echo ""
68
111
  echo "merge-lock: $pass passed, $fail failed"
@@ -22,6 +22,9 @@ no() { echo "FAIL: $1"; fail=$((fail + 1)); }
22
22
  TMPBASE="$(mktemp -d)"
23
23
  trap 'rm -rf "$TMPBASE"' EXIT
24
24
 
25
+ # Keep merge-lock.sh's audit log out of the real $CREW_HOME during tests.
26
+ export CREW_HOME="$TMPBASE/home"
27
+
25
28
  # new_fixture <name> — fresh git repo, one commit on main; exports CREW_REPO.
26
29
  new_fixture() {
27
30
  local dir="$TMPBASE/$1"
@@ -90,21 +93,32 @@ echo "$out" | grep -q "→ FAILED" && ok "4: FAILED line printed (no false 'remo
90
93
  [ -d .worktrees/T4 ] && ok "4: failed worktree left in place" || no "4: worktree gone despite failed removal"
91
94
  git worktree unlock .worktrees/T4 2>/dev/null || true
92
95
 
93
- # --- Case 5: lock path ---
96
+ # --- Case 5: lock path (lease-based; bug 2fc8f52f) ---
94
97
  new_fixture case5
95
98
  mkdir -p .worktrees
96
- printf 'T5 2020-01-01T00:00:00Z 999999\n' > .worktrees/.merge-lock
99
+ # 5a: expired lease, no running session → released (logged to the audit log).
100
+ printf 'task_id=T5\nholder=T5/run-old\nacquired_at=1577836800\nlease_seconds=600\n' > .worktrees/.merge-lock
97
101
  out=$(CREW_ACTIVE_TASKS="" "$SWEEP" clean 2>&1); code=$?
98
- [ ! -f .worktrees/.merge-lock ] && ok "5a: stale lock released" || no "5a: stale lock file still present"
99
- echo "$out" | grep -q "released (dead PID)" && ok "5a: release reported" || no "5a: no release line in output"
102
+ [ ! -f .worktrees/.merge-lock ] && ok "5a: expired-lease lock released" || no "5a: lock file still present"
103
+ echo "$out" | grep -q "STALE_LOCK" && ok "5a: STALE_LOCK line printed" || no "5a: no STALE_LOCK line in output"
104
+ echo "$out" | grep -q "released (lease expired)" && ok "5a: release reported" || no "5a: no release line in output"
100
105
  [ "$code" -eq 0 ] && ok "5a: exit 0" || no "5a: exit $code, want 0"
101
106
 
102
- printf '%s 2020-01-01T00:00:00Z 999999\n' "$UUID_LOCK" > .worktrees/.merge-lock
107
+ # 5b: valid lease held by an active run → untouched regardless of lease age.
108
+ now=$(date +%s)
109
+ printf 'task_id=%s\nholder=%s/run-live\nacquired_at=%s\nlease_seconds=600\n' "$UUID_LOCK" "$UUID_LOCK" "$now" > .worktrees/.merge-lock
103
110
  out=$(CREW_ACTIVE_TASKS="$UUID_LOCK" "$SWEEP" clean 2>&1); code=$?
104
111
  [ -f .worktrees/.merge-lock ] && ok "5b: active holder's lock not released" || no "5b: active holder's lock was released"
105
112
  echo "$out" | grep -q "ACTIVE_LOCK" && ok "5b: ACTIVE_LOCK line printed" || no "5b: no ACTIVE_LOCK line in output"
106
113
  [ "$code" -eq 0 ] && ok "5b: exit 0" || no "5b: exit $code, want 0"
107
114
 
115
+ # 5c: valid lease, no running session → held, NOT released (lease not expired).
116
+ printf 'task_id=T6\nholder=T6/run-live\nacquired_at=%s\nlease_seconds=600\n' "$now" > .worktrees/.merge-lock
117
+ out=$(CREW_ACTIVE_TASKS="" "$SWEEP" clean 2>&1); code=$?
118
+ [ -f .worktrees/.merge-lock ] && ok "5c: unexpired-lease lock not released" || no "5c: unexpired-lease lock was released"
119
+ echo "$out" | grep -q "HELD_LOCK" && ok "5c: HELD_LOCK line printed" || no "5c: no HELD_LOCK line in output"
120
+ [ "$code" -eq 0 ] && ok "5c: exit 0" || no "5c: exit $code, want 0"
121
+
108
122
  # --- Case 6: report mode without active data ---
109
123
  new_fixture case6
110
124
  merged_worktree T7
@@ -202,13 +202,39 @@ cmd_integrate() {
202
202
  return 0
203
203
  fi
204
204
 
205
- # Acquire merge lock
205
+ # Acquire the merge lock (serializes version assignment across concurrent
206
+ # runs). The holder is the opaque task+run identity from WORKFLOW_RUN_ID —
207
+ # never a PID (bug 2fc8f52f: short-lived agent PIDs made every concurrent
208
+ # acquire take the stale path). Another task mid Integrate/Publish is
209
+ # normal under simultaneity>1: bounded backoff (30s sleeps, 10min cap,
210
+ # overridable via MERGE_LOCK_RETRY_SECS / MERGE_LOCK_RETRY_CAP_SECS), then
211
+ # give up with LOCK_HELD so the workflow records the failure with a reason
212
+ # (dispatcher retries under its cap, parks after the cap).
213
+ local holder="${WORKFLOW_RUN_ID:-$task_id}"
214
+ local retry_secs="${MERGE_LOCK_RETRY_SECS:-30}"
215
+ local retry_cap="${MERGE_LOCK_RETRY_CAP_SECS:-600}"
216
+ local waited=0
206
217
  local lock_result
207
- if ! lock_result=$("$MERGE_LOCK" acquire "$task_id" "$$"); then
208
- echo "LOCK_HELD: $lock_result"
209
- exit 2
210
- fi
211
- echo "$lock_result"
218
+ while true; do
219
+ if lock_result=$("$MERGE_LOCK" acquire "$task_id" "$holder"); then
220
+ echo "$lock_result"
221
+ break
222
+ fi
223
+ case "$lock_result" in
224
+ HELD*)
225
+ if [ "$waited" -ge "$retry_cap" ]; then
226
+ echo "LOCK_HELD: $lock_result (waited ${waited}s of ${retry_cap}s backoff)"
227
+ exit 2
228
+ fi
229
+ sleep "$retry_secs"
230
+ waited=$((waited + retry_secs))
231
+ ;;
232
+ *)
233
+ echo "LOCK_ERROR: $lock_result"
234
+ exit 2
235
+ ;;
236
+ esac
237
+ done
212
238
 
213
239
  # Merge
214
240
  if ! git merge --no-ff "$branch" -m "$commit_msg" 2>&1; then
@@ -338,7 +364,7 @@ cmd_sweep() {
338
364
  cmd_refresh_lock() {
339
365
  local task_id="$1"
340
366
  validate_task_id "$task_id"
341
- "$MERGE_LOCK" refresh "$task_id" "$$"
367
+ "$MERGE_LOCK" refresh "$task_id"
342
368
  }
343
369
 
344
370
  # --- dispatch ---
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "muse-crew",
3
- "version": "0.4.5",
3
+ "version": "0.4.6",
4
4
  "description": "Opinionated orchestration for Muse \u2014 workflows, identities, and tooling for autonomous software development.",
5
5
  "license": "UNLICENSED",
6
6
  "private": false,
@@ -510,6 +510,12 @@ if (missingInitialPins.length > 0) {
510
510
  }
511
511
  log("Lifecycle scripts pinned to " + RUN_LIB);
512
512
 
513
+ // Merge-lock holder identity (bug 2fc8f52f): the opaque task+run identity
514
+ // minted at this run's first claim (never a PID — short-lived agent PIDs
515
+ // made every concurrent acquire take the stale path). Set exactly once on
516
+ // the first claim; stable for the rest of the run's lifetime.
517
+ let lockHolder = taskId;
518
+
513
519
  while (i < STEPS.length) {
514
520
  const step = STEPS[i];
515
521
  const isFirstClaim = (i === startStepIndex && totalReworkCount === 0);
@@ -693,6 +699,13 @@ while (i < STEPS.length) {
693
699
  activeSessionId = claimResult.session_id;
694
700
  }
695
701
 
702
+ // Mint the merge-lock holder identity once: task+session of this run's
703
+ // first claim. The task ID alone cannot distinguish two runs of the same
704
+ // task (re-dispatch after park), and a PID is not a valid holder at all.
705
+ if (lockHolder === taskId && activeSessionId) {
706
+ lockHolder = taskId + "/" + activeSessionId;
707
+ }
708
+
696
709
  // ── Capture: baseline evidence for experiential tasks ─────────────
697
710
  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
698
711
  // Sage flagged experiential. The capture itself is parent-driven (the
@@ -870,11 +883,11 @@ while (i < STEPS.length) {
870
883
 
871
884
  } else if (step.name === "Integrate") {
872
885
  instructions = "Merge the approved task branch into main.\n\n" +
873
- "Run: "+ LIFECYCLE_ENV + " integrate " + taskId + " \"merge: fix: " + safeTitle + "\"\n\n" +
886
+ "Run: "+ LIFECYCLE_ENV + "WORKFLOW_RUN_ID=" + lockHolder + " integrate " + taskId + " \"merge: fix: " + safeTitle + "\"\n\n" +
874
887
  "Read the output:\n" +
875
888
  "- If it contains MERGED, integration succeeded. Report the merged commit hash.\n" +
876
889
  "- If it contains MERGED_EMPTY, the branch had no commits ahead of main (a runtime-state deliverable, declared by Build as repo_diff: none). Integration succeeded vacuously: the merge lock was NOT taken and there is no new commit. Report 'merged empty: no repo changes — deliverable was runtime state', then end your report with exactly this line: VERDICT: PASS. SKIP STEP 2 (push): there is no new commit to push.\n" +
877
- "- If it contains LOCK_HELD, another task holds the merge lock (mid Integrate/Publish). Report 'merge lock held', then end your report with exactly this line: VERDICT: FAIL.\n" +
890
+ "- If it contains LOCK_HELD, another task holds the merge lock (mid Integrate/Publish) and the 10-minute bounded backoff is exhausted. Report 'merge lock held after bounded backoff', then end your report with exactly this line: VERDICT: FAIL.\n" +
878
891
  "- If it contains CONFLICT, the plain merge failed — the merge was aborted, main is clean, and your task still holds the merge lock. Do NOT fail yet. Resolve it:\n" +
879
892
  "RESOLUTION:\n" +
880
893
  "R1. Refresh the merge lock FIRST (a long resolution must not silently lose the lock to the orphan sweep): "+ LIFECYCLE_ENV + " refresh-lock " + taskId + ". Create a scratch worktree WITH A NEW BRANCH (main is already checked out in the repo checkout, so git forbids checking it out a second time): cd " + REPO_PATH + " && git worktree add -b resolve/" + taskId + " /tmp/crew-resolve-" + taskId + " main. Reproduce the conflict in the scratch worktree: cd /tmp/crew-resolve-" + taskId + " && git merge " + TASK_BRANCH + ". This reproduces the exact conflict (main has not moved — the lock was held throughout). The task branch " + TASK_BRANCH + " is never modified.\n" +
@@ -962,14 +975,46 @@ while (i < STEPS.length) {
962
975
  }
963
976
  var publishFailure = null;
964
977
  if (rebuildTrigger.edit_started) {
965
- // STEP 1b (mechanical): bounded poll for build completion.
966
- var buildPoll = await agent(
967
- "Poll artifact_status for slug \"" + PUBLISH_SLUG + "\" until no build is running. Check every 30 seconds, up to 20 checks (10 minutes max).\n" +
968
- "Return JSON { \"build_done\": <true if no build is running within budget, false on timeout>, \"status\": \"<final status or timeout note>\" } and nothing else.",
969
- { key: attemptKey("publish-artifact-poll-" + taskId, totalReworkCount), label: "Waiting for artifact build to complete",
970
- schema: { type: "object", properties: { build_done: { type: "boolean" }, status: { type: "string" } }, required: ["build_done"] },
971
- timeoutMs: 660000 }
972
- );
978
+ // STEP 1b (mechanical): bounded poll for build completion, chunked so
979
+ // the merge-lock lease is refreshed before it can expire. The 600s
980
+ // lease is shorter than the worst-case 10-minute build poll, so the
981
+ // poll runs in three chunks (7 checks x 30s ~= 3.5 min each) with a
982
+ // holder-only lease refresh between chunks. If a refresh fails, the
983
+ // lock was lost: stop the run and park the task never continue to
984
+ // a provenance stamp or version assignment without holding the lock.
985
+ var buildPoll = null;
986
+ for (var chunk = 1; chunk <= 3; chunk++) {
987
+ if (chunk > 1) {
988
+ var refreshPoll = await agent(
989
+ "Refresh the merge lock for task " + taskId + ".\n" +
990
+ "Run: " + LIFECYCLE_ENV + " refresh-lock " + taskId + "\n" +
991
+ "If the output contains REFRESHED, return JSON { \"held\": true } and nothing else. Otherwise return JSON { \"held\": false, \"output\": \"<verbatim output>\" } and nothing else.",
992
+ { key: attemptKey("publish-lock-refresh-poll-" + taskId + "-c" + chunk, totalReworkCount), label: "Refreshing merge lock during build poll",
993
+ schema: { type: "object", properties: { held: { type: "boolean" }, output: { type: "string" } }, required: ["held"] },
994
+ timeoutMs: 60000 }
995
+ );
996
+ if (!refreshPoll || refreshPoll.held !== true) {
997
+ return await parkTask("Merge-lock lease lost during the artifact build poll (refresh before chunk " + chunk + " of 3 failed: " + ((refreshPoll && refreshPoll.output) || "no output") + "). Fail-closed: stopping before any provenance stamp or version assignment.");
998
+ }
999
+ }
1000
+ // Chunk 1 keeps the original stable key; later chunks use -c<N>
1001
+ // suffixed keys. All are attemptKey-scoped so rework passes stay
1002
+ // disjoint (replay-key contract).
1003
+ var pollKey = (chunk === 1)
1004
+ ? attemptKey("publish-artifact-poll-" + taskId, totalReworkCount)
1005
+ : attemptKey("publish-artifact-poll-" + taskId + "-c" + chunk, totalReworkCount);
1006
+ buildPoll = await agent(
1007
+ "Poll artifact_status for slug \"" + PUBLISH_SLUG + "\" until no build is running. Check every 30 seconds, up to 7 checks (3.5 minutes max).\n" +
1008
+ "Return JSON { \"build_done\": <true if no build is running within budget, false on timeout>, \"status\": \"<final status or timeout note>\" } and nothing else.",
1009
+ { key: pollKey, label: "Waiting for artifact build to complete (chunk " + chunk + " of 3)",
1010
+ schema: { type: "object", properties: { build_done: { type: "boolean" }, status: { type: "string" } }, required: ["build_done"] },
1011
+ timeoutMs: 270000 }
1012
+ );
1013
+ if (buildPoll && buildPoll.build_done) { break; }
1014
+ }
1015
+ if (!buildPoll || !buildPoll.build_done) {
1016
+ buildPoll = { build_done: false, status: (buildPoll && buildPoll.status) || "build still running after the 10.5-minute bounded poll" };
1017
+ }
973
1018
  if (buildPoll.build_done) {
974
1019
  // STEP 1c (mechanical): stamp provenance from workflow-computed values.
975
1020
  var provStamp = await agent(
@@ -468,6 +468,12 @@ if (missingInitialPins.length > 0) {
468
468
  }
469
469
  log("Lifecycle scripts pinned to " + RUN_LIB);
470
470
 
471
+ // Merge-lock holder identity (bug 2fc8f52f): the opaque task+run identity
472
+ // minted at this run's first claim (never a PID — short-lived agent PIDs
473
+ // made every concurrent acquire take the stale path). Set exactly once on
474
+ // the first claim; stable for the rest of the run's lifetime.
475
+ let lockHolder = taskId;
476
+
471
477
  while (i < STEPS.length) {
472
478
  const step = STEPS[i];
473
479
  const isFirstClaim = (i === startStepIndex && reworkCount === 0);
@@ -651,6 +657,13 @@ while (i < STEPS.length) {
651
657
  activeSessionId = claimResult.session_id;
652
658
  }
653
659
 
660
+ // Mint the merge-lock holder identity once: task+session of this run's
661
+ // first claim. The task ID alone cannot distinguish two runs of the same
662
+ // task (re-dispatch after park), and a PID is not a valid holder at all.
663
+ if (lockHolder === taskId && activeSessionId) {
664
+ lockHolder = taskId + "/" + activeSessionId;
665
+ }
666
+
654
667
  // ── Capture: baseline evidence for experiential tasks ─────────────
655
668
  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
656
669
  // Sage flagged experiential. The capture itself is parent-driven (the
@@ -807,11 +820,11 @@ while (i < STEPS.length) {
807
820
 
808
821
  } else if (step.name === "Integrate") {
809
822
  instructions = "Merge the approved task branch into main.\n\n" +
810
- "Run: "+ LIFECYCLE_ENV + " integrate " + taskId + " \"merge: chore: " + safeTitle + "\"\n\n" +
823
+ "Run: "+ LIFECYCLE_ENV + "WORKFLOW_RUN_ID=" + lockHolder + " integrate " + taskId + " \"merge: chore: " + safeTitle + "\"\n\n" +
811
824
  "Read the output:\n" +
812
825
  "- If it contains MERGED, integration succeeded. Report the merged commit hash.\n" +
813
826
  "- If it contains MERGED_EMPTY, the branch had no commits ahead of main (a runtime-state deliverable, declared by Build as repo_diff: none). Integration succeeded vacuously: the merge lock was NOT taken and there is no new commit. Report 'merged empty: no repo changes — deliverable was runtime state', then end your report with exactly this line: VERDICT: PASS. SKIP STEP 2 (push): there is no new commit to push.\n" +
814
- "- If it contains LOCK_HELD, another task holds the merge lock (mid Integrate/Publish). Report 'merge lock held', then end your report with exactly this line: VERDICT: FAIL.\n" +
827
+ "- If it contains LOCK_HELD, another task holds the merge lock (mid Integrate/Publish) and the 10-minute bounded backoff is exhausted. Report 'merge lock held after bounded backoff', then end your report with exactly this line: VERDICT: FAIL.\n" +
815
828
  "- If it contains CONFLICT, the plain merge failed — the merge was aborted, main is clean, and your task still holds the merge lock. Do NOT fail yet. Resolve it:\n" +
816
829
  "RESOLUTION:\n" +
817
830
  "R1. Refresh the merge lock FIRST (a long resolution must not silently lose the lock to the orphan sweep): "+ LIFECYCLE_ENV + " refresh-lock " + taskId + ". Create a scratch worktree WITH A NEW BRANCH (main is already checked out in the repo checkout, so git forbids checking it out a second time): cd " + REPO_PATH + " && git worktree add -b resolve/" + taskId + " /tmp/crew-resolve-" + taskId + " main. Reproduce the conflict in the scratch worktree: cd /tmp/crew-resolve-" + taskId + " && git merge " + TASK_BRANCH + ". This reproduces the exact conflict (main has not moved — the lock was held throughout). The task branch " + TASK_BRANCH + " is never modified.\n" +
@@ -899,14 +912,46 @@ while (i < STEPS.length) {
899
912
  }
900
913
  var publishFailure = null;
901
914
  if (rebuildTrigger.edit_started) {
902
- // STEP 1b (mechanical): bounded poll for build completion.
903
- var buildPoll = await agent(
904
- "Poll artifact_status for slug \"" + PUBLISH_SLUG + "\" until no build is running. Check every 30 seconds, up to 20 checks (10 minutes max).\n" +
905
- "Return JSON { \"build_done\": <true if no build is running within budget, false on timeout>, \"status\": \"<final status or timeout note>\" } and nothing else.",
906
- { key: attemptKey("publish-artifact-poll-" + taskId, reworkCount), label: "Waiting for artifact build to complete",
907
- schema: { type: "object", properties: { build_done: { type: "boolean" }, status: { type: "string" } }, required: ["build_done"] },
908
- timeoutMs: 660000 }
909
- );
915
+ // STEP 1b (mechanical): bounded poll for build completion, chunked so
916
+ // the merge-lock lease is refreshed before it can expire. The 600s
917
+ // lease is shorter than the worst-case 10-minute build poll, so the
918
+ // poll runs in three chunks (7 checks x 30s ~= 3.5 min each) with a
919
+ // holder-only lease refresh between chunks. If a refresh fails, the
920
+ // lock was lost: stop the run and park the task never continue to
921
+ // a provenance stamp or version assignment without holding the lock.
922
+ var buildPoll = null;
923
+ for (var chunk = 1; chunk <= 3; chunk++) {
924
+ if (chunk > 1) {
925
+ var refreshPoll = await agent(
926
+ "Refresh the merge lock for task " + taskId + ".\n" +
927
+ "Run: " + LIFECYCLE_ENV + " refresh-lock " + taskId + "\n" +
928
+ "If the output contains REFRESHED, return JSON { \"held\": true } and nothing else. Otherwise return JSON { \"held\": false, \"output\": \"<verbatim output>\" } and nothing else.",
929
+ { key: attemptKey("publish-lock-refresh-poll-" + taskId + "-c" + chunk, reworkCount), label: "Refreshing merge lock during build poll",
930
+ schema: { type: "object", properties: { held: { type: "boolean" }, output: { type: "string" } }, required: ["held"] },
931
+ timeoutMs: 60000 }
932
+ );
933
+ if (!refreshPoll || refreshPoll.held !== true) {
934
+ return await parkTask("Merge-lock lease lost during the artifact build poll (refresh before chunk " + chunk + " of 3 failed: " + ((refreshPoll && refreshPoll.output) || "no output") + "). Fail-closed: stopping before any provenance stamp or version assignment.");
935
+ }
936
+ }
937
+ // Chunk 1 keeps the original stable key; later chunks use -c<N>
938
+ // suffixed keys. All are attemptKey-scoped so rework passes stay
939
+ // disjoint (replay-key contract).
940
+ var pollKey = (chunk === 1)
941
+ ? attemptKey("publish-artifact-poll-" + taskId, reworkCount)
942
+ : attemptKey("publish-artifact-poll-" + taskId + "-c" + chunk, reworkCount);
943
+ buildPoll = await agent(
944
+ "Poll artifact_status for slug \"" + PUBLISH_SLUG + "\" until no build is running. Check every 30 seconds, up to 7 checks (3.5 minutes max).\n" +
945
+ "Return JSON { \"build_done\": <true if no build is running within budget, false on timeout>, \"status\": \"<final status or timeout note>\" } and nothing else.",
946
+ { key: pollKey, label: "Waiting for artifact build to complete (chunk " + chunk + " of 3)",
947
+ schema: { type: "object", properties: { build_done: { type: "boolean" }, status: { type: "string" } }, required: ["build_done"] },
948
+ timeoutMs: 270000 }
949
+ );
950
+ if (buildPoll && buildPoll.build_done) { break; }
951
+ }
952
+ if (!buildPoll || !buildPoll.build_done) {
953
+ buildPoll = { build_done: false, status: (buildPoll && buildPoll.status) || "build still running after the 10.5-minute bounded poll" };
954
+ }
910
955
  if (buildPoll.build_done) {
911
956
  // STEP 1c (mechanical): stamp provenance from workflow-computed values.
912
957
  var provStamp = await agent(
@@ -509,6 +509,12 @@ if (missingInitialPins.length > 0) {
509
509
  }
510
510
  log("Lifecycle scripts pinned to " + RUN_LIB);
511
511
 
512
+ // Merge-lock holder identity (bug 2fc8f52f): the opaque task+run identity
513
+ // minted at this run's first claim (never a PID — short-lived agent PIDs
514
+ // made every concurrent acquire take the stale path). Set exactly once on
515
+ // the first claim; stable for the rest of the run's lifetime.
516
+ let lockHolder = taskId;
517
+
512
518
  while (i < STEPS.length) {
513
519
  const step = STEPS[i];
514
520
  const isFirstClaim = (i === startStepIndex && totalReworkCount === 0);
@@ -692,6 +698,13 @@ while (i < STEPS.length) {
692
698
  activeSessionId = claimResult.session_id;
693
699
  }
694
700
 
701
+ // Mint the merge-lock holder identity once: task+session of this run's
702
+ // first claim. The task ID alone cannot distinguish two runs of the same
703
+ // task (re-dispatch after park), and a PID is not a valid holder at all.
704
+ if (lockHolder === taskId && activeSessionId) {
705
+ lockHolder = taskId + "/" + activeSessionId;
706
+ }
707
+
695
708
  // ── Capture: baseline evidence for experiential tasks ─────────────
696
709
  // Hazel's QA capture pass runs right after Triage, before Map, for tasks
697
710
  // Sage flagged experiential. The capture itself is parent-driven (the
@@ -860,11 +873,11 @@ while (i < STEPS.length) {
860
873
 
861
874
  } else if (step.name === "Integrate") {
862
875
  instructions = "Merge the approved task branch into main.\n\n" +
863
- "Run: "+ LIFECYCLE_ENV + " integrate " + taskId + " \"merge: " + safeTitle + "\"\n\n" +
876
+ "Run: "+ LIFECYCLE_ENV + "WORKFLOW_RUN_ID=" + lockHolder + " integrate " + taskId + " \"merge: " + safeTitle + "\"\n\n" +
864
877
  "Read the output:\n" +
865
878
  "- If it contains MERGED, integration succeeded. Report the merged commit hash.\n" +
866
879
  "- If it contains MERGED_EMPTY, the branch had no commits ahead of main (a runtime-state deliverable, declared by Build as repo_diff: none). Integration succeeded vacuously: the merge lock was NOT taken and there is no new commit. Report 'merged empty: no repo changes — deliverable was runtime state', then end your report with exactly this line: VERDICT: PASS. SKIP STEP 2 (push): there is no new commit to push.\n" +
867
- "- If it contains LOCK_HELD, another task holds the merge lock (mid Integrate/Publish). Report 'merge lock held', then end your report with exactly this line: VERDICT: FAIL.\n" +
880
+ "- If it contains LOCK_HELD, another task holds the merge lock (mid Integrate/Publish) and the 10-minute bounded backoff is exhausted. Report 'merge lock held after bounded backoff', then end your report with exactly this line: VERDICT: FAIL.\n" +
868
881
  "- If it contains CONFLICT, the plain merge failed — the merge was aborted, main is clean, and your task still holds the merge lock. Do NOT fail yet. Resolve it:\n" +
869
882
  "RESOLUTION:\n" +
870
883
  "R1. Refresh the merge lock FIRST (a long resolution must not silently lose the lock to the orphan sweep): "+ LIFECYCLE_ENV + " refresh-lock " + taskId + ". Create a scratch worktree WITH A NEW BRANCH (main is already checked out in the repo checkout, so git forbids checking it out a second time): cd " + REPO_PATH + " && git worktree add -b resolve/" + taskId + " /tmp/crew-resolve-" + taskId + " main. Reproduce the conflict in the scratch worktree: cd /tmp/crew-resolve-" + taskId + " && git merge " + TASK_BRANCH + ". This reproduces the exact conflict (main has not moved — the lock was held throughout). The task branch " + TASK_BRANCH + " is never modified.\n" +
@@ -952,14 +965,46 @@ while (i < STEPS.length) {
952
965
  }
953
966
  var publishFailure = null;
954
967
  if (rebuildTrigger.edit_started) {
955
- // STEP 1b (mechanical): bounded poll for build completion.
956
- var buildPoll = await agent(
957
- "Poll artifact_status for slug \"" + PUBLISH_SLUG + "\" until no build is running. Check every 30 seconds, up to 20 checks (10 minutes max).\n" +
958
- "Return JSON { \"build_done\": <true if no build is running within budget, false on timeout>, \"status\": \"<final status or timeout note>\" } and nothing else.",
959
- { key: attemptKey("publish-artifact-poll-" + taskId, totalReworkCount), label: "Waiting for artifact build to complete",
960
- schema: { type: "object", properties: { build_done: { type: "boolean" }, status: { type: "string" } }, required: ["build_done"] },
961
- timeoutMs: 660000 }
962
- );
968
+ // STEP 1b (mechanical): bounded poll for build completion, chunked so
969
+ // the merge-lock lease is refreshed before it can expire. The 600s
970
+ // lease is shorter than the worst-case 10-minute build poll, so the
971
+ // poll runs in three chunks (7 checks x 30s ~= 3.5 min each) with a
972
+ // holder-only lease refresh between chunks. If a refresh fails, the
973
+ // lock was lost: stop the run and park the task never continue to
974
+ // a provenance stamp or version assignment without holding the lock.
975
+ var buildPoll = null;
976
+ for (var chunk = 1; chunk <= 3; chunk++) {
977
+ if (chunk > 1) {
978
+ var refreshPoll = await agent(
979
+ "Refresh the merge lock for task " + taskId + ".\n" +
980
+ "Run: " + LIFECYCLE_ENV + " refresh-lock " + taskId + "\n" +
981
+ "If the output contains REFRESHED, return JSON { \"held\": true } and nothing else. Otherwise return JSON { \"held\": false, \"output\": \"<verbatim output>\" } and nothing else.",
982
+ { key: attemptKey("publish-lock-refresh-poll-" + taskId + "-c" + chunk, totalReworkCount), label: "Refreshing merge lock during build poll",
983
+ schema: { type: "object", properties: { held: { type: "boolean" }, output: { type: "string" } }, required: ["held"] },
984
+ timeoutMs: 60000 }
985
+ );
986
+ if (!refreshPoll || refreshPoll.held !== true) {
987
+ return await parkTask("Merge-lock lease lost during the artifact build poll (refresh before chunk " + chunk + " of 3 failed: " + ((refreshPoll && refreshPoll.output) || "no output") + "). Fail-closed: stopping before any provenance stamp or version assignment.");
988
+ }
989
+ }
990
+ // Chunk 1 keeps the original stable key; later chunks use -c<N>
991
+ // suffixed keys. All are attemptKey-scoped so rework passes stay
992
+ // disjoint (replay-key contract).
993
+ var pollKey = (chunk === 1)
994
+ ? attemptKey("publish-artifact-poll-" + taskId, totalReworkCount)
995
+ : attemptKey("publish-artifact-poll-" + taskId + "-c" + chunk, totalReworkCount);
996
+ buildPoll = await agent(
997
+ "Poll artifact_status for slug \"" + PUBLISH_SLUG + "\" until no build is running. Check every 30 seconds, up to 7 checks (3.5 minutes max).\n" +
998
+ "Return JSON { \"build_done\": <true if no build is running within budget, false on timeout>, \"status\": \"<final status or timeout note>\" } and nothing else.",
999
+ { key: pollKey, label: "Waiting for artifact build to complete (chunk " + chunk + " of 3)",
1000
+ schema: { type: "object", properties: { build_done: { type: "boolean" }, status: { type: "string" } }, required: ["build_done"] },
1001
+ timeoutMs: 270000 }
1002
+ );
1003
+ if (buildPoll && buildPoll.build_done) { break; }
1004
+ }
1005
+ if (!buildPoll || !buildPoll.build_done) {
1006
+ buildPoll = { build_done: false, status: (buildPoll && buildPoll.status) || "build still running after the 10.5-minute bounded poll" };
1007
+ }
963
1008
  if (buildPoll.build_done) {
964
1009
  // STEP 1c (mechanical): stamp provenance from workflow-computed values.
965
1010
  var provStamp = await agent(