ace-overseer 0.30.1 → 0.30.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c02cb4e6cc0d37cd3db64cfb66c25acfdf35d9d52dbbf9d50fda87d21ffa6359
4
- data.tar.gz: 339ddccc9081d5905d8a00178c8144925504a53f48e8baf29f135d6a7543b85a
3
+ metadata.gz: d988bf0e0a1bbd916a1612f1e8172f59bc9eedfb2c93660e52c59e5bc03562ff
4
+ data.tar.gz: a16e7ec132f9dc325ed860f944c578bdf7ece8b2ef44bd44c2de3ba188387df4
5
5
  SHA512:
6
- metadata.gz: 993aa5aa36125ca59af277d7e4b1c22ad4aec9cd23e048ecc75a2d8684e51b0a6aa8d429c2ad8cf02aae9e8bd3e849e71ea1c95c9d4bde747fb0b604419a055a
7
- data.tar.gz: c9236b380d6db7507402ffe6ae049248c191707fc9857c66b02d1672c598045831b1bf971d64fc69e6b05da29219ef64788a04b3f031f574c97fd8c33abf2e5e
6
+ metadata.gz: 0b5415e44739dc7f32e903c9d70860bf351419a7ef3532d90c53c54e9c3756d86bb675a4d836722fbab638ab5be12211d9d74788c1c33885083670b2f66b087c
7
+ data.tar.gz: e95bb169b326d8b27c5c764c6960e58ddd402247b8d270347099490f6a0a398d599b097a92bcee55a5fac85868b241f58e4056b00b667067f0d64028396e7ff0
data/CHANGELOG.md CHANGED
@@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.30.2] - 2026-11-12
11
+
12
+ ### Fixed
13
+
14
+ - Resume a failed executor in a fresh pane after native terminal identity replacement, preserving assignment/session and reconciling uncertain dispatch without duplicates.
15
+ - Retry failed session resumption only after native proof that the original terminal contains only its shell.
16
+ - Retrying work-on resumes a positively dead executor's existing Pi transcript and running assignment in its original terminal with fresh role credentials; uncertain start or prompt outcomes cannot duplicate execution.
17
+ - Project-session observation recognizes one live replacement when a dead predecessor panel remains after restart, matching managed start ownership checks.
18
+ - Judge recovery progress from changed owner artifacts against the recorded baseline, rather than an unchanged native state-transition counter or missing final report.
19
+ - Distinguish same-session continuation after provider failure from duplicate startup dispatch; reconcile current owners before acknowledging completion and bound recovery with the existing eyes deadline.
20
+ - Observe native worker turn completions even when provider failures leave a live panel without an assignment report; unchanged completion sequences remain silent.
21
+ - Preserve native worker launch receipts while Pi publishes its session identity; resume the same verified idle executor without duplicating a process or an uncertain prompt.
22
+ - Direct Forgejo sessions to their configured client before treating anonymous API refusal as missing access or requesting temporary credentials.
23
+ - Namespace native overseer agent names by canonical root so lab and project sessions coexist on one Herdr server.
24
+ - Consume configured project reports from the lab, bind all outcomes to their project, and resolve completion and continuation in the project checkout while retaining the lab task owner.
25
+ - Specify the supported bootstrap workflow command and refuse substituting checkout instructions in installed mode.
26
+
10
27
  ## [0.30.1] - 2026-11-11
11
28
 
12
29
  ### Fixed
@@ -249,7 +249,7 @@ the path returned by `ace-task show` supplement these fields, never replace them
249
249
  Read back the recorded identity before acknowledging the eyes revision.
250
250
  Never put a preparation readiness ID on a worker, verifier or confirm-start step.
251
251
 
252
- After accepting the worker's report, use `ace-overseer process-reports` to
252
+ After accepting the worker's report, use `ace-overseer process-reports --owner <configured-lab-recipient>` to
253
253
  retire its recorded executor and continue the prepared sequence. Preserve
254
254
  the exact report and assignment identity before starting the verifier.
255
255
  Do not launch a second executor while a predecessor is active or unknown.
@@ -61,9 +61,12 @@ Captain order.
61
61
  1. Read the lab topology, current private tasks and real assignment/session
62
62
  state. Resume this commissioned bootstrap before unrelated backlog. Resolve
63
63
  actual project paths; ask only about ambiguous scope or missing authority.
64
- 2. Read `wfi://overseer/bootstrap-delegation` through `ace-bundle` to understand
64
+ 2. Run exactly `ace-bundle wfi://overseer/bootstrap-delegation` (there is no
65
+ `read` subcommand). Read the output file that command returns to understand
65
66
  the task → worker → observation → verification → start procedure. Do not
66
- dispatch during this initialization turn.
67
+ dispatch during this initialization turn. In installed mode use the packaged
68
+ workflow, never a similarly named file in the project checkout. If resolution
69
+ fails, record that error and end the turn; do not substitute checkout instructions.
67
70
  3. Arm observation with the `ace_eyes` tool, action `start`, using the lab's
68
71
  eyes configuration path `<lab>/.ace-local/overseer/eyes.json`. Check enabled=true, breaker=null, errors=0 and
69
72
  a successful observation with every required source healthy before dispatch.
@@ -160,7 +160,7 @@ If the exact session is still idle, interactive-ready, has no
160
160
  completed turn, and the pinned assignment has no activity or report, record
161
161
  one recovery attempt before sending the original commission to that same
162
162
  terminal. The commission must still name that exact assignment and role.
163
- Never submit a second dispatch or prompt a session that has begun work.
163
+ Never repeat the original startup dispatch or commission after a session has begun work.
164
164
  After confirmed recovery delivery, arm one follow-up deadline
165
165
  `dispatch-recovery-<id>` with the same assignment revision and a fresh
166
166
  300000-millisecond window. Cancel it on assignment activity or a report.
@@ -171,6 +171,60 @@ delivery was confirmed but the assignment still has no activity at the
171
171
  deadline. Keep the unfinished assignment pending or blocked. Reconcile
172
172
  reports through their owners before acknowledging events.
173
173
 
174
+ ### Interrupted executor processes
175
+
176
+ When native Herdr positively reports that a previously running executor is
177
+ absent, preserve its assignment and worktree. Assess unfinished work with
178
+ `ace-coworker resume --config <recorded-prepare-config> --dry-run` from that
179
+ worktree. For a running assignment, retry
180
+ `ace-overseer work-on --task <same-project-task> --runtime herdr` from the
181
+ project root. This resumes the recorded Pi transcript with fresh role credentials
182
+ and a continuation prompt. Normally it uses the original terminal. After a failed
183
+ resume and native terminal replacement, the product may verify executor absence
184
+ and resume that same transcript in a fresh pane, leaving the old pane untouched.
185
+ It preserves the assignment and does not advance to the next role. Record the
186
+ returned pane/session identity and any explicit previous-terminal receipt; arm a
187
+ single five-minute recovery deadline and cancel on verified progress.
188
+ Do not use `handoff` to bypass an unfinished role. Missing transcript, unverified
189
+ terminal replacement or uncertain start/prompt remains a blocker: preserve the
190
+ binding and escalate; never clear it or mark steps complete to force a launch.
191
+
192
+ ### Ended worker turns and provider failures
193
+
194
+ An executor event's `completion_seq` means a native turn ended, not that its
195
+ assignment succeeded. Before acknowledging that revision, read the current
196
+ native session, pinned assignment and reports. Do not reuse an earlier
197
+ `working` observation. If the assignment is complete, adopt its report through
198
+ `process-reports`. If the session is currently working, leave it alone.
199
+ `state_change_seq` records state transitions, not a progress heartbeat: an
200
+ unchanged value while working does not establish a stall. At a recovery
201
+ deadline, inspect the active command and current owner artifacts before
202
+ escalating. A reviewer may have saved rounds in `.ace-local/review/sessions/`,
203
+ feedback dispositions, commits or a new PR HEAD while its assignment step is
204
+ still unfinished; preserve those results and continue that step. Count progress
205
+ only when artifacts are new or changed since the recovery baseline recorded
206
+ in the existing coordination leg (assignment revision, review round, commit or
207
+ PR HEAD). Historical artifacts alone cannot cancel its deadline. Absence of a
208
+ final assignment report does not mean absence of work.
209
+
210
+ If the same interactive session ended with a provider error and its assignment
211
+ is unfinished, inspect its last output and recorded progress. Record recovery
212
+ in the existing coordination leg, then send a continuation to that same
213
+ terminal naming the assignment and unfinished step. Preserve completed work,
214
+ commits and PRs; do not resend the original commission or create another agent.
215
+ This continuation is distinct from startup recovery above. Acknowledge the
216
+ handled revision only after recording the outcome, then end the turn with eyes
217
+ armed. A prompt receipt alone does not prove progress. After submission, arm
218
+ one existing eyes deadline `provider-recovery-<assignment>` pinned to the
219
+ assignment revision, with a 300000-millisecond window. Cancel it only on
220
+ verified assignment or work-product progress. Do not rearm it merely because
221
+ another timeout or completion event arrived. On expiry, inspect the current
222
+ owners; if there has been no progress, stop automatic retries and escalate the
223
+ concrete provider failure through HITL. Preserve the deadline while handling
224
+ repeated failures so a stalled continuation cannot silently wait forever.
225
+ An unknown outcome or ambiguous session also requires reconciliation/escalation,
226
+ not a speculative continuation.
227
+
174
228
  ## Escalation (HITL)
175
229
 
176
230
  - Decisions outside your granted autonomy go to the Captain:
@@ -224,10 +278,9 @@ settlement alone is not proof of handling.
224
278
  grant. Credentials and OTP values never enter prompts, reports or tracked
225
279
  files. Configured credential files must remain private (`0600`).
226
280
  When forge access is missing, do not search host credential stores or set up
227
- a persistent `fj` login yourself. Request the existing session-scoped auth
228
- handoff for the required role and repository using the procedure below; if
229
- it is unavailable, create a precise HITL blocker and end the turn. Never ask a human
230
- to paste a token or replace the lease with a durable shared credential.
281
+ a persistent `fj` login yourself. Use the configured client as described below.
282
+ If that client reports missing access, record its exact error and end the turn.
283
+ Never ask a human to paste a token.
231
284
  - Remote issue and PR observation (`observe.remote.issues` and
232
285
  `observe.remote.pull_requests`) and the HITL reply
233
286
  receiver (`ace-hitl reply CASE --stdin`) are delivered surfaces: watch the
@@ -239,7 +292,12 @@ settlement alone is not proof of handling.
239
292
  ## Project-scoped forge access
240
293
 
241
294
  The session starts with the selected project's read or role credential in its
242
- environment or private provider config. Use that configured forge normally.
295
+ environment or private provider config. For Forgejo, use the installed `fj`
296
+ client with the configured host/repository (consult its help for command syntax).
297
+ It reads its prepared login from the session's XDG_DATA_HOME; no interactive login
298
+ or auth lease request is needed for a static role credential. An anonymous curl
299
+ 403 is not evidence that the configured client lacks access. Do not probe host
300
+ lease inventory or search credential stores before trying the configured client.
243
301
  Continuous observation needs a read credential valid for the observer's
244
302
  lifetime. An unavailable login is a visible configuration blocker; do not
245
303
  read host administrator credentials or invent another login method.
@@ -47,6 +47,9 @@ module Ace
47
47
  def self.evaluate(event:, managers:)
48
48
  return {status: :unmatched, reason: "event has no report identity"} if event["id"].to_s.strip.empty?
49
49
 
50
+ if event["project_root"]
51
+ managers = managers.select { |manager| manager.metadata["project_root"] == event["project_root"] }
52
+ end
50
53
  matched = match(event, managers)
51
54
  return {status: :unmatched, reason: unmatched_reason(event)} unless matched
52
55
 
@@ -1,5 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "shellwords"
3
4
  require "ace/coworker/molecules/execution_binding"
4
5
  require "ace/herdr"
5
6
  require "ace/herdr/atoms/native_agent_session"
@@ -12,7 +13,9 @@ module Ace::Overseer::Molecules
12
13
  class CoworkerLauncher
13
14
  def initialize(executor: Ace::Herdr::Molecules::HerdrExecutor.new, env: ENV,
14
15
  environment: nil, bindings: Ace::Coworker::Molecules::ExecutionBinding.new,
15
- brief_factory: nil, readiness: nil, eyes: nil, root_resolver: nil)
16
+ brief_factory: nil, readiness: nil, eyes: nil, root_resolver: nil,
17
+ identity_timeout: 10, clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }, sleeper: ->(seconds) { sleep(seconds) })
18
+ @identity_timeout, @clock, @sleeper = identity_timeout, clock, sleeper
16
19
  @executor, @env, @bindings = executor, env, bindings
17
20
  @environment = environment || SessionEnvironment.new(env: env)
18
21
  @brief_factory = brief_factory || ->(root, path) { Ace::Coworker::Organisms::StartOrchestrator.new(project_root: root).call(config_path: path) }
@@ -48,7 +51,18 @@ module Ace::Overseer::Molecules
48
51
  raise Ace::Overseer::Error, "executor commission changed during validation"
49
52
  end
50
53
  if document["execution"]
51
- next reuse(document.fetch("execution"))
54
+ execution = document.fetch("execution")
55
+ if %w[resuming resumed].include?(execution["state"])
56
+ next finish_resume(execution, brief, worktree_path, save)
57
+ end
58
+ if %w[launching started].include?(execution["state"])
59
+ unless %w[paused empty].include?(brief.assignment_snapshot&.state.to_s)
60
+ raise Ace::Overseer::Error, "unconfirmed executor already has assignment activity"
61
+ end
62
+ recover_unprompted(execution, worktree_path, save)
63
+ next submit_prompt(execution, brief, save)
64
+ end
65
+ next reuse(execution, brief, worktree_path, save)
52
66
  end
53
67
  unless %w[paused empty].include?(brief.assignment_snapshot&.state.to_s)
54
68
  raise Ace::Overseer::Error, "unbound assignment is not ready for a new executor"
@@ -72,20 +86,14 @@ module Ace::Overseer::Molecules
72
86
  result = dispatcher.dispatch(label: name, prompt: "", kind: "pi", cwd: worktree_path,
73
87
  env_file: file, args: Array(profile_args) + ["--append-system-prompt", brief.instructions_source])
74
88
  execution.merge!(result.to_h).merge!("state" => "started")
89
+ save.call # Preserve the native receipt before asynchronous session publication.
75
90
  pane = pane_identity(execution.fetch("pane"))
76
91
  execution["terminal_id"] = pane.fetch("terminal_id")
77
- execution["agent_session"] = session_identity(pane)
92
+ save.call
93
+ execution["agent_session"] = await_session(execution)
78
94
  save.call
79
95
  end
80
- # Record uncertainty BEFORE submission. A crash after input reaches Pi
81
- # cannot cause the same task to be submitted again automatically.
82
- execution["state"] = "prompting"
83
- save.call
84
- prompt = "#{brief.config.start_prompt}\n\n#{Ace::Coworker::Atoms::BriefFormatter.format_table(brief)}"
85
- @executor.agent_prompt(pane: execution.fetch("pane"), text: prompt)
86
- execution["state"] = "running"
87
- save.call
88
- execution.reject { |key, _| key == "credentials_directory" }.merge("reused" => false)
96
+ submit_prompt(execution, brief, save)
89
97
  end
90
98
  rescue Ace::Coworker::Error, Ace::Herdr::ExecutorError => e
91
99
  raise Ace::Overseer::Error, "executor launch blocked: #{e.message}"
@@ -120,7 +128,60 @@ module Ace::Overseer::Molecules
120
128
 
121
129
  private
122
130
 
123
- def reuse(execution)
131
+ def submit_prompt(execution, brief, save)
132
+ # A crash after input reaches Pi must never repeat the commission.
133
+ execution["state"] = "prompting"
134
+ save.call
135
+ prompt = "#{brief.config.start_prompt}\n\n#{Ace::Coworker::Atoms::BriefFormatter.format_table(brief)}"
136
+ @executor.agent_prompt(pane: execution.fetch("pane"), text: prompt)
137
+ execution["state"] = "running"
138
+ save.call
139
+ execution.reject { |key, _| key == "credentials_directory" }.merge("reused" => false)
140
+ end
141
+
142
+ def await_session(execution)
143
+ deadline = @clock.call + @identity_timeout
144
+ loop do
145
+ pane = pane_identity(execution.fetch("pane"))
146
+ unless pane["terminal_id"] == execution.fetch("terminal_id")
147
+ raise Ace::Overseer::Error, "executor terminal changed before session publication"
148
+ end
149
+ identity = Ace::Herdr::Atoms::NativeAgentSession.normalize(pane["agent_session"], agent: "pi")
150
+ return identity if identity
151
+ raise Ace::Overseer::Error, "native executor session identity is unavailable; retry the same handoff" if @clock.call >= deadline
152
+ @sleeper.call(0.2)
153
+ end
154
+ end
155
+
156
+ def recover_unprompted(execution, worktree_path, save)
157
+ row = @executor.agent_get(execution["pane"] || execution.fetch("agent")).parsed_json&.dig("result", "agent")
158
+ unless row.is_a?(Hash) && row["name"] == execution["agent"] &&
159
+ row["cwd"] == worktree_path && row["agent_status"] == "idle" &&
160
+ row["interactive_ready"] == true && row["pane_id"] && row["terminal_id"]
161
+ raise Ace::Overseer::Error, "unconfirmed launch is not the expected idle executor"
162
+ end
163
+ if execution["terminal_id"] && execution["terminal_id"] != row["terminal_id"]
164
+ raise Ace::Overseer::Error, "executor terminal changed before recovery"
165
+ end
166
+ identity = session_identity(row)
167
+ if execution["agent_session"] && execution["agent_session"] != identity
168
+ raise Ace::Overseer::Error, "executor session changed before recovery"
169
+ end
170
+ # Without a saved Pi identity, bind only the unique private session
171
+ # directory allocated for this dispatch, never an arbitrary same-name pane.
172
+ unless execution["agent_session"]
173
+ directory = execution["credentials_directory"]
174
+ native_session = row.fetch("agent_session")
175
+ unless directory && native_session["kind"] == "path" && native_session["value"].start_with?("#{directory}/sessions/")
176
+ raise Ace::Overseer::Error, "unconfirmed launch has no matching session provenance"
177
+ end
178
+ end
179
+ execution.merge!("pane" => row["pane_id"], "terminal_id" => row["terminal_id"],
180
+ "agent_session" => identity, "state" => "started")
181
+ save.call
182
+ end
183
+
184
+ def reuse(execution, brief, worktree_path, save)
124
185
  if execution["state"] == "retired"
125
186
  raise Ace::Overseer::Error, "recorded executor is retired; reconcile the assignment before starting another"
126
187
  end
@@ -128,16 +189,170 @@ module Ace::Overseer::Molecules
128
189
  raise Ace::Overseer::Error, "executor start is unconfirmed; inspect the recorded assignment before retrying"
129
190
  end
130
191
  pane = pane_identity(execution.fetch("pane"))
131
- unless same_executor?(pane, execution)
192
+ unless pane["terminal_id"] == execution["terminal_id"]
132
193
  raise Ace::Overseer::Error, "recorded executor terminal no longer matches"
133
194
  end
134
- agent = @executor.agent_get(execution.fetch("pane")).parsed_json&.dig("result", "agent")
195
+ begin
196
+ agent = @executor.agent_get(execution.fetch("pane")).parsed_json&.dig("result", "agent")
197
+ rescue Ace::Herdr::AgentNotFoundError
198
+ return resume_dead(execution, brief, worktree_path, pane, save)
199
+ end
135
200
  unless agent.is_a?(Hash) && agent["interactive_ready"] == true && same_executor?(agent, execution)
136
201
  raise Ace::Overseer::Error, "recorded executor is not a live interactive agent"
137
202
  end
138
203
  execution.reject { |key, _| key == "credentials_directory" }.merge("reused" => true)
139
204
  end
140
205
 
206
+ def resume_dead(execution, brief, worktree_path, pane, save)
207
+ unless brief.assignment_snapshot&.state.to_s == "running" && pane["cwd"] == worktree_path
208
+ raise Ace::Overseer::Error, "dead executor recovery requires its running assignment and original directory"
209
+ end
210
+ transcript = resume_transcript(execution, worktree_path)
211
+ SessionEnvironment.cleanup(execution["credentials_directory"])
212
+ @environment.with_environment(root: worktree_path, role: brief.config.role) do |file, directory, args|
213
+ execution.merge!("state" => "resuming", "resume_transcript" => transcript,
214
+ "credentials_directory" => directory)
215
+ save.call # An uncertain native start must be inspected, never repeated.
216
+ @executor.agent_start(name: execution.fetch("agent"), kind: "pi", pane: execution.fetch("pane"),
217
+ timeout_ms: 60_000, env_file: file,
218
+ args: Array(args) + ["--session", transcript, "--append-system-prompt", brief.instructions_source])
219
+ identity = await_session(execution)
220
+ raise Ace::Overseer::Error, "resumed executor session changed" unless identity == execution["agent_session"]
221
+ execution["state"] = "resumed"
222
+ save.call
223
+ end
224
+ finish_resume(execution, brief, worktree_path, save)
225
+ end
226
+
227
+ def finish_resume(execution, brief, worktree_path, save)
228
+ raise Ace::Overseer::Error, "resumed assignment is no longer running" unless brief.assignment_snapshot&.state.to_s == "running"
229
+ if execution["replacement_pending"]
230
+ recover_replacement(execution, worktree_path, save)
231
+ end
232
+ begin
233
+ agent = @executor.agent_get(execution.fetch("pane")).parsed_json&.dig("result", "agent")
234
+ rescue Ace::Herdr::AgentNotFoundError
235
+ raise unless execution["state"] == "resuming"
236
+ pane = pane_identity(execution.fetch("pane"))
237
+ if pane["terminal_id"] != execution["terminal_id"]
238
+ return replace_resume_terminal(execution, brief, worktree_path, pane, save)
239
+ end
240
+ pane = confirmed_resume_shell(execution, worktree_path)
241
+ return resume_dead(execution, brief, worktree_path, pane, save)
242
+ end
243
+ unless agent.is_a?(Hash) && same_executor?(agent, execution) && agent["cwd"] == worktree_path &&
244
+ agent["name"] == execution["agent"] && agent["interactive_ready"] == true && agent["agent_status"] == "idle"
245
+ raise Ace::Overseer::Error, "resumed executor is not the expected idle session; inspect before retrying"
246
+ end
247
+ execution["state"] = "resume_prompting"
248
+ save.call
249
+ @executor.agent_prompt(pane: execution.fetch("pane"), text:
250
+ "Continue the existing assignment #{brief.config.assignment} after process interruption. " \
251
+ "Read its current owner state and preserved reports; resume unfinished work only. " \
252
+ "Do not repeat completed effects or create a replacement assignment. " \
253
+ "Use ace-coworker resume --config #{Shellwords.escape(brief.config.source_path)} to assess continuation.")
254
+ execution["state"] = "running"
255
+ save.call
256
+ execution.reject { |key, _| key == "credentials_directory" }.merge("reused" => true, "resumed" => true)
257
+ end
258
+
259
+ def resume_inventory
260
+ rows = @executor.agent_list.parsed_json&.dig("result", "agents")
261
+ unless rows.is_a?(Array) && rows.all? { |row| row.is_a?(Hash) && row["name"].is_a?(String) && row["pane_id"].is_a?(String) &&
262
+ Ace::Herdr::Atoms::NativeAgentSession.normalize(row["agent_session"], agent: row["agent"]) }
263
+ raise Ace::Overseer::Error, "native executor inventory is incomplete"
264
+ end
265
+ rows
266
+ end
267
+
268
+ def replacement_matches(execution)
269
+ resume_inventory.select do |row|
270
+ row["name"] == execution["agent"] ||
271
+ Ace::Herdr::Atoms::NativeAgentSession.normalize(row["agent_session"], agent: "pi") == execution["agent_session"]
272
+ end
273
+ end
274
+
275
+ def recover_replacement(execution, worktree_path, save)
276
+ rows = replacement_matches(execution)
277
+ row = rows.one? && rows.first
278
+ unless row && row["name"] == execution["agent"] && row["cwd"] == worktree_path && row["terminal_id"] &&
279
+ row["interactive_ready"] == true && row["agent_status"] == "idle" && session_identity(row) == execution["agent_session"]
280
+ raise Ace::Overseer::Error, "replacement resume is unconfirmed; inspect before retrying"
281
+ end
282
+ execution.merge!("pane" => row["pane_id"], "terminal_id" => row["terminal_id"])
283
+ execution.delete("replacement_pending")
284
+ save.call
285
+ end
286
+
287
+ def replace_resume_terminal(execution, brief, worktree_path, pane, save)
288
+ transcript = resume_transcript(execution, worktree_path)
289
+ info = @executor.pane_process_info(execution.fetch("pane")).parsed_json&.dig("result", "process_info")
290
+ unless pane["cwd"] == worktree_path && shell_only?(info, execution["pane"]) && replacement_matches(execution).empty?
291
+ raise Ace::Overseer::Error, "old executor absence is unconfirmed; replacement refused"
292
+ end
293
+ # A native live handoff may replace terminal identity. Never send input
294
+ # to or close that old pane; resume the same Pi transcript in a new pane.
295
+ SessionEnvironment.cleanup(execution["credentials_directory"])
296
+ @environment.with_environment(root: worktree_path, role: brief.config.role) do |file, directory, args|
297
+ execution["previous_terminal"] = execution.slice("pane", "terminal_id")
298
+ execution.merge!("replacement_pending" => true, "credentials_directory" => directory, "resume_transcript" => transcript)
299
+ save.call
300
+ dispatcher = Ace::Herdr::Organisms::Dispatcher.from_config(executor: @executor)
301
+ result = dispatcher.dispatch(label: execution.fetch("agent"), prompt: "", kind: "pi", cwd: worktree_path,
302
+ workspace_id: execution["workspace"], env_file: file,
303
+ args: Array(args) + ["--session", transcript, "--append-system-prompt", brief.instructions_source])
304
+ execution.merge!(result.to_h)
305
+ execution["terminal_id"] = pane_identity(execution.fetch("pane")).fetch("terminal_id")
306
+ save.call
307
+ identity = await_session(execution)
308
+ raise Ace::Overseer::Error, "replacement executor session changed" unless identity == execution["agent_session"]
309
+ execution.delete("replacement_pending")
310
+ save.call
311
+ end
312
+ finish_resume(execution, brief, worktree_path, save)
313
+ end
314
+
315
+ def shell_only?(info, pane)
316
+ info.is_a?(Hash) && info["pane_id"] == pane && info["shell_pid"].is_a?(Integer) && info["shell_pid"].positive? &&
317
+ info["foreground_process_group_id"] == info["shell_pid"] && info["foreground_processes"].is_a?(Array) &&
318
+ info["foreground_processes"].length == 1 && info["foreground_processes"].first.is_a?(Hash) &&
319
+ info["foreground_processes"].first["pid"] == info["shell_pid"]
320
+ end
321
+
322
+ def confirmed_resume_shell(execution, worktree_path)
323
+ pane = pane_identity(execution.fetch("pane"))
324
+ info = @executor.pane_process_info(execution.fetch("pane")).parsed_json&.dig("result", "process_info")
325
+ unless info.is_a?(Hash) && pane["terminal_id"] == execution["terminal_id"] && pane["cwd"] == worktree_path &&
326
+ info["pane_id"] == execution["pane"] && info.dig("guarded_prompt_origin", "terminal_id") == execution["terminal_id"] &&
327
+ info["guarded_prompt"] == true && info["guarded_input_drain"] == true &&
328
+ shell_only?(info, execution["pane"])
329
+ raise Ace::Overseer::Error, "failed resume has no confirmed original shell; inspect before retrying"
330
+ end
331
+ pane
332
+ end
333
+
334
+ def resume_transcript(execution, worktree_path)
335
+ candidates = if execution["resume_transcript"]
336
+ [execution.fetch("resume_transcript")]
337
+ else
338
+ directory = execution["credentials_directory"]
339
+ raise Ace::Overseer::Error, "executor has no private session directory" unless directory
340
+ Dir.glob(File.join(directory, "sessions", "**", "*.jsonl")).select do |path|
341
+ Ace::Herdr::Atoms::NativeAgentSession.normalize({"agent" => "pi", "kind" => "path", "value" => path}, agent: "pi") == execution["agent_session"]
342
+ end
343
+ end
344
+ raise Ace::Overseer::Error, "executor transcript is missing or ambiguous" unless candidates.length == 1
345
+ path = candidates.first
346
+ raise Ace::Overseer::Error, "executor transcript is not a regular file" unless File.file?(path) && !File.symlink?(path)
347
+ header = File.open(path) { |file| JSON.parse(file.readline) }
348
+ unless header["type"] == "session" && header["id"] == execution.dig("agent_session", "value") && header["cwd"] == worktree_path
349
+ raise Ace::Overseer::Error, "executor transcript identity does not match"
350
+ end
351
+ path
352
+ rescue JSON::ParserError, EOFError, SystemCallError
353
+ raise Ace::Overseer::Error, "executor transcript is unavailable or invalid"
354
+ end
355
+
141
356
  def pane_identity(pane)
142
357
  reply = @executor.pane_get(pane).parsed_json
143
358
  row = reply&.dig("result", "pane")
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "ace/hitl/molecules/event_observation"
4
+ require_relative "project_observations"
4
5
 
5
6
  module Ace
6
7
  module Overseer
@@ -12,13 +13,13 @@ module Ace
12
13
  # until answered — both are the overseer's inbox.
13
14
  class HitlInbox
14
15
  # @param manager [Ace::Hitl::Organisms::HitlManager, nil]
15
- def initialize(manager: nil, owner: "overseer", scope: nil)
16
+ def initialize(manager: nil, owner: "overseer", scope: nil, projects: ProjectObservations.new)
16
17
  @manager = manager || Ace::Hitl::Organisms::HitlManager.new
17
18
  raise ArgumentError, "configured HITL owner is required" if owner.to_s.strip.empty?
18
19
  unless scope.nil? || %w[current all].include?(scope)
19
20
  raise ArgumentError, "HITL scope must be current or all"
20
21
  end
21
- @owner, @scope = owner, scope
22
+ @owner, @scope, @projects = owner, scope, projects
22
23
  @observation = Ace::Hitl::Molecules::EventObservation.new(manager: @manager)
23
24
  end
24
25
 
@@ -30,7 +31,19 @@ module Ace
30
31
  events = @observation.incoming_events(owner: @owner, scope: @scope).select do |event|
31
32
  event.type == "report" || (event.type == "question" && event.status == "pending")
32
33
  end
33
- events.sort_by { |event| event.created_at || Time.now }.map { |event| normalize(event) }
34
+ local = events.sort_by { |event| event.created_at || Time.now }.map { |event| normalize(event) }
35
+ projects = @projects.call(owner: @owner, scope: @scope || "all", limit: 500)
36
+ unless projects.fetch("sources").values.all? { |source| source["status"] == "ok" }
37
+ raise Ace::Overseer::Error, "project report sources are degraded; preserve pending reports"
38
+ end
39
+ remote = projects.fetch("events").select do |event|
40
+ event["type"] == "report" || (event["type"] == "question" && event["status"] == "pending")
41
+ end
42
+ combined = local + remote
43
+ if combined.group_by { |event| event.fetch("id") }.any? { |_, rows| rows.size > 1 }
44
+ raise Ace::Overseer::Error, "ambiguous report IDs across configured projects"
45
+ end
46
+ combined
34
47
  end
35
48
 
36
49
  private
@@ -195,7 +195,13 @@ module Ace
195
195
  cwd.is_a?(String) && cwd.start_with?("/")
196
196
  raise Error, "executor session identity is invalid"
197
197
  end
198
- sessions[cwd] << {"terminal" => terminal, "agent" => agent["name"]}
198
+ # A provider failure can end a turn while its panel stays alive and
199
+ # no report is written. Herdr exposes completion_seq when done and
200
+ # omits it while working: completion/resumption each revise once,
201
+ # while repeated observations of either state remain silent.
202
+ # This is a signal to inspect the assignment, never success proof.
203
+ sessions[cwd] << {"terminal" => terminal, "agent" => agent["name"],
204
+ "completion_seq" => agent["completion_seq"]}
199
205
  end
200
206
  sessions.transform_values { |rows| rows.sort_by { |row| row.fetch("terminal") } }
201
207
  end
@@ -74,6 +74,18 @@ module Ace
74
74
  end
75
75
 
76
76
 
77
+ def live_agent?(pane)
78
+ json = @executor.agent_get(pane).parsed_json
79
+ raise Error, "herdr returned malformed agent state" unless json.is_a?(Hash)
80
+ agent = json["agent"]
81
+ agent ||= json.dig("result", "agent") if json["result"].is_a?(Hash)
82
+ agent ||= json["result"] if json["result"].is_a?(Hash)
83
+ raise Error, "herdr returned no agent identity" unless agent.is_a?(Hash)
84
+ true
85
+ rescue Ace::Herdr::AgentNotFoundError
86
+ false
87
+ end
88
+
77
89
  def project_row(id, root, tabs, panes)
78
90
  label = Atoms::StartContract.session_label(root)
79
91
  owners = tabs.select { |tab| tab[:title] == label }.flat_map do |tab|
@@ -81,7 +93,12 @@ module Ace
81
93
  raise Error, "project overseer tab has no unique pane: #{label}" unless matches.length == 1
82
94
  matches
83
95
  end
84
- raise Error, "project overseer has ambiguous sessions: #{label}" if owners.length > 1
96
+ if owners.length > 1
97
+ # Reconciliation preserves dead panels for diagnostics. Native
98
+ # agent presence, not the retained tab, identifies the live owner.
99
+ owners = owners.select { |owner| live_agent?(owner.fetch(:id)) }
100
+ raise Error, "project overseer has ambiguous sessions: #{label}" unless owners.length == 1
101
+ end
85
102
  pane = owners.first
86
103
  raise Error, "project overseer session did not report a working directory: #{label}" if pane && pane[:cwd].to_s.empty?
87
104
  if pane && @root_resolver.resolve(cwd: pane[:cwd]) != root
@@ -7,7 +7,7 @@ module Ace
7
7
  # the queue accepts success. This check owns no persisted state.
8
8
  class ReportCompletionCheck
9
9
  def initialize(handoff_probe: nil, assignment_probe_factory: nil, worktree_finder: nil)
10
- @worktree_finder = worktree_finder || Ace::Git::Worktree::Organisms::WorktreeManager.new
10
+ @worktree_finder = worktree_finder
11
11
  @handoff_probe = handoff_probe || HandoffProbe.new
12
12
  @assignment_probe_factory = assignment_probe_factory || ->(root) {
13
13
  Ace::Coworker::Molecules::AssignmentProbe.new(project_root: root)
@@ -30,7 +30,13 @@ module Ace
30
30
  return "confirm-start requires coordinator session evidence" if role == "confirm-start"
31
31
  return "report assignment disagrees with the allocation" unless step.metadata["assignment"].to_s == assignment
32
32
 
33
- found = @worktree_finder.switch(task)
33
+ project_root = manager.metadata["project_root"].to_s
34
+ if event["project_root"] && event["project_root"] != project_root
35
+ return "report project disagrees with the allocation"
36
+ end
37
+ finder = @worktree_finder || Ace::Git::Worktree::Organisms::WorktreeManager.new(
38
+ project_root: project_root.empty? ? Dir.pwd : project_root)
39
+ found = Dir.chdir(project_root.empty? ? Dir.pwd : project_root) { finder.switch(task) }
34
40
  return "task has no usable worktree: #{found[:error] || found[:message]}" unless found[:success]
35
41
  root = found.fetch(:worktree_path)
36
42
  legs = @handoff_probe.probe(root, task_id: task).fetch(:legs)
@@ -20,7 +20,7 @@ module Ace
20
20
  @completion_check = completion_check || Molecules::ReportCompletionCheck.new
21
21
  @inbox = inbox || Molecules::HitlInbox.new(owner: owner, scope: scope)
22
22
  @task_manager = task_manager || Ace::Task::Organisms::TaskManager.new
23
- @handoff = handoff || Organisms::HandoffOrchestrator.new(task_loader: @task_manager)
23
+ @handoff = handoff
24
24
  @work_on = work_on || Organisms::WorkOnOrchestrator.new(task_loader: @task_manager)
25
25
  @retire = retire || Molecules::CoworkerLauncher.new.method(:retire)
26
26
  end
@@ -157,7 +157,15 @@ module Ace
157
157
 
158
158
  begin
159
159
  if family_worktree(manager)
160
- result = @handoff.call(task_ref: manager.metadata["project_task"].to_s)
160
+ project_root = manager.metadata["project_root"].to_s
161
+ result = if @handoff
162
+ @handoff.call(task_ref: manager.metadata["project_task"].to_s)
163
+ else
164
+ Dir.chdir(project_root.empty? ? Dir.pwd : project_root) do
165
+ Organisms::HandoffOrchestrator.new.call(task_ref: manager.metadata["project_task"].to_s,
166
+ runtime: Ace::Support::Cli::Toolchain.installed? ? "herdr" : nil)
167
+ end
168
+ end
161
169
  record_allocation(next_step, result) if result[:execution]
162
170
  {action: "handoff", step_ref: next_step.id.to_s,
163
171
  next: result[:next], status: result[:status], worktree_path: result[:worktree_path],
@@ -11,6 +11,7 @@ require "ace/herdr/molecules/installed_pi_extensions"
11
11
  require "ace/lab"
12
12
  require "ace/support/cli/toolchain"
13
13
  require "json"
14
+ require "digest"
14
15
  require "open3"
15
16
 
16
17
  module Ace
@@ -594,7 +595,7 @@ module Ace
594
595
  @executor.pane_run(pane, command)
595
596
  @executor.pane_wait_output(pane, pattern: marker, timeout_ms: START_TIMEOUT_MS)
596
597
  end
597
- @executor.agent_start(name: AGENT_NAME, kind: plan["profile"]["harness"],
598
+ @executor.agent_start(name: "#{AGENT_NAME}-#{Digest::SHA256.hexdigest(root)[0, 12]}", kind: plan["profile"]["harness"],
598
599
  pane: pane, timeout_ms: START_TIMEOUT_MS, args: args)
599
600
  end
600
601
  pane
@@ -611,7 +612,7 @@ module Ace
611
612
  credentials = nil
612
613
  Molecules::SessionEnvironment.new(env: @env).with_environment(root: root, role: role) do |file, directory, _profile|
613
614
  credentials = directory
614
- @executor.agent_start(name: AGENT_NAME, kind: plan["profile"]["harness"],
615
+ @executor.agent_start(name: "#{AGENT_NAME}-#{Digest::SHA256.hexdigest(root)[0, 12]}", kind: plan["profile"]["harness"],
615
616
  pane: pane, timeout_ms: START_TIMEOUT_MS, args: args, env_file: file)
616
617
  end
617
618
  rescue StandardError, Interrupt
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Ace
4
4
  module Overseer
5
- VERSION = "0.30.1"
5
+ VERSION = "0.30.2"
6
6
  end
7
7
  end
metadata CHANGED
@@ -1,13 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ace-overseer
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.30.1
4
+ version: 0.30.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Michal Czyz
8
8
  bindir: exe
9
9
  cert_chain: []
10
- date: 2026-11-11 00:00:00.000000000 Z
10
+ date: 2026-11-12 00:00:00.000000000 Z
11
11
  dependencies:
12
12
  - !ruby/object:Gem::Dependency
13
13
  name: ace-support-cli
@@ -80,7 +80,7 @@ dependencies:
80
80
  version: '0.70'
81
81
  - - ">="
82
82
  - !ruby/object:Gem::Version
83
- version: 0.73.1
83
+ version: 0.73.3
84
84
  type: :runtime
85
85
  prerelease: false
86
86
  version_requirements: !ruby/object:Gem::Requirement
@@ -90,7 +90,7 @@ dependencies:
90
90
  version: '0.70'
91
91
  - - ">="
92
92
  - !ruby/object:Gem::Version
93
- version: 0.73.1
93
+ version: 0.73.3
94
94
  - !ruby/object:Gem::Dependency
95
95
  name: ace-git
96
96
  requirement: !ruby/object:Gem::Requirement
@@ -160,7 +160,7 @@ dependencies:
160
160
  version: '0.8'
161
161
  - - ">="
162
162
  - !ruby/object:Gem::Version
163
- version: 0.9.0
163
+ version: 0.9.1
164
164
  type: :runtime
165
165
  prerelease: false
166
166
  version_requirements: !ruby/object:Gem::Requirement
@@ -170,7 +170,7 @@ dependencies:
170
170
  version: '0.8'
171
171
  - - ">="
172
172
  - !ruby/object:Gem::Version
173
- version: 0.9.0
173
+ version: 0.9.1
174
174
  - !ruby/object:Gem::Dependency
175
175
  name: ace-hitl
176
176
  requirement: !ruby/object:Gem::Requirement
@@ -180,7 +180,7 @@ dependencies:
180
180
  version: '0.24'
181
181
  - - ">="
182
182
  - !ruby/object:Gem::Version
183
- version: 0.25.1
183
+ version: 0.26.0
184
184
  type: :runtime
185
185
  prerelease: false
186
186
  version_requirements: !ruby/object:Gem::Requirement
@@ -190,7 +190,7 @@ dependencies:
190
190
  version: '0.24'
191
191
  - - ">="
192
192
  - !ruby/object:Gem::Version
193
- version: 0.25.1
193
+ version: 0.26.0
194
194
  - !ruby/object:Gem::Dependency
195
195
  name: ace-runtime
196
196
  requirement: !ruby/object:Gem::Requirement
@@ -234,7 +234,7 @@ dependencies:
234
234
  version: '0.9'
235
235
  - - ">="
236
236
  - !ruby/object:Gem::Version
237
- version: 0.11.0
237
+ version: 0.11.1
238
238
  type: :runtime
239
239
  prerelease: false
240
240
  version_requirements: !ruby/object:Gem::Requirement
@@ -244,7 +244,7 @@ dependencies:
244
244
  version: '0.9'
245
245
  - - ">="
246
246
  - !ruby/object:Gem::Version
247
- version: 0.11.0
247
+ version: 0.11.1
248
248
  - !ruby/object:Gem::Dependency
249
249
  name: ace-lab
250
250
  requirement: !ruby/object:Gem::Requirement