kairos-chain 3.78.0 → 3.79.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +60 -0
- data/lib/kairos_mcp/version.rb +1 -1
- data/templates/skillsets/agent/bin/agent_step_worker.rb +59 -10
- data/templates/skillsets/agent/lib/agent/step_delegation.rb +82 -6
- data/templates/skillsets/agent/test/test_agent_worker_exit_and_review_parse.rb +241 -0
- data/templates/skillsets/agent/tools/agent_status.rb +6 -0
- data/templates/skillsets/agent/tools/agent_step.rb +92 -9
- data/templates/skillsets/agent/tools/agent_wait.rb +4 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 22b92a50151e63d6f243858700e0a0982dd569ce6df05133362dcda4077d6fd6
|
|
4
|
+
data.tar.gz: d6d5559901257355633ba3a9d26a43af62293757b67f98c011bd795be4dee8f7
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8f1bdd48fc2dd110b151e1bc2a8aff18278ea2bc0d97d801dcf3bdd086462e8d6d97b27fdd0a81b75dac820f84aa0786a076eeef84b8ae44d5012e1e435141c0
|
|
7
|
+
data.tar.gz: 0c401f4330cff3dc4af3bf6a4277fcf68412e93fec79acc6eb50a12f7e19c84c6eec298ba5527938e8fc8e2dc983a192383f7babec0c1741dde0e47aca993985
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,66 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
This project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [3.79.0] - 2026-08-26
|
|
8
|
+
|
|
9
|
+
### Fixed
|
|
10
|
+
|
|
11
|
+
- **Field defect D4 — the delegated worker's watchdog killed healthy slow
|
|
12
|
+
cycles at minute 25, and killed them silently.** 3.77.0's first night of
|
|
13
|
+
field use ran the agent SkillSet on real goals; twice, a worker died with
|
|
14
|
+
nothing written, and the second death gave the mechanism away: spawn 20:31,
|
|
15
|
+
last heartbeat 20:56 — spawn plus exactly `WORKER_SELF_TIMEOUT_SECONDS`
|
|
16
|
+
(1500). The watchdog's design assumption was that only a HUNG gated call
|
|
17
|
+
reaches the bound; a healthy cycle with several LLM subprocess calls
|
|
18
|
+
routinely runs past 25 minutes. The bound is now two-fold: a stall bound
|
|
19
|
+
(default 45 min) measured against real session-dir activity — heartbeats
|
|
20
|
+
and lock files excluded, because the heartbeat thread outlives a hung main
|
|
21
|
+
thread and would blind the detector — and an absolute hard cap (default
|
|
22
|
+
3 h). Both are env-overridable (`KAIROS_WORKER_STALL_SECONDS`,
|
|
23
|
+
`KAIROS_WORKER_TIMEOUT_SECONDS`), and both write a `worker_exit.json`
|
|
24
|
+
record before `exit!(124)`. Every other exit the worker can see coming —
|
|
25
|
+
superseded, trapped signal, setsid failure, bootstrap failure, uncaught
|
|
26
|
+
error, normal completion — now leaves the same one-line record, and a
|
|
27
|
+
`crashed` report from `agent_wait` / `agent_status` carries the record (or
|
|
28
|
+
says explicitly that the process vanished without one, which is what a
|
|
29
|
+
SIGKILL still looks like). The record is deliberately NOT a result file:
|
|
30
|
+
the collector's crash path stays the crash path, so a committed advance is
|
|
31
|
+
still recovered from the gate log and never masked.
|
|
32
|
+
|
|
33
|
+
- **Field defect D5 — a reviewer reply that defeats parsing lost both the
|
|
34
|
+
cycle and the evidence.** The high-complexity plan review extracts JSON
|
|
35
|
+
from the reviewer's reply; a long reply can contain several JSON blocks
|
|
36
|
+
(echoed plan fragments in code fences, code in findings), and taking the
|
|
37
|
+
first parseable one produced a hash with no `overall_verdict` — REVISE by
|
|
38
|
+
fallback, twice, then `review_max_retries` with zero cycles run, while the
|
|
39
|
+
5,819-character reply itself was discarded (snapshots keep metadata only).
|
|
40
|
+
The parser now scans all candidate blocks — whole reply, every fenced
|
|
41
|
+
block, every balanced object around an `"overall_verdict"` occurrence —
|
|
42
|
+
and prefers the one carrying a String verdict; and when parsing still
|
|
43
|
+
falls back, the raw reply is persisted beside the session records
|
|
44
|
+
(`review_raw_<ts>.txt`, 64 KiB cap) with the path named in the review's
|
|
45
|
+
findings.
|
|
46
|
+
|
|
47
|
+
### Added
|
|
48
|
+
|
|
49
|
+
- **Norm (g)** in the agent's standing norms, from field defect D2: what
|
|
50
|
+
`operator_report` receives must already be the finished text. The executor
|
|
51
|
+
feeds no step's output into another step's arguments, so a body written as
|
|
52
|
+
an assembly instruction is delivered verbatim as the report — cycle 1 of
|
|
53
|
+
the first field run did exactly that, and cycle 2 corrected it on its own.
|
|
54
|
+
The norm makes the correction standing.
|
|
55
|
+
|
|
56
|
+
- `test_agent_worker_exit_and_review_parse.rb` — 19 tests, 42 assertions:
|
|
57
|
+
exit-record round-trip and token matching, stall-clock exclusion of
|
|
58
|
+
heartbeats and locks, env-overridable bounds, decoy-JSON regression for
|
|
59
|
+
the review parser, raw-reply persistence, norm (g) presence.
|
|
60
|
+
|
|
61
|
+
Field context: defects D1 (worker bootstrap vs host-profile add-on guards,
|
|
62
|
+
fixed and proven in 3.77.0's field night) through D5 are ledgered in L2
|
|
63
|
+
`agent_3_77_0_field_defects` with the two field runs that surfaced them.
|
|
64
|
+
D3 (read-only status calls queue behind a blocking wait on the stdio
|
|
65
|
+
transport) remains open and recorded.
|
|
66
|
+
|
|
7
67
|
## [3.78.0] - 2026-08-26
|
|
8
68
|
|
|
9
69
|
### Fixed
|
data/lib/kairos_mcp/version.rb
CHANGED
|
@@ -62,6 +62,17 @@ rescue ScriptError, StandardError => e
|
|
|
62
62
|
write_raw_result(session_dir, boot_identity,
|
|
63
63
|
{ 'status' => 'error',
|
|
64
64
|
'error' => "worker bootstrap failed: #{e.class}: #{e.message}" })
|
|
65
|
+
# StepDelegation itself failed to load, so write the exit record by hand
|
|
66
|
+
# (field defect D4: every exit this worker can see coming leaves one).
|
|
67
|
+
begin
|
|
68
|
+
File.write(File.join(session_dir, 'worker_exit.json'),
|
|
69
|
+
JSON.generate({ 'timestamp' => Time.now.utc.iso8601, 'pid' => Process.pid,
|
|
70
|
+
'exit_class' => 'bootstrap_failure',
|
|
71
|
+
'step_token' => boot_identity['step_token'],
|
|
72
|
+
'error' => "#{e.class}: #{e.message}" }))
|
|
73
|
+
rescue StandardError
|
|
74
|
+
# best effort
|
|
75
|
+
end
|
|
65
76
|
exit 1
|
|
66
77
|
end
|
|
67
78
|
|
|
@@ -81,6 +92,7 @@ rescue StandardError => e
|
|
|
81
92
|
delegation.write_result({ 'status' => 'error',
|
|
82
93
|
'error' => "worker setsid failed: #{e.message}" },
|
|
83
94
|
identity: boot_identity)
|
|
95
|
+
delegation.write_worker_exit(boot_identity, 'setsid_failure', 'error' => e.message)
|
|
84
96
|
exit 125
|
|
85
97
|
end
|
|
86
98
|
|
|
@@ -96,16 +108,36 @@ heartbeat_thread = Thread.new do
|
|
|
96
108
|
end
|
|
97
109
|
end
|
|
98
110
|
|
|
99
|
-
#
|
|
100
|
-
#
|
|
101
|
-
#
|
|
102
|
-
#
|
|
103
|
-
#
|
|
104
|
-
#
|
|
105
|
-
|
|
111
|
+
# Watchdog (field defect D4, 2026-08-26): two bounds instead of one wall
|
|
112
|
+
# clock. A hung gated call must not hold the advance flock forever — but the
|
|
113
|
+
# old fixed 1500 s assumed only a hung call could reach it, and killed
|
|
114
|
+
# healthy LLM-bound cycles at minute 25 with nothing written. Now the worker
|
|
115
|
+
# dies when the session dir has gone SILENT for the stall bound (heartbeats
|
|
116
|
+
# and locks excluded — the heartbeat thread outlives a hung main thread), or
|
|
117
|
+
# when the hard cap elapses, whichever comes first. Either firing writes a
|
|
118
|
+
# worker_exit record BEFORE exit!(124). That record is not a result, so the
|
|
119
|
+
# collector still takes the crash path and a committed advance is still
|
|
120
|
+
# recovered from the gate log, never masked.
|
|
121
|
+
worker_started = Time.now
|
|
122
|
+
stall_s = KairosMcp::SkillSets::Agent::StepDelegation.worker_stall_seconds
|
|
123
|
+
cap_s = KairosMcp::SkillSets::Agent::StepDelegation.worker_hard_cap_seconds
|
|
106
124
|
watchdog = Thread.new do
|
|
107
|
-
|
|
108
|
-
|
|
125
|
+
loop do
|
|
126
|
+
sleep 30
|
|
127
|
+
elapsed = (Time.now - worker_started).round
|
|
128
|
+
silent = (Time.now - delegation.last_activity_time).round
|
|
129
|
+
if elapsed > cap_s
|
|
130
|
+
delegation.write_worker_exit(boot_identity, 'self_timeout_hard_cap',
|
|
131
|
+
'elapsed_seconds' => elapsed,
|
|
132
|
+
'silent_seconds' => silent)
|
|
133
|
+
exit!(124)
|
|
134
|
+
elsif silent > stall_s
|
|
135
|
+
delegation.write_worker_exit(boot_identity, 'self_timeout_stalled',
|
|
136
|
+
'elapsed_seconds' => elapsed,
|
|
137
|
+
'silent_seconds' => silent)
|
|
138
|
+
exit!(124)
|
|
139
|
+
end
|
|
140
|
+
end
|
|
109
141
|
end
|
|
110
142
|
|
|
111
143
|
begin
|
|
@@ -115,6 +147,12 @@ begin
|
|
|
115
147
|
# KAIROS_DATA_DIR was set in the worker env by spawn_worker so ToolRegistry
|
|
116
148
|
# / Session resolve the server's effective .kairos even under --data-dir.
|
|
117
149
|
require 'kairos_mcp/tool_registry'
|
|
150
|
+
# Host-profile add-on SkillSets (codex_projection, opencode_projection) guard
|
|
151
|
+
# their lib entry point with `raise unless defined?(PluginProjector::HostProfile)`.
|
|
152
|
+
# The server satisfies that guard as a side effect of protocol.rb; this worker
|
|
153
|
+
# bootstraps only tool_registry, so it must load the core explicitly or
|
|
154
|
+
# Skillset#load! trips the guard and the whole registry bootstrap dies.
|
|
155
|
+
require 'kairos_mcp/plugin_projector'
|
|
118
156
|
|
|
119
157
|
pending = delegation.pending
|
|
120
158
|
raise "no pending delegation in #{session_dir}" unless pending
|
|
@@ -126,6 +164,8 @@ begin
|
|
|
126
164
|
# worker owns this delegation, and the gate would replay/serialize our stale
|
|
127
165
|
# call anyway. Exiting leaves the current handle to its rightful worker.
|
|
128
166
|
if my_token.nil? || pending['step_token'] != my_token
|
|
167
|
+
delegation.write_worker_exit(boot_identity, 'superseded',
|
|
168
|
+
'current_token' => pending['step_token'])
|
|
129
169
|
exit 0
|
|
130
170
|
end
|
|
131
171
|
|
|
@@ -136,7 +176,10 @@ begin
|
|
|
136
176
|
args = (pending['arguments'] || {}).merge('session_id' => session_id)
|
|
137
177
|
args.delete('execution') # never recurse into another delegation
|
|
138
178
|
|
|
139
|
-
|
|
179
|
+
if shutdown[:requested]
|
|
180
|
+
delegation.write_worker_exit(boot_identity, 'trapped_signal')
|
|
181
|
+
exit 130
|
|
182
|
+
end
|
|
140
183
|
|
|
141
184
|
registry = KairosMcp::ToolRegistry.new
|
|
142
185
|
raw = registry.call_tool('agent_step', args)
|
|
@@ -158,6 +201,7 @@ begin
|
|
|
158
201
|
# races a concurrently-opened fresh delegation by clearing state it may no
|
|
159
202
|
# longer own or by mislabeling its result as a newer delegation's.
|
|
160
203
|
delegation.write_result(response, identity: identity)
|
|
204
|
+
delegation.write_worker_exit(identity, 'normal')
|
|
161
205
|
exit 0
|
|
162
206
|
rescue SystemExit, SignalException
|
|
163
207
|
# A deliberate exit (including our own `exit 0`) or a signal is not a
|
|
@@ -176,6 +220,11 @@ rescue Exception => e # rubocop:disable Lint/RescueException
|
|
|
176
220
|
rescue StandardError
|
|
177
221
|
# best effort
|
|
178
222
|
end
|
|
223
|
+
begin
|
|
224
|
+
delegation.write_worker_exit(boot_identity, 'uncaught', 'error' => "#{e.class}: #{e.message}")
|
|
225
|
+
rescue StandardError
|
|
226
|
+
# best effort
|
|
227
|
+
end
|
|
179
228
|
exit 1
|
|
180
229
|
ensure
|
|
181
230
|
watchdog&.kill
|
|
@@ -44,14 +44,23 @@ module KairosMcp
|
|
|
44
44
|
HEARTBEAT_FILE = 'delegation.heartbeat'
|
|
45
45
|
RESULT_FILE = 'delegation_result.json'
|
|
46
46
|
LOG_FILE = 'delegation_worker.log'
|
|
47
|
+
WORKER_EXIT_FILE = 'worker_exit.json'
|
|
47
48
|
|
|
48
49
|
HEARTBEAT_INTERVAL_SECONDS = 2
|
|
49
50
|
HEARTBEAT_STALE_THRESHOLD_SECONDS = 15
|
|
50
51
|
STARTUP_GRACE_SECONDS = 30
|
|
51
|
-
#
|
|
52
|
-
#
|
|
53
|
-
#
|
|
54
|
-
|
|
52
|
+
# Two-bound watchdog for the delegated worker (field defect D4,
|
|
53
|
+
# 2026-08-26). The original single 1500 s wall clock assumed only a
|
|
54
|
+
# hung call could reach it; a healthy cycle with several LLM
|
|
55
|
+
# subprocess calls routinely runs past 25 minutes, so the clock killed
|
|
56
|
+
# sound workers — silently, because its exit wrote nothing. The stall
|
|
57
|
+
# bound frees a hung call's flock once the session dir has gone quiet
|
|
58
|
+
# (heartbeats and locks excluded: the heartbeat thread outlives a hung
|
|
59
|
+
# main thread, so it must not count as activity); the hard cap remains
|
|
60
|
+
# the absolute ceiling. Both are env-overridable per run and both
|
|
61
|
+
# record their firing in WORKER_EXIT_FILE before exiting.
|
|
62
|
+
WORKER_STALL_SECONDS_DEFAULT = 2700
|
|
63
|
+
WORKER_HARD_CAP_SECONDS_DEFAULT = 10800
|
|
55
64
|
|
|
56
65
|
WORKER_SCRIPT = File.expand_path('../../bin/agent_step_worker.rb', __dir__)
|
|
57
66
|
|
|
@@ -195,6 +204,68 @@ module KairosMcp
|
|
|
195
204
|
atomic_write(result_path, JSON.pretty_generate(payload))
|
|
196
205
|
end
|
|
197
206
|
|
|
207
|
+
# ---- worker exit record (field defect D4) ----
|
|
208
|
+
|
|
209
|
+
# One line, written by the worker at every exit it can see coming
|
|
210
|
+
# (self-timeout, supersession, trapped signal, setsid/bootstrap
|
|
211
|
+
# failure, uncaught error, normal completion). Deliberately NOT the
|
|
212
|
+
# result file: the collector's crash path must stay the crash path, so
|
|
213
|
+
# a committed advance is still recovered from the gate log and never
|
|
214
|
+
# masked by an error result. Overwrites are fine — readers match on
|
|
215
|
+
# step_token, so a stale record for an older worker is ignored.
|
|
216
|
+
def write_worker_exit(identity, exit_class, detail = {})
|
|
217
|
+
payload = {
|
|
218
|
+
'timestamp' => Time.now.utc.iso8601,
|
|
219
|
+
'pid' => Process.pid,
|
|
220
|
+
'exit_class' => exit_class.to_s,
|
|
221
|
+
'step_token' => identity && identity['step_token'],
|
|
222
|
+
'issue_anchor' => identity && identity['issue_anchor']
|
|
223
|
+
}.merge(detail)
|
|
224
|
+
atomic_write(worker_exit_path, JSON.generate(payload))
|
|
225
|
+
payload
|
|
226
|
+
rescue StandardError
|
|
227
|
+
nil
|
|
228
|
+
end
|
|
229
|
+
|
|
230
|
+
def worker_exit
|
|
231
|
+
JSON.parse(File.read(worker_exit_path))
|
|
232
|
+
rescue Errno::ENOENT, JSON::ParserError
|
|
233
|
+
nil
|
|
234
|
+
end
|
|
235
|
+
|
|
236
|
+
# The exit record for THIS token, or a positive "no record" marker —
|
|
237
|
+
# so a 'crashed' report can say WHY (e.g. self_timeout_stalled after
|
|
238
|
+
# N s of silence) or say explicitly that the process vanished without
|
|
239
|
+
# writing anything (SIGKILL and kin, or a pre-3.78 worker).
|
|
240
|
+
def crash_detail(token)
|
|
241
|
+
rec = worker_exit
|
|
242
|
+
if rec && rec['step_token'] == token
|
|
243
|
+
rec
|
|
244
|
+
else
|
|
245
|
+
{ 'exit_class' => 'no_record',
|
|
246
|
+
'note' => 'process gone without an exit record (killed, or a pre-3.78 worker)' }
|
|
247
|
+
end
|
|
248
|
+
end
|
|
249
|
+
|
|
250
|
+
# Most recent write anywhere in the session dir EXCLUDING liveness and
|
|
251
|
+
# locking artifacts: heartbeats tick every 2 s even when the main
|
|
252
|
+
# thread hangs, so counting them would blind the stall bound.
|
|
253
|
+
def last_activity_time
|
|
254
|
+
newest = nil
|
|
255
|
+
Dir.glob(File.join(@dir, '*')).each do |f|
|
|
256
|
+
base = File.basename(f)
|
|
257
|
+
next if base.start_with?(HEARTBEAT_FILE)
|
|
258
|
+
next if base == LOCK_FILE || base.end_with?('.lock')
|
|
259
|
+
t = begin
|
|
260
|
+
File.mtime(f)
|
|
261
|
+
rescue StandardError
|
|
262
|
+
next
|
|
263
|
+
end
|
|
264
|
+
newest = t if newest.nil? || t > newest
|
|
265
|
+
end
|
|
266
|
+
newest || Time.now
|
|
267
|
+
end
|
|
268
|
+
|
|
198
269
|
# Collect-once, atomically and self-contained: under the lock, consume
|
|
199
270
|
# the result ONLY if it belongs to the CURRENT pending handle (same
|
|
200
271
|
# issue_anchor + action_key + step_token), clearing both the result and
|
|
@@ -274,8 +345,12 @@ module KairosMcp
|
|
|
274
345
|
pid
|
|
275
346
|
end
|
|
276
347
|
|
|
277
|
-
def self.
|
|
278
|
-
|
|
348
|
+
def self.worker_stall_seconds
|
|
349
|
+
(ENV['KAIROS_WORKER_STALL_SECONDS'] || WORKER_STALL_SECONDS_DEFAULT).to_i
|
|
350
|
+
end
|
|
351
|
+
|
|
352
|
+
def self.worker_hard_cap_seconds
|
|
353
|
+
(ENV['KAIROS_WORKER_TIMEOUT_SECONDS'] || WORKER_HARD_CAP_SECONDS_DEFAULT).to_i
|
|
279
354
|
end
|
|
280
355
|
|
|
281
356
|
private
|
|
@@ -338,6 +413,7 @@ module KairosMcp
|
|
|
338
413
|
def pending_path = File.join(@dir, PENDING_FILE)
|
|
339
414
|
def heartbeat_path(token) = File.join(@dir, "#{HEARTBEAT_FILE}.#{token}")
|
|
340
415
|
def result_path = File.join(@dir, RESULT_FILE)
|
|
416
|
+
def worker_exit_path = File.join(@dir, WORKER_EXIT_FILE)
|
|
341
417
|
end
|
|
342
418
|
end
|
|
343
419
|
end
|
|
@@ -0,0 +1,241 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# Field defects D4 and D5 (2026-08-26, ledger: L2 agent_3_77_0_field_defects).
|
|
5
|
+
#
|
|
6
|
+
# D4 — the delegated worker's watchdog killed healthy slow cycles at a fixed
|
|
7
|
+
# 1500 s and exited with exit!(124) writing nothing. Covered here:
|
|
8
|
+
# the two-bound configuration (stall / hard cap, env-overridable), the
|
|
9
|
+
# worker exit record round-trip, crash_detail's token matching and its
|
|
10
|
+
# positive "no record" marker, and last_activity_time's exclusion of
|
|
11
|
+
# heartbeat and lock files (counting heartbeats would blind the stall
|
|
12
|
+
# bound: the heartbeat thread outlives a hung main thread).
|
|
13
|
+
# D5 — a reviewer reply carrying several JSON blocks defeated the parser
|
|
14
|
+
# (first parseable block had no overall_verdict). Covered here: the
|
|
15
|
+
# candidate scan prefers the block that carries a String verdict, the
|
|
16
|
+
# fallback reasons stay honest, and an unparseable reply is persisted
|
|
17
|
+
# raw beside the session records.
|
|
18
|
+
#
|
|
19
|
+
# Reads the SHIPPED template copies, not the instance copies under .kairos.
|
|
20
|
+
# Usage: ruby test_agent_worker_exit_and_review_parse.rb
|
|
21
|
+
|
|
22
|
+
require 'minitest/autorun'
|
|
23
|
+
require 'json'
|
|
24
|
+
require 'fileutils'
|
|
25
|
+
require 'tmpdir'
|
|
26
|
+
require 'rbconfig'
|
|
27
|
+
|
|
28
|
+
$test_kairos_dir = Dir.mktmpdir('agent_worker_exit_test')
|
|
29
|
+
|
|
30
|
+
module Autoexec
|
|
31
|
+
def self.loaded?
|
|
32
|
+
true
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
module KairosMcp
|
|
37
|
+
def self.data_dir
|
|
38
|
+
$test_kairos_dir
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
module Tools
|
|
42
|
+
class BaseTool
|
|
43
|
+
def text_content(text)
|
|
44
|
+
text
|
|
45
|
+
end
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
require File.expand_path('../autonomos/lib/autonomos', File.dirname(__dir__))
|
|
51
|
+
require_relative '../lib/agent/mandate_adapter'
|
|
52
|
+
|
|
53
|
+
$LOAD_PATH.unshift File.expand_path('../../../lib', __dir__)
|
|
54
|
+
$LOAD_PATH.unshift File.expand_path('../lib', __dir__)
|
|
55
|
+
$LOAD_PATH.unshift File.expand_path('../../autoexec/lib', __dir__)
|
|
56
|
+
require 'kairos_mcp/invocation_context'
|
|
57
|
+
require 'kairos_mcp/tools/base_tool'
|
|
58
|
+
require 'kairos_mcp/tool_registry'
|
|
59
|
+
require_relative '../lib/agent'
|
|
60
|
+
require_relative '../tools/agent_step'
|
|
61
|
+
require_relative '../lib/agent/step_delegation'
|
|
62
|
+
|
|
63
|
+
StepDelegation = KairosMcp::SkillSets::Agent::StepDelegation
|
|
64
|
+
AgentStepTool = KairosMcp::SkillSets::Agent::Tools::AgentStep
|
|
65
|
+
|
|
66
|
+
FakeSession = Struct.new(:session_dir)
|
|
67
|
+
|
|
68
|
+
class TestWorkerExitRecord < Minitest::Test
|
|
69
|
+
def setup
|
|
70
|
+
@dir = Dir.mktmpdir('wx_')
|
|
71
|
+
@delegation = StepDelegation.new(@dir)
|
|
72
|
+
@identity = { 'step_token' => 'tok-1', 'issue_anchor' => '1:checkpoint:1' }
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
def teardown
|
|
76
|
+
FileUtils.remove_entry(@dir)
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def test_write_and_read_round_trip
|
|
80
|
+
@delegation.write_worker_exit(@identity, 'self_timeout_stalled',
|
|
81
|
+
'elapsed_seconds' => 100, 'silent_seconds' => 90)
|
|
82
|
+
rec = @delegation.worker_exit
|
|
83
|
+
assert_equal 'self_timeout_stalled', rec['exit_class']
|
|
84
|
+
assert_equal 'tok-1', rec['step_token']
|
|
85
|
+
assert_equal 90, rec['silent_seconds']
|
|
86
|
+
assert_equal Process.pid, rec['pid']
|
|
87
|
+
refute_nil rec['timestamp']
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def test_crash_detail_matches_token
|
|
91
|
+
@delegation.write_worker_exit(@identity, 'superseded')
|
|
92
|
+
assert_equal 'superseded', @delegation.crash_detail('tok-1')['exit_class']
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def test_crash_detail_ignores_other_tokens_record
|
|
96
|
+
@delegation.write_worker_exit(@identity, 'normal')
|
|
97
|
+
detail = @delegation.crash_detail('tok-2')
|
|
98
|
+
assert_equal 'no_record', detail['exit_class']
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
def test_crash_detail_without_any_record_is_positive_no_record
|
|
102
|
+
detail = @delegation.crash_detail('tok-1')
|
|
103
|
+
assert_equal 'no_record', detail['exit_class']
|
|
104
|
+
assert_match(/without an exit record/, detail['note'])
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
def test_worker_exit_survives_nil_identity
|
|
108
|
+
rec = @delegation.write_worker_exit(nil, 'bootstrap_failure')
|
|
109
|
+
assert_equal 'bootstrap_failure', rec['exit_class']
|
|
110
|
+
assert_nil rec['step_token']
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
class TestStallActivityClock < Minitest::Test
|
|
115
|
+
def setup
|
|
116
|
+
@dir = Dir.mktmpdir('act_')
|
|
117
|
+
@delegation = StepDelegation.new(@dir)
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
def teardown
|
|
121
|
+
FileUtils.remove_entry(@dir)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def test_heartbeats_and_locks_do_not_count_as_activity
|
|
125
|
+
old = Time.now - 300
|
|
126
|
+
content = File.join(@dir, 'progress.jsonl')
|
|
127
|
+
File.write(content, 'x')
|
|
128
|
+
File.utime(old, old, content)
|
|
129
|
+
# Fresh liveness/locking artifacts that must NOT rejuvenate the clock.
|
|
130
|
+
File.write(File.join(@dir, 'delegation.heartbeat.tok-1'), '')
|
|
131
|
+
File.write(File.join(@dir, 'delegation.lock'), '')
|
|
132
|
+
File.write(File.join(@dir, 'advance.lock'), '')
|
|
133
|
+
|
|
134
|
+
assert_in_delta old.to_f, @delegation.last_activity_time.to_f, 5.0
|
|
135
|
+
end
|
|
136
|
+
|
|
137
|
+
def test_real_writes_do_count
|
|
138
|
+
File.write(File.join(@dir, 'llm_snapshots.jsonl'), 'x')
|
|
139
|
+
assert_in_delta Time.now.to_f, @delegation.last_activity_time.to_f, 5.0
|
|
140
|
+
end
|
|
141
|
+
|
|
142
|
+
def test_empty_dir_falls_back_to_now
|
|
143
|
+
assert_in_delta Time.now.to_f, @delegation.last_activity_time.to_f, 5.0
|
|
144
|
+
end
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
class TestWatchdogBounds < Minitest::Test
|
|
148
|
+
def test_defaults
|
|
149
|
+
assert_equal 2700, StepDelegation.worker_stall_seconds
|
|
150
|
+
assert_equal 10_800, StepDelegation.worker_hard_cap_seconds
|
|
151
|
+
end
|
|
152
|
+
|
|
153
|
+
def test_env_overrides
|
|
154
|
+
ENV['KAIROS_WORKER_STALL_SECONDS'] = '60'
|
|
155
|
+
ENV['KAIROS_WORKER_TIMEOUT_SECONDS'] = '120'
|
|
156
|
+
assert_equal 60, StepDelegation.worker_stall_seconds
|
|
157
|
+
assert_equal 120, StepDelegation.worker_hard_cap_seconds
|
|
158
|
+
ensure
|
|
159
|
+
ENV.delete('KAIROS_WORKER_STALL_SECONDS')
|
|
160
|
+
ENV.delete('KAIROS_WORKER_TIMEOUT_SECONDS')
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
def test_worker_script_parses
|
|
164
|
+
script = File.expand_path('../bin/agent_step_worker.rb', __dir__)
|
|
165
|
+
assert system(RbConfig.ruby, '-c', script, out: File::NULL, err: File::NULL),
|
|
166
|
+
'agent_step_worker.rb must stay syntactically valid'
|
|
167
|
+
end
|
|
168
|
+
end
|
|
169
|
+
|
|
170
|
+
class TestReviewParseHardening < Minitest::Test
|
|
171
|
+
def setup
|
|
172
|
+
@tool = AgentStepTool.allocate
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
def parse(content)
|
|
176
|
+
@tool.send(:parse_persona_review, content)
|
|
177
|
+
end
|
|
178
|
+
|
|
179
|
+
def test_clean_json_still_parses
|
|
180
|
+
r = parse('{"overall_verdict": "approve", "key_findings": []}')
|
|
181
|
+
assert_equal 'APPROVE', r[:overall_verdict]
|
|
182
|
+
refute r[:parse_error]
|
|
183
|
+
end
|
|
184
|
+
|
|
185
|
+
def test_verdict_found_past_a_decoy_json_block
|
|
186
|
+
# D5 regression shape: the first parseable JSON block (echoed plan
|
|
187
|
+
# fragment in a code fence) has no overall_verdict; the real verdict
|
|
188
|
+
# object comes later in prose.
|
|
189
|
+
content = <<~REPLY
|
|
190
|
+
Looking at the plan first:
|
|
191
|
+
```json
|
|
192
|
+
{"task_id": "t-99", "steps": [{"tool_name": "safe_file_read"}]}
|
|
193
|
+
```
|
|
194
|
+
My assessment as a panel follows.
|
|
195
|
+
{"overall_verdict": "revise", "key_findings": ["step 3 unclear"]}
|
|
196
|
+
REPLY
|
|
197
|
+
r = parse(content)
|
|
198
|
+
assert_equal 'REVISE', r[:overall_verdict]
|
|
199
|
+
assert_equal ['step 3 unclear'], r[:key_findings]
|
|
200
|
+
refute r[:parse_error]
|
|
201
|
+
end
|
|
202
|
+
|
|
203
|
+
def test_hash_without_verdict_still_falls_back_honestly
|
|
204
|
+
r = parse('{"verdict": "APPROVE"}')
|
|
205
|
+
assert r[:parse_error]
|
|
206
|
+
assert_match(/invalid overall_verdict type: NilClass/, r[:key_findings].first)
|
|
207
|
+
assert_equal 'REVISE', r[:overall_verdict]
|
|
208
|
+
end
|
|
209
|
+
|
|
210
|
+
def test_no_json_at_all_falls_back
|
|
211
|
+
r = parse('The plan looks fine to me, ship it.')
|
|
212
|
+
assert r[:parse_error]
|
|
213
|
+
assert_match(/no JSON found/, r[:key_findings].first)
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
def test_fenced_verdict_object_parses
|
|
217
|
+
content = "verdict below\n```json\n{\"overall_verdict\": \"REJECT\"}\n```\n"
|
|
218
|
+
assert_equal 'REJECT', parse(content)[:overall_verdict]
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
def test_persist_raw_review_writes_capped_file
|
|
222
|
+
Dir.mktmpdir('raw_') do |dir|
|
|
223
|
+
session = FakeSession.new(dir)
|
|
224
|
+
path = @tool.send(:persist_raw_review, session, 'a' * 70_000)
|
|
225
|
+
refute_nil path
|
|
226
|
+
assert File.exist?(path)
|
|
227
|
+
assert_equal 65_536, File.size(path)
|
|
228
|
+
assert File.basename(path).start_with?('review_raw_')
|
|
229
|
+
end
|
|
230
|
+
end
|
|
231
|
+
|
|
232
|
+
def test_persist_raw_review_nil_content_is_nil
|
|
233
|
+
assert_nil @tool.send(:persist_raw_review, FakeSession.new('/tmp'), nil)
|
|
234
|
+
end
|
|
235
|
+
|
|
236
|
+
def test_norm_g_present
|
|
237
|
+
norms = @tool.send(:permission_norms)
|
|
238
|
+
assert_match(/\(g\)/, norms)
|
|
239
|
+
assert_match(/finished text/, norms)
|
|
240
|
+
end
|
|
241
|
+
end
|
|
@@ -71,6 +71,12 @@ module KairosMcp
|
|
|
71
71
|
'status' => dstatus,
|
|
72
72
|
'step_token' => delegation.pending&.dig('step_token')
|
|
73
73
|
}.compact
|
|
74
|
+
if dstatus == 'crashed'
|
|
75
|
+
# Field defect D4: surface the worker's own exit record so
|
|
76
|
+
# the operator sees WHY, not just 'crashed'.
|
|
77
|
+
payload['delegation']['worker_exit'] =
|
|
78
|
+
delegation.crash_detail(delegation.pending&.dig('step_token'))
|
|
79
|
+
end
|
|
74
80
|
if %w[ready still_pending].include?(dstatus)
|
|
75
81
|
payload['next_move'] = {
|
|
76
82
|
'tool' => 'agent_wait',
|
|
@@ -1843,10 +1843,29 @@ module KairosMcp
|
|
|
1843
1843
|
result = review_loop.run_phase('review', persona_review_system_prompt, messages, [])
|
|
1844
1844
|
|
|
1845
1845
|
parsed = parse_persona_review(result['content'])
|
|
1846
|
+
if parsed[:parse_error]
|
|
1847
|
+
# Field defect D5: without the raw reply on disk a parse failure
|
|
1848
|
+
# is undiagnosable — llm_snapshots keeps only metadata.
|
|
1849
|
+
raw_path = persist_raw_review(session, result['content'])
|
|
1850
|
+
parsed[:key_findings] = Array(parsed[:key_findings]) +
|
|
1851
|
+
["raw reviewer reply saved: #{raw_path}"] if raw_path
|
|
1852
|
+
end
|
|
1846
1853
|
parsed[:llm_calls] = review_loop.total_calls
|
|
1847
1854
|
parsed
|
|
1848
1855
|
end
|
|
1849
1856
|
|
|
1857
|
+
# Persist an unparseable reviewer reply beside the session records
|
|
1858
|
+
# (capped at 64 KiB). Returns the path, or nil when nothing usable.
|
|
1859
|
+
def persist_raw_review(session, content)
|
|
1860
|
+
return nil if content.nil? || content.to_s.empty?
|
|
1861
|
+
name = "review_raw_#{Time.now.utc.strftime('%Y%m%dT%H%M%SZ')}.txt"
|
|
1862
|
+
path = File.join(session.session_dir, name)
|
|
1863
|
+
File.write(path, content.to_s[0, 65_536])
|
|
1864
|
+
path
|
|
1865
|
+
rescue StandardError
|
|
1866
|
+
nil
|
|
1867
|
+
end
|
|
1868
|
+
|
|
1850
1869
|
def run_lightweight_review(session, decision_payload, ar_result)
|
|
1851
1870
|
prompt = build_lightweight_review_prompt(decision_payload, ar_result)
|
|
1852
1871
|
review_loop = CognitiveLoop.new(self, session)
|
|
@@ -2062,18 +2081,76 @@ module KairosMcp
|
|
|
2062
2081
|
def parse_persona_review(content)
|
|
2063
2082
|
return review_parse_fallback('no content') unless content
|
|
2064
2083
|
|
|
2065
|
-
|
|
2066
|
-
|
|
2084
|
+
# Field defect D5 (2026-08-26): a long reviewer reply can contain
|
|
2085
|
+
# SEVERAL JSON blocks — echoed plan fragments, code in findings —
|
|
2086
|
+
# and taking the first parseable one yielded a hash with no
|
|
2087
|
+
# overall_verdict at all. Scan every candidate and prefer the one
|
|
2088
|
+
# that actually carries a String overall_verdict; fall back to the
|
|
2089
|
+
# first hash only when none does, so the failure reason stays
|
|
2090
|
+
# honest ("invalid overall_verdict type").
|
|
2091
|
+
candidates = extract_json_candidates(content)
|
|
2092
|
+
return review_parse_fallback('no JSON found') if candidates.empty?
|
|
2093
|
+
|
|
2094
|
+
chosen = nil
|
|
2095
|
+
candidates.each do |json_str|
|
|
2096
|
+
parsed = begin
|
|
2097
|
+
JSON.parse(json_str)
|
|
2098
|
+
rescue JSON::ParserError
|
|
2099
|
+
next
|
|
2100
|
+
end
|
|
2101
|
+
next unless parsed.is_a?(Hash)
|
|
2102
|
+
if parsed['overall_verdict'].is_a?(String)
|
|
2103
|
+
chosen = parsed
|
|
2104
|
+
break
|
|
2105
|
+
end
|
|
2106
|
+
chosen ||= parsed
|
|
2107
|
+
end
|
|
2108
|
+
return review_parse_fallback('no JSON found') if chosen.nil?
|
|
2067
2109
|
|
|
2068
|
-
|
|
2069
|
-
verdict = parsed['overall_verdict']
|
|
2110
|
+
verdict = chosen['overall_verdict']
|
|
2070
2111
|
unless verdict.is_a?(String)
|
|
2071
2112
|
return review_parse_fallback("invalid overall_verdict type: #{verdict.class}")
|
|
2072
2113
|
end
|
|
2073
|
-
|
|
2074
|
-
|
|
2075
|
-
|
|
2076
|
-
|
|
2114
|
+
chosen['overall_verdict'] = verdict.upcase
|
|
2115
|
+
chosen.transform_keys(&:to_sym)
|
|
2116
|
+
end
|
|
2117
|
+
|
|
2118
|
+
# All plausible JSON blocks in a reviewer reply, in preference order:
|
|
2119
|
+
# the whole reply, every fenced block, every balanced object around
|
|
2120
|
+
# an "overall_verdict" occurrence, then the crude first-{ to last-}
|
|
2121
|
+
# span. Invalid candidates are skipped by the caller's JSON.parse.
|
|
2122
|
+
def extract_json_candidates(content)
|
|
2123
|
+
begin
|
|
2124
|
+
JSON.parse(content)
|
|
2125
|
+
return [content]
|
|
2126
|
+
rescue JSON::ParserError
|
|
2127
|
+
# fall through to scanning
|
|
2128
|
+
end
|
|
2129
|
+
|
|
2130
|
+
candidates = []
|
|
2131
|
+
content.scan(/```(?:json)?\s*\n?(.*?)\n?```/m) { |m| candidates << m[0] }
|
|
2132
|
+
|
|
2133
|
+
search_from = 0
|
|
2134
|
+
while (key_at = content.index('"overall_verdict"', search_from))
|
|
2135
|
+
open_at = content.rindex('{', key_at)
|
|
2136
|
+
if open_at
|
|
2137
|
+
depth = 0
|
|
2138
|
+
i = open_at
|
|
2139
|
+
while i < content.length
|
|
2140
|
+
depth += 1 if content[i] == '{'
|
|
2141
|
+
depth -= 1 if content[i] == '}'
|
|
2142
|
+
if depth.zero?
|
|
2143
|
+
candidates << content[open_at..i]
|
|
2144
|
+
break
|
|
2145
|
+
end
|
|
2146
|
+
i += 1
|
|
2147
|
+
end
|
|
2148
|
+
end
|
|
2149
|
+
search_from = key_at + 1
|
|
2150
|
+
end
|
|
2151
|
+
|
|
2152
|
+
candidates << Regexp.last_match(1) if content =~ /(\{.*\})/m
|
|
2153
|
+
candidates.compact.uniq
|
|
2077
2154
|
end
|
|
2078
2155
|
|
|
2079
2156
|
def review_parse_fallback(reason)
|
|
@@ -2401,7 +2478,13 @@ module KairosMcp
|
|
|
2401
2478
|
"place it so that everything not depending on it comes first. Marked steps are " \
|
|
2402
2479
|
"set aside and the rest of the plan still runs; the operator is shown what was " \
|
|
2403
2480
|
"set aside at the end of the cycle. Do not drop such a step, and do not " \
|
|
2404
|
-
"substitute a reversible half-measure without saying so in the summary
|
|
2481
|
+
"substitute a reversible half-measure without saying so in the summary.\n" \
|
|
2482
|
+
" (g) What `operator_report` receives must already be the finished text. " \
|
|
2483
|
+
"The executor does not feed one step's output into another step's " \
|
|
2484
|
+
"arguments, so a body written as an instruction — assemble this, summarize " \
|
|
2485
|
+
"that — is delivered to the operator verbatim as the report. When the " \
|
|
2486
|
+
"deliverable depends on outputs of steps in this plan, deliver it in the " \
|
|
2487
|
+
"next cycle, written from what those steps actually returned."
|
|
2405
2488
|
end
|
|
2406
2489
|
|
|
2407
2490
|
def reflect_system_prompt
|
|
@@ -174,6 +174,10 @@ module KairosMcp
|
|
|
174
174
|
text_content(JSON.generate({
|
|
175
175
|
'status' => 'crashed', 'session_id' => session.session_id,
|
|
176
176
|
'step_token' => pending&.dig('step_token'),
|
|
177
|
+
# Field defect D4: say WHY when the worker left an exit record
|
|
178
|
+
# (self_timeout_stalled / hard_cap / superseded / uncaught …),
|
|
179
|
+
# or say explicitly that it vanished without one.
|
|
180
|
+
'worker_exit' => delegation.crash_detail(pending&.dig('step_token')),
|
|
177
181
|
'next_action' => status_hint(session.session_id,
|
|
178
182
|
'the delegated worker died before committing; the gate makes a ' \
|
|
179
183
|
're-issue safe (exactly-once) — read agent_status and follow next_move')
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kairos-chain
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.
|
|
4
|
+
version: 3.79.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Masaomi Hatakeyama
|
|
@@ -431,6 +431,7 @@ files:
|
|
|
431
431
|
- templates/skillsets/agent/test/test_agent_m4.rb
|
|
432
432
|
- templates/skillsets/agent/test/test_agent_native_body.rb
|
|
433
433
|
- templates/skillsets/agent/test/test_agent_verdict.rb
|
|
434
|
+
- templates/skillsets/agent/test/test_agent_worker_exit_and_review_parse.rb
|
|
434
435
|
- templates/skillsets/agent/test/test_decide_prompt_contract.rb
|
|
435
436
|
- templates/skillsets/agent/test/test_review_hint.rb
|
|
436
437
|
- templates/skillsets/agent/tools/agent_execute.rb
|