kairos-chain 3.88.0 → 3.88.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c37ee945277e2917d116b02aeb8b47957f97363d8428aae7b54e17a732ca0d94
|
|
4
|
+
data.tar.gz: 62df9a6d5f05afd01c3aae950959d6398a22e979270183a154e4fd211d2b78c8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 74101139a4c243c77119530a5a723c0e1de9fdb6224c1b5f776f78b250040c1e4aade60b93cb75d5cbef3afac2609e2832763a079fc38bddd6ecac84e73f2c32
|
|
7
|
+
data.tar.gz: 0bf5337f8e6b5985967711300dfe64554e7ff55bb67f7f1dff1d3fca1aefa28fa74677477d5ed3c32cc9b843556434ca692a7254d26a74d68a32807408091621
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,45 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
This project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [3.88.1] - 2026-10-06
|
|
8
|
+
|
|
9
|
+
### Fixed — agent guard: the verdict reaches the record
|
|
10
|
+
|
|
11
|
+
Found in a guarded trial on 3.88.0. The guard judged correctly, but its
|
|
12
|
+
judgment never reached the record: the chain held only autoexec's "execution
|
|
13
|
+
complete" for an act the guard had failed, progress said `completed`, and the
|
|
14
|
+
FAIL survived only in a mutable session file.
|
|
15
|
+
|
|
16
|
+
- **The verdict is chain-recorded.** Right after judging, before any halt
|
|
17
|
+
checkpoint and before any merge, the driver records `kind:
|
|
18
|
+
agent_guard_verdict`: the verdict, the pinned spec hash, the failed checks,
|
|
19
|
+
the planned route, and the act's error if it had one.
|
|
20
|
+
- **A verdict the chain does not take stops the loop.** Whatever the verdict,
|
|
21
|
+
nothing merges and the cycle halts for the operator. The halt reason names
|
|
22
|
+
the lost verdict and its reason, why the record failed, and whether the act
|
|
23
|
+
already took effect (in-process) or stays quarantined (confined). Responses
|
|
24
|
+
and the progress `guard_record` carry `verdict_recorded: false` and
|
|
25
|
+
`lost_verdict`. On a backend whose chain cannot be appended to (sqlite and
|
|
26
|
+
postgresql do not declare a ledger path), every guarded cycle now halts
|
|
27
|
+
this way instead of running unrecorded.
|
|
28
|
+
- **Progress and responses follow the verdict.** A failed verdict is recorded
|
|
29
|
+
and reported as `failed`, not `completed`; REFLECT is shown the verdict, and
|
|
30
|
+
the next cycle's executor context says the cycle failed. Manual responses
|
|
31
|
+
carry `guard` (verdict, failed checks) and `act_error`. The manual risk-resume
|
|
32
|
+
path reports a guard halt as `guard_halt`, not `ok`. Autonomous responses
|
|
33
|
+
carry the halt reason and, per cycle, `guard_verdict`, `guard_reason` and
|
|
34
|
+
`verdict_recorded`; steps set aside for a person are still listed when a
|
|
35
|
+
cycle halts. A result carrying only an error is no longer reported as
|
|
36
|
+
`completed`.
|
|
37
|
+
|
|
38
|
+
Not fixed here: the confined route still cannot run. `agent_execute` is not
|
|
39
|
+
registered by the agent SkillSet, so a plan naming a file tool fails at ACT
|
|
40
|
+
(with the guard on, that FAIL is now recorded), and a file acceptance check
|
|
41
|
+
cannot PASS in this version. Wiring it is separate work.
|
|
42
|
+
|
|
43
|
+
Also ships `minimum_nomic`'s `bin/run_gm.rb` instance-local changes (a11d103),
|
|
44
|
+
so `system_upgrade` stops reverting them.
|
|
45
|
+
|
|
7
46
|
## [3.88.0] - 2026-09-30
|
|
8
47
|
|
|
9
48
|
### Added — `multi_llm_review` 0.11.1: an artifact named by path and sha256
|
data/lib/kairos_mcp/version.rb
CHANGED
|
@@ -0,0 +1,448 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# The guard verdict reaches the record, and what is reported follows it.
|
|
5
|
+
#
|
|
6
|
+
# Field observation (guarded trial on another instance, gem 3.88.0, 2026-10-06):
|
|
7
|
+
# the guard FAILed an act that produced no file, but the chain held only
|
|
8
|
+
# autoexec's "internal_execute_complete", progress said "completed", and the
|
|
9
|
+
# FAIL survived only in the mutable advance_log.jsonl.
|
|
10
|
+
#
|
|
11
|
+
# These probes drive the REAL agent_start / agent_step and the REAL
|
|
12
|
+
# chain_record tool through the registry, with the REAL TaskDsl, Verdict and
|
|
13
|
+
# Confinement. Stubbed: llm_call, autoexec_plan/run, knowledge_get, the merge
|
|
14
|
+
# target directory, the storage underneath chain_record (never the real
|
|
15
|
+
# chain), and agent_execute.
|
|
16
|
+
#
|
|
17
|
+
# agent_execute is injected into the registry here. In a real server it is not
|
|
18
|
+
# registered (skillset.json tool_classes does not list it), so the confined
|
|
19
|
+
# route is not reachable yet; wiring it is separate work. The confined probes
|
|
20
|
+
# below test the driver's verdict/record/merge ordering, not that route.
|
|
21
|
+
# Usage: ruby test_agent_guard_verdict_record.rb
|
|
22
|
+
|
|
23
|
+
$LOAD_PATH.unshift File.expand_path('../../lib', __dir__)
|
|
24
|
+
$LOAD_PATH.unshift File.expand_path('../../../../lib', __dir__)
|
|
25
|
+
|
|
26
|
+
require 'json'
|
|
27
|
+
require 'yaml'
|
|
28
|
+
require 'fileutils'
|
|
29
|
+
require 'tmpdir'
|
|
30
|
+
require 'digest'
|
|
31
|
+
require 'time'
|
|
32
|
+
require 'securerandom'
|
|
33
|
+
require 'kairos_mcp/invocation_context'
|
|
34
|
+
require 'kairos_mcp/tools/base_tool'
|
|
35
|
+
require 'kairos_mcp/tools/chain_record'
|
|
36
|
+
require 'kairos_mcp/tool_registry'
|
|
37
|
+
require_relative '../lib/agent'
|
|
38
|
+
require_relative '../tools/agent_start'
|
|
39
|
+
require_relative '../tools/agent_step'
|
|
40
|
+
require File.expand_path('../autoexec/lib/autoexec/task_dsl', File.dirname(__dir__))
|
|
41
|
+
# TaskDsl.validate reads Autoexec.config (normally provided by autoexec.rb).
|
|
42
|
+
module Autoexec
|
|
43
|
+
def self.config = {} unless respond_to?(:config)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
$pass = 0
|
|
47
|
+
$fail = 0
|
|
48
|
+
|
|
49
|
+
def assert(description)
|
|
50
|
+
result = yield
|
|
51
|
+
if result
|
|
52
|
+
$pass += 1
|
|
53
|
+
puts " PASS: #{description}"
|
|
54
|
+
else
|
|
55
|
+
$fail += 1
|
|
56
|
+
puts " FAIL: #{description}"
|
|
57
|
+
end
|
|
58
|
+
rescue StandardError => e
|
|
59
|
+
$fail += 1
|
|
60
|
+
puts " FAIL: #{description} (#{e.class}: #{e.message})"
|
|
61
|
+
puts " #{e.backtrace.first(3).join("\n ")}"
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def section(title)
|
|
65
|
+
puts "\n#{'=' * 60}\nTEST: #{title}\n#{'=' * 60}"
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
TMPDIR = Dir.mktmpdir('agent_guard_record_test')
|
|
69
|
+
PROJECT_ROOT = File.join(TMPDIR, 'project')
|
|
70
|
+
FileUtils.mkdir_p(File.join(PROJECT_ROOT, '.kairos'))
|
|
71
|
+
|
|
72
|
+
module Autonomos
|
|
73
|
+
@storage_base = TMPDIR
|
|
74
|
+
def self.storage_path(subpath)
|
|
75
|
+
path = File.join(@storage_base, subpath)
|
|
76
|
+
FileUtils.mkdir_p(path)
|
|
77
|
+
path
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
def self.config
|
|
81
|
+
{}
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
require File.expand_path('../autonomos/lib/autonomos/mandate', File.dirname(__dir__))
|
|
86
|
+
require File.expand_path('../autonomos/lib/autonomos/ooda', File.dirname(__dir__))
|
|
87
|
+
|
|
88
|
+
Session = KairosMcp::SkillSets::Agent::Session
|
|
89
|
+
Verdict = KairosMcp::SkillSets::Agent::Verdict
|
|
90
|
+
Confinement = KairosMcp::SkillSets::Agent::Confinement
|
|
91
|
+
|
|
92
|
+
# The storage under the real chain_record tool. Chain.new must never reach the
|
|
93
|
+
# real chain: the prepended initialize does not call super.
|
|
94
|
+
module FakeChainStore
|
|
95
|
+
class << self
|
|
96
|
+
attr_accessor :blocks, :raise_with, :on_add
|
|
97
|
+
end
|
|
98
|
+
self.blocks = []
|
|
99
|
+
end
|
|
100
|
+
KairosMcp::KairosChain::Chain.prepend(Module.new do
|
|
101
|
+
def initialize(*); end
|
|
102
|
+
|
|
103
|
+
def add_block(logs)
|
|
104
|
+
raise FakeChainStore.raise_with if FakeChainStore.raise_with
|
|
105
|
+
|
|
106
|
+
FakeChainStore.on_add&.call(logs)
|
|
107
|
+
FakeChainStore.blocks << logs
|
|
108
|
+
Struct.new(:index, :hash).new(200 + FakeChainStore.blocks.size, 'cd' * 32)
|
|
109
|
+
end
|
|
110
|
+
end)
|
|
111
|
+
|
|
112
|
+
# The shipped agent.yml has the guard off; these probes need it on.
|
|
113
|
+
GUARD_ON = Module.new do
|
|
114
|
+
def load_config
|
|
115
|
+
super.merge('guard' => { 'enabled' => true, 'admission' => { 'extra_denied_tools' => [] } })
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
KairosMcp::SkillSets::Agent::Tools::AgentStart.prepend(GUARD_ON)
|
|
119
|
+
|
|
120
|
+
# ---- Mocks: only the external seams the driver calls ----
|
|
121
|
+
|
|
122
|
+
class MockLlmCall < KairosMcp::Tools::BaseTool
|
|
123
|
+
@@responses = []
|
|
124
|
+
@@seen = []
|
|
125
|
+
def self.queue(r) = @@responses << r
|
|
126
|
+
def self.seen = @@seen
|
|
127
|
+
|
|
128
|
+
def self.clear!
|
|
129
|
+
@@responses.clear
|
|
130
|
+
@@seen.clear
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
def name = 'llm_call'
|
|
134
|
+
def description = 'mock'
|
|
135
|
+
def input_schema = { type: 'object', properties: {} }
|
|
136
|
+
|
|
137
|
+
def call(arguments)
|
|
138
|
+
@@seen << JSON.generate(arguments)
|
|
139
|
+
resp = @@responses.shift ||
|
|
140
|
+
{ 'content' => 'default', 'tool_use' => nil, 'stop_reason' => 'end_turn' }
|
|
141
|
+
text_content(JSON.generate({ 'status' => 'ok', 'provider' => 'mock', 'model' => 'mock-1',
|
|
142
|
+
'response' => resp,
|
|
143
|
+
'usage' => { 'input_tokens' => 1, 'output_tokens' => 1 },
|
|
144
|
+
'snapshot' => { 'model' => 'mock-1', 'timestamp' => Time.now.iso8601 } }))
|
|
145
|
+
end
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
class MockAutoexecPlan < KairosMcp::Tools::BaseTool
|
|
149
|
+
def name = 'autoexec_plan'
|
|
150
|
+
def description = 'mock'
|
|
151
|
+
def input_schema = { type: 'object', properties: {} }
|
|
152
|
+
|
|
153
|
+
def call(arguments)
|
|
154
|
+
task_json = JSON.parse(arguments['task_json'])
|
|
155
|
+
text_content(JSON.generate({ 'status' => 'ok', 'task_id' => task_json['task_id'],
|
|
156
|
+
'plan_hash' => Digest::SHA256.hexdigest(arguments['task_json'])[0..15],
|
|
157
|
+
'steps' => task_json['steps']&.length || 0 }))
|
|
158
|
+
end
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# autoexec reports the run complete — exactly what it told the chain in the field.
|
|
162
|
+
class MockAutoexecRun < KairosMcp::Tools::BaseTool
|
|
163
|
+
def name = 'autoexec_run'
|
|
164
|
+
def description = 'mock'
|
|
165
|
+
def input_schema = { type: 'object', properties: {} }
|
|
166
|
+
|
|
167
|
+
def call(_arguments)
|
|
168
|
+
text_content(JSON.generate({ 'task_id' => 'mock_task', 'mode' => 'internal_execute',
|
|
169
|
+
'outcome' => 'internal_execute_complete',
|
|
170
|
+
'steps_processed' => 1, 'steps' => [] }))
|
|
171
|
+
end
|
|
172
|
+
end
|
|
173
|
+
|
|
174
|
+
# Stands in for the confined executor: writes into its own scratch area and
|
|
175
|
+
# returns the driver-visible evidence the real tool returns.
|
|
176
|
+
class MockAgentExecute < KairosMcp::Tools::BaseTool
|
|
177
|
+
@@content = "implementation e9d9ef1\nanalysis fe66272\n"
|
|
178
|
+
def name = 'agent_execute'
|
|
179
|
+
def description = 'mock'
|
|
180
|
+
def input_schema = { type: 'object', properties: {} }
|
|
181
|
+
|
|
182
|
+
def call(_arguments)
|
|
183
|
+
scratch = Dir.mktmpdir('agent_act_', TMPDIR)
|
|
184
|
+
FileUtils.mkdir_p(File.join(scratch, 'out'))
|
|
185
|
+
File.write(File.join(scratch, 'out', 'note.md'), @@content)
|
|
186
|
+
text_content(JSON.generate({ 'status' => 'ok', 'result' => 'done',
|
|
187
|
+
'scratch_dir' => scratch,
|
|
188
|
+
'manifest' => Confinement.manifest(scratch),
|
|
189
|
+
'files_modified' => [], 'tool_calls_count' => 1 }))
|
|
190
|
+
end
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
class MockKnowledgeGet < KairosMcp::Tools::BaseTool
|
|
194
|
+
def name = 'knowledge_get'
|
|
195
|
+
def description = 'mock'
|
|
196
|
+
def input_schema = { type: 'object', properties: {} }
|
|
197
|
+
def call(args) = text_content(JSON.generate({ 'name' => args['name'], 'content' => 'mock' }))
|
|
198
|
+
end
|
|
199
|
+
|
|
200
|
+
def build_registry
|
|
201
|
+
registry = KairosMcp::ToolRegistry.allocate
|
|
202
|
+
registry.instance_variable_set(:@safety, KairosMcp::Safety.new)
|
|
203
|
+
KairosMcp::ToolRegistry.clear_gates!
|
|
204
|
+
registry.instance_variable_set(:@tools, {
|
|
205
|
+
'llm_call' => MockLlmCall.new(nil, registry: registry),
|
|
206
|
+
'knowledge_get' => MockKnowledgeGet.new(nil, registry: registry),
|
|
207
|
+
'autoexec_plan' => MockAutoexecPlan.new(nil, registry: registry),
|
|
208
|
+
'autoexec_run' => MockAutoexecRun.new(nil, registry: registry),
|
|
209
|
+
'agent_execute' => MockAgentExecute.new(nil, registry: registry),
|
|
210
|
+
'chain_record' => KairosMcp::Tools::ChainRecord.new(nil, registry: registry),
|
|
211
|
+
'agent_start' => KairosMcp::SkillSets::Agent::Tools::AgentStart.new(nil, registry: registry),
|
|
212
|
+
'agent_step' => KairosMcp::SkillSets::Agent::Tools::AgentStep.new(nil, registry: registry)
|
|
213
|
+
})
|
|
214
|
+
registry
|
|
215
|
+
end
|
|
216
|
+
|
|
217
|
+
REGISTRY = build_registry
|
|
218
|
+
START_TOOL = REGISTRY.instance_variable_get(:@tools)['agent_start']
|
|
219
|
+
STEP_TOOL = REGISTRY.instance_variable_get(:@tools)['agent_step']
|
|
220
|
+
# A PASS merges into the live tree; point that at a scratch project, never the repo.
|
|
221
|
+
STEP_TOOL.define_singleton_method(:project_root_for_merge) { PROJECT_ROOT }
|
|
222
|
+
MERGED = File.join(PROJECT_ROOT, 'out', 'note.md')
|
|
223
|
+
|
|
224
|
+
ACCEPTANCE = {
|
|
225
|
+
'acceptance' => [
|
|
226
|
+
{ 'type' => 'file_exists', 'path' => 'out/note.md' },
|
|
227
|
+
{ 'type' => 'file_contains', 'path' => 'out/note.md', 'substring' => 'e9d9ef1' },
|
|
228
|
+
{ 'type' => 'file_contains', 'path' => 'out/note.md', 'substring' => 'fe66272' },
|
|
229
|
+
{ 'type' => 'manifest_not_empty' },
|
|
230
|
+
{ 'type' => 'execution_completed' }
|
|
231
|
+
],
|
|
232
|
+
'layer_surface' => []
|
|
233
|
+
}.freeze
|
|
234
|
+
|
|
235
|
+
def decision(tool_name, args)
|
|
236
|
+
JSON.generate({
|
|
237
|
+
'summary' => "plan via #{tool_name}",
|
|
238
|
+
'task_json' => {
|
|
239
|
+
'task_id' => "t_#{SecureRandom.hex(3)}", 'meta' => { 'description' => 't', 'risk_default' => 'low' },
|
|
240
|
+
'steps' => [{ 'step_id' => 's1', 'action' => 'produce the note', 'tool_name' => tool_name,
|
|
241
|
+
'tool_arguments' => args, 'risk' => 'low', 'depends_on' => [],
|
|
242
|
+
'requires_human_cognition' => false }]
|
|
243
|
+
},
|
|
244
|
+
'review_hint' => { 'needed' => false, 'reason' => nil, 'urgency' => nil },
|
|
245
|
+
'complexity_hint' => { 'level' => 'low', 'signals' => [] }
|
|
246
|
+
})
|
|
247
|
+
end
|
|
248
|
+
|
|
249
|
+
REPORT_PLAN = decision('operator_report', { 'title' => 'Guard trial', 'body' => 'e9d9ef1 fe66272' })
|
|
250
|
+
WRITE_PLAN = decision('file_write', { 'file_path' => 'out/note.md',
|
|
251
|
+
'content' => "implementation e9d9ef1\nanalysis fe66272\n" })
|
|
252
|
+
|
|
253
|
+
def reset_chain!
|
|
254
|
+
FakeChainStore.blocks = []
|
|
255
|
+
FakeChainStore.raise_with = nil
|
|
256
|
+
FakeChainStore.on_add = nil
|
|
257
|
+
end
|
|
258
|
+
|
|
259
|
+
def start_proposed(plan)
|
|
260
|
+
sid = JSON.parse(START_TOOL.call({ 'goal_name' => "guard_record_#{SecureRandom.hex(3)}",
|
|
261
|
+
'guard' => ACCEPTANCE })[0][:text])['session_id']
|
|
262
|
+
MockLlmCall.clear!
|
|
263
|
+
MockLlmCall.queue({ 'content' => 'orient', 'tool_use' => nil, 'stop_reason' => 'end_turn' })
|
|
264
|
+
MockLlmCall.queue({ 'content' => plan, 'tool_use' => nil, 'stop_reason' => 'end_turn' })
|
|
265
|
+
r = JSON.parse(STEP_TOOL.call({ 'session_id' => sid, 'action' => 'approve' })[0][:text])
|
|
266
|
+
[sid, r]
|
|
267
|
+
end
|
|
268
|
+
|
|
269
|
+
def act(sid)
|
|
270
|
+
MockLlmCall.queue({ 'content' => JSON.generate({ 'confidence' => 0.9, 'achieved' => ['done'],
|
|
271
|
+
'remaining' => [] }),
|
|
272
|
+
'tool_use' => nil, 'stop_reason' => 'end_turn' })
|
|
273
|
+
JSON.parse(STEP_TOOL.call({ 'session_id' => sid, 'action' => 'approve' })[0][:text])
|
|
274
|
+
end
|
|
275
|
+
|
|
276
|
+
def verdict_records
|
|
277
|
+
FakeChainStore.blocks.flatten.map { |l| JSON.parse(l) }.select { |r| r['kind'] == 'agent_guard_verdict' }
|
|
278
|
+
end
|
|
279
|
+
|
|
280
|
+
# ------------------------------------------------------------------
|
|
281
|
+
|
|
282
|
+
section 'Field case reproduced: in-process act, no file, guard FAIL'
|
|
283
|
+
|
|
284
|
+
reset_chain!
|
|
285
|
+
sid5, proposed5 = start_proposed(REPORT_PLAN)
|
|
286
|
+
assert('DECIDE produced a proposal') { proposed5['state'] == 'proposed' }
|
|
287
|
+
r5 = act(sid5)
|
|
288
|
+
pinned_sha = File.read(File.join(Session.load(sid5).guard_dir, Verdict::SHA_FILE)).strip
|
|
289
|
+
rec5 = verdict_records.last
|
|
290
|
+
|
|
291
|
+
assert('the FAIL verdict is written to the chain through the real chain_record') do
|
|
292
|
+
rec5 && rec5['verdict'] == Verdict::FAIL
|
|
293
|
+
end
|
|
294
|
+
assert('the chain record carries the pinned spec hash') { rec5 && rec5['spec_sha256'] == pinned_sha }
|
|
295
|
+
assert('the chain record names the failed checks') do
|
|
296
|
+
rec5 && rec5['failed_checks'].map { |c| c['type'] }.sort ==
|
|
297
|
+
%w[file_contains file_contains file_exists manifest_not_empty]
|
|
298
|
+
end
|
|
299
|
+
assert('the chain record names the planned route as in_process') { rec5 && rec5['planned_route'] == 'in_process' }
|
|
300
|
+
assert("the response reports 'failed', not 'completed'") { r5['act_summary'] == 'failed' }
|
|
301
|
+
assert('the response shows the verdict and its failed checks') do
|
|
302
|
+
r5.dig('guard', 'verdict') == Verdict::FAIL && r5.dig('guard', 'failed_checks').include?('file_exists')
|
|
303
|
+
end
|
|
304
|
+
assert('REFLECT was shown the verdict, not only the executor\'s "completed"') do
|
|
305
|
+
MockLlmCall.seen.last.include?('guard_verdict_summary') && MockLlmCall.seen.last.include?(Verdict::FAIL)
|
|
306
|
+
end
|
|
307
|
+
|
|
308
|
+
prog5 = Session.load(sid5).load_progress.last
|
|
309
|
+
assert("progress records 'failed', not 'completed'") { prog5['act_summary'] == 'failed' }
|
|
310
|
+
assert('the progress guard_record carries the verdict') { prog5.dig('guard_record', 'verdict') == Verdict::FAIL }
|
|
311
|
+
assert('the progress guard_record says the verdict was recorded, and where') do
|
|
312
|
+
prog5.dig('guard_record', 'verdict_recorded') == true &&
|
|
313
|
+
prog5.dig('guard_record', 'verdict_chain_block', 'index').is_a?(Integer)
|
|
314
|
+
end
|
|
315
|
+
assert('the next cycle is told the previous cycle failed') do
|
|
316
|
+
STEP_TOOL.send(:build_agent_execute_context, Session.load(sid5)).include?('Cycle 1: failed')
|
|
317
|
+
end
|
|
318
|
+
|
|
319
|
+
section 'Confined PASS: the verdict is recorded first, then the file is merged'
|
|
320
|
+
|
|
321
|
+
reset_chain!
|
|
322
|
+
FileUtils.rm_rf(File.join(PROJECT_ROOT, 'out'))
|
|
323
|
+
merged_at_record_time = nil
|
|
324
|
+
FakeChainStore.on_add = ->(_logs) { merged_at_record_time = File.exist?(MERGED) }
|
|
325
|
+
sid6, = start_proposed(WRITE_PLAN)
|
|
326
|
+
r6 = act(sid6)
|
|
327
|
+
rec6 = verdict_records.last
|
|
328
|
+
assert('the PASS verdict is written to the chain') { rec6 && rec6['verdict'] == Verdict::PASS }
|
|
329
|
+
assert('the chain record names the planned route as confined') { rec6 && rec6['planned_route'] == 'confined' }
|
|
330
|
+
assert('nothing had been merged when the verdict was recorded') { merged_at_record_time == false }
|
|
331
|
+
assert('the file reached the project tree after the record') { File.read(MERGED).include?('fe66272') }
|
|
332
|
+
assert("the response reports 'completed' with a PASS verdict") do
|
|
333
|
+
r6['act_summary'] == 'completed' && r6.dig('guard', 'verdict') == Verdict::PASS
|
|
334
|
+
end
|
|
335
|
+
|
|
336
|
+
section 'A PASS the chain does not take: halt for the operator, nothing merged'
|
|
337
|
+
|
|
338
|
+
reset_chain!
|
|
339
|
+
FileUtils.rm_rf(File.join(PROJECT_ROOT, 'out'))
|
|
340
|
+
sid7, = start_proposed(WRITE_PLAN)
|
|
341
|
+
FakeChainStore.raise_with = RuntimeError.new('storage unavailable')
|
|
342
|
+
r7 = act(sid7)
|
|
343
|
+
FakeChainStore.raise_with = nil
|
|
344
|
+
assert('nothing was merged into the project tree') { !File.exist?(MERGED) }
|
|
345
|
+
assert('the response is a guard halt, not success and not a plain failure') { r7['status'] == 'guard_halt' }
|
|
346
|
+
assert('the response says the verdict was not recorded') { r7['verdict_recorded'] == false }
|
|
347
|
+
assert('the halt reason names the lost verdict, the cause, and that results stay quarantined') do
|
|
348
|
+
reason = r7['guard_reason'].to_s
|
|
349
|
+
reason.include?("verdict #{Verdict::PASS} (") && reason.include?('RuntimeError: storage unavailable') &&
|
|
350
|
+
reason.include?('quarantined')
|
|
351
|
+
end
|
|
352
|
+
prog7 = Session.load(sid7).load_progress.last
|
|
353
|
+
assert('progress records the halt, that the verdict was not recorded, and which verdict was lost') do
|
|
354
|
+
prog7['act_summary'].start_with?('guard halt') && prog7.dig('guard_record', 'verdict_recorded') == false &&
|
|
355
|
+
prog7.dig('guard_record', 'lost_verdict') == Verdict::PASS
|
|
356
|
+
end
|
|
357
|
+
|
|
358
|
+
section 'A FAIL the chain does not take also halts (it must not cycle on silently)'
|
|
359
|
+
|
|
360
|
+
reset_chain!
|
|
361
|
+
sid9, = start_proposed(REPORT_PLAN)
|
|
362
|
+
FakeChainStore.raise_with = RuntimeError.new('storage unavailable')
|
|
363
|
+
r9 = act(sid9)
|
|
364
|
+
FakeChainStore.raise_with = nil
|
|
365
|
+
assert('the response is a guard halt') { r9['status'] == 'guard_halt' && r9['verdict_recorded'] == false }
|
|
366
|
+
assert('the halt reason names the lost FAIL and that the in-process act already took effect') do
|
|
367
|
+
r9['guard_reason'].to_s.include?("verdict #{Verdict::FAIL} (") && r9['guard_reason'].to_s.include?('already took effect')
|
|
368
|
+
end
|
|
369
|
+
|
|
370
|
+
section 'Guard HALT before ACT (tampered spec) is recorded and reported as a halt'
|
|
371
|
+
|
|
372
|
+
reset_chain!
|
|
373
|
+
sid8, = start_proposed(REPORT_PLAN)
|
|
374
|
+
File.write(File.join(Session.load(sid8).guard_dir, Verdict::SPEC_FILE), '{"acceptance":[],"layer_surface":[]}')
|
|
375
|
+
r8 = act(sid8)
|
|
376
|
+
rec8 = verdict_records.last
|
|
377
|
+
assert('the HALT verdict is written to the chain') { rec8 && rec8['verdict'] == Verdict::HALT }
|
|
378
|
+
assert('the response is a guard halt, not a completion') { r8['status'] == 'guard_halt' && r8['verdict_recorded'] == true }
|
|
379
|
+
|
|
380
|
+
section 'Pre-ACT HALT whose record fails: the diagnostic names both problems'
|
|
381
|
+
|
|
382
|
+
reset_chain!
|
|
383
|
+
sid10, = start_proposed(REPORT_PLAN)
|
|
384
|
+
File.write(File.join(Session.load(sid10).guard_dir, Verdict::SPEC_FILE), '{"acceptance":[],"layer_surface":[]}')
|
|
385
|
+
FakeChainStore.raise_with = RuntimeError.new('storage unavailable')
|
|
386
|
+
r10 = act(sid10)
|
|
387
|
+
FakeChainStore.raise_with = nil
|
|
388
|
+
assert('the response is a guard halt with the verdict not recorded') do
|
|
389
|
+
r10['status'] == 'guard_halt' && r10['verdict_recorded'] == false
|
|
390
|
+
end
|
|
391
|
+
assert('the reason keeps the original tamper reason and adds the record failure') do
|
|
392
|
+
r10['guard_reason'].to_s.include?('tamper') && r10['guard_reason'].to_s.include?('storage unavailable') &&
|
|
393
|
+
r10['guard_reason'].to_s.include?('did not run')
|
|
394
|
+
end
|
|
395
|
+
|
|
396
|
+
section 'Manual risk-resume path: a guard halt is reported as a halt, not as ok'
|
|
397
|
+
|
|
398
|
+
reset_chain!
|
|
399
|
+
risky = decision('knowledge_update', { 'name' => 'x', 'content' => 'y' })
|
|
400
|
+
sid11, = start_proposed(risky)
|
|
401
|
+
paused = act(sid11)
|
|
402
|
+
assert('a medium-risk plan under a low budget pauses first') { paused['state'] == 'paused_risk' }
|
|
403
|
+
# The operator raises the budget, as the pause response asks, then resumes.
|
|
404
|
+
m11 = ::Autonomos::Mandate.load(Session.load(sid11).mandate_id)
|
|
405
|
+
m11[:risk_budget] = 'medium'
|
|
406
|
+
::Autonomos::Mandate.save(m11[:mandate_id], m11)
|
|
407
|
+
FakeChainStore.raise_with = RuntimeError.new('storage unavailable')
|
|
408
|
+
MockLlmCall.queue({ 'content' => '{"confidence":0.9}', 'tool_use' => nil, 'stop_reason' => 'end_turn' })
|
|
409
|
+
r11 = JSON.parse(STEP_TOOL.call({ 'session_id' => sid11, 'action' => 'approve' })[0][:text])
|
|
410
|
+
FakeChainStore.raise_with = nil
|
|
411
|
+
assert('resuming from the risk pause reports guard_halt with the verdict not recorded') do
|
|
412
|
+
r11['status'] == 'guard_halt' && r11['verdict_recorded'] == false
|
|
413
|
+
end
|
|
414
|
+
|
|
415
|
+
section 'Autonomous loop: a guard halt carries its reason and the record state'
|
|
416
|
+
|
|
417
|
+
reset_chain!
|
|
418
|
+
sid12 = JSON.parse(START_TOOL.call({ 'goal_name' => "guard_auto_#{SecureRandom.hex(3)}", 'guard' => ACCEPTANCE,
|
|
419
|
+
'autonomous' => true })[0][:text])['session_id']
|
|
420
|
+
MockLlmCall.clear!
|
|
421
|
+
MockLlmCall.queue({ 'content' => 'orient', 'tool_use' => nil, 'stop_reason' => 'end_turn' })
|
|
422
|
+
MockLlmCall.queue({ 'content' => REPORT_PLAN, 'tool_use' => nil, 'stop_reason' => 'end_turn' })
|
|
423
|
+
FakeChainStore.raise_with = RuntimeError.new('storage unavailable')
|
|
424
|
+
r12 = JSON.parse(STEP_TOOL.call({ 'session_id' => sid12, 'action' => 'approve' })[0][:text])
|
|
425
|
+
FakeChainStore.raise_with = nil
|
|
426
|
+
row12 = Array(r12['cycle_results']).last || {}
|
|
427
|
+
assert('the autonomous warning carries the halt reason and says the verdict was not recorded') do
|
|
428
|
+
r12['warning'].to_s.include?('could not be recorded') && r12['warning'].to_s.include?('verdict not recorded')
|
|
429
|
+
end
|
|
430
|
+
assert('the autonomous cycle row says guard_halt, the reason, and verdict_recorded false') do
|
|
431
|
+
row12['act_summary'] == 'guard_halt' && row12['guard_reason'].to_s.include?('storage unavailable') &&
|
|
432
|
+
row12['verdict_recorded'] == false
|
|
433
|
+
end
|
|
434
|
+
|
|
435
|
+
section 'Summaries in autonomous cycle_results and error-only results'
|
|
436
|
+
|
|
437
|
+
halt_r = STEP_TOOL.send(:guard_halt_result, Session.load(sid8), { 'guard_halt' => true },
|
|
438
|
+
Verdict.halt_verdict('x'))
|
|
439
|
+
assert("a guard-halt cycle result summarises as 'guard_halt'") do
|
|
440
|
+
STEP_TOOL.send(:response_act_summary, halt_r) == 'guard_halt'
|
|
441
|
+
end
|
|
442
|
+
assert("a result carrying only an error is not reported as 'completed'") do
|
|
443
|
+
STEP_TOOL.send(:response_act_summary, { act_error: 'No decision payload found', llm_calls: 0 }) == 'failed'
|
|
444
|
+
end
|
|
445
|
+
|
|
446
|
+
FileUtils.rm_rf(TMPDIR)
|
|
447
|
+
puts "\n#{'=' * 60}\nRESULT: #{$pass} passed, #{$fail} failed\n#{'=' * 60}"
|
|
448
|
+
exit($fail.zero? ? 0 : 1)
|
|
@@ -527,23 +527,53 @@ module KairosMcp
|
|
|
527
527
|
# is not an act at all — it must checkpoint WITHOUT running REFLECT,
|
|
528
528
|
# recording a cycle, or consuming a mandate cycle. Short-circuit here.
|
|
529
529
|
if act_result.is_a?(Hash) && act_result['guard_halt']
|
|
530
|
+
halt = act_result['guard_verdict'] ||
|
|
531
|
+
verdict_const.halt_verdict(act_result['error'].to_s)
|
|
532
|
+
block = record_guard_verdict(session, decision_payload, act_result, halt)
|
|
533
|
+
return guard_halt_result(session, act_result, halt, verdict_block: block) if block
|
|
534
|
+
|
|
530
535
|
return guard_halt_result(session, act_result,
|
|
531
|
-
act_result
|
|
532
|
-
|
|
536
|
+
unrecorded_verdict_halt(halt, act_result),
|
|
537
|
+
verdict_recorded: false, lost_verdict: halt['verdict'])
|
|
533
538
|
end
|
|
534
539
|
|
|
535
540
|
# Guard track (AGT-3): the mechanical verdict decides cycle success
|
|
536
541
|
# from driver-observed evidence, before and above REFLECT. On HALT or
|
|
537
542
|
# FAIL the cycle is not a success no matter what REFLECT later says.
|
|
538
543
|
guard_verdict = guard_judge(session, act_result)
|
|
544
|
+
|
|
545
|
+
# The verdict goes on the chain before anything acts on it — before
|
|
546
|
+
# the halt checkpoint and before the merge. Without this the chain
|
|
547
|
+
# held only autoexec's "execution complete" for an act the verdict
|
|
548
|
+
# had FAILed, and the FAIL survived only in a mutable session file
|
|
549
|
+
# (observed 2026-10-06 in a guarded trial).
|
|
550
|
+
verdict_block = guard_verdict && record_guard_verdict(session, decision_payload,
|
|
551
|
+
act_result, guard_verdict)
|
|
552
|
+
|
|
553
|
+
# A verdict the chain did not take stops the loop for the operator,
|
|
554
|
+
# whatever the verdict was. Nothing merges, so no change reaches the
|
|
555
|
+
# live tree without the record that licenses it. Counting the act as
|
|
556
|
+
# failed instead would invite the next cycle to redo effects that an
|
|
557
|
+
# in-process act has already applied, and a FAIL would otherwise
|
|
558
|
+
# keep cycling with its record missing — the original defect again.
|
|
559
|
+
if guard_verdict && verdict_block.nil?
|
|
560
|
+
return guard_halt_result(session, act_result, unrecorded_verdict_halt(guard_verdict, act_result),
|
|
561
|
+
verdict_recorded: false, lost_verdict: guard_verdict['verdict'])
|
|
562
|
+
end
|
|
539
563
|
if guard_verdict && guard_verdict['verdict'] == verdict_const::HALT
|
|
540
|
-
return guard_halt_result(session, act_result, guard_verdict)
|
|
564
|
+
return guard_halt_result(session, act_result, guard_verdict, verdict_block: verdict_block)
|
|
541
565
|
end
|
|
542
566
|
|
|
543
567
|
# AGT-1 return path: a PASS delegated act's results reach the live
|
|
544
568
|
# tree only now, through the driver's verdict-gated merge. A FAIL
|
|
545
569
|
# verdict leaves the scratch area unmerged (quarantined).
|
|
546
|
-
|
|
570
|
+
if guard_verdict && guard_verdict['verdict'] == verdict_const::PASS
|
|
571
|
+
merge_guard_pass(session, act_result, guard_verdict)
|
|
572
|
+
end
|
|
573
|
+
|
|
574
|
+
# REFLECT reads the act result; without this it would see only the
|
|
575
|
+
# executor's own "completed" for an act the verdict failed.
|
|
576
|
+
act_result['guard_verdict_summary'] = guard_verdict_summary(guard_verdict) if guard_verdict
|
|
547
577
|
|
|
548
578
|
session.update_state('reflecting')
|
|
549
579
|
reflect_loop = CognitiveLoop.new(self, session)
|
|
@@ -572,10 +602,11 @@ module KairosMcp
|
|
|
572
602
|
record_agent_cycle(session, decision_payload, act_result, reflect_result,
|
|
573
603
|
force_evaluation: forced)
|
|
574
604
|
|
|
575
|
-
act_summary = act_result
|
|
605
|
+
act_summary = reported_act_summary(act_result, guard_failed)
|
|
576
606
|
decision_summary = decision_payload['summary'] || ''
|
|
577
607
|
session.save_progress(reflect_result, session.cycle_number + 1, act_summary, decision_summary,
|
|
578
|
-
guard_record: build_guard_record(act_result, guard_verdict
|
|
608
|
+
guard_record: build_guard_record(act_result, guard_verdict,
|
|
609
|
+
verdict_block: verdict_block))
|
|
579
610
|
|
|
580
611
|
session.increment_cycle
|
|
581
612
|
session.save
|
|
@@ -589,17 +620,101 @@ module KairosMcp
|
|
|
589
620
|
|
|
590
621
|
result = { act: act_result, reflect: reflect_result, cycle: session.cycle_number,
|
|
591
622
|
act_error: act_result['error'], act_succeeded: act_succeeded,
|
|
592
|
-
llm_calls: reflect_loop.total_calls }
|
|
623
|
+
act_summary: act_summary, llm_calls: reflect_loop.total_calls }
|
|
593
624
|
result[:guard_verdict] = guard_verdict if guard_verdict
|
|
594
625
|
result
|
|
595
626
|
end
|
|
596
627
|
|
|
628
|
+
# What the record and every response call this act. A failing guard
|
|
629
|
+
# verdict outranks the executor's own summary: an executor reporting
|
|
630
|
+
# "completed" for an act the verdict failed is the very case the guard
|
|
631
|
+
# exists to catch. This string is also what the next cycle's executor
|
|
632
|
+
# is told about this one (build_agent_execute_context), so a false
|
|
633
|
+
# "completed" here would mislead the next cycle, not only the reader.
|
|
634
|
+
def reported_act_summary(act_result, guard_failed)
|
|
635
|
+
return 'failed' if guard_failed
|
|
636
|
+
|
|
637
|
+
act_result['summary'] || act_result['error'] || 'completed'
|
|
638
|
+
end
|
|
639
|
+
|
|
640
|
+
# The summary a response shows for a cycle result. Prefers the one the
|
|
641
|
+
# driver recorded; a guard halt result carries 'guard_halt'. A result
|
|
642
|
+
# that carries only an error is not reported as 'completed'.
|
|
643
|
+
def response_act_summary(result)
|
|
644
|
+
result[:act_summary] || result.dig(:act, 'summary') ||
|
|
645
|
+
(result[:act_error] ? 'failed' : 'completed')
|
|
646
|
+
end
|
|
647
|
+
|
|
648
|
+
# What a response shows the operator about the guard for one cycle.
|
|
649
|
+
# nil when the guard was off.
|
|
650
|
+
def guard_verdict_summary(guard_verdict)
|
|
651
|
+
return nil unless guard_verdict.is_a?(Hash)
|
|
652
|
+
|
|
653
|
+
{
|
|
654
|
+
'verdict' => guard_verdict['verdict'],
|
|
655
|
+
'failed_checks' => failed_guard_checks(guard_verdict).map { |c| c['type'] },
|
|
656
|
+
'reason' => guard_verdict['reason'].to_s[0, 200]
|
|
657
|
+
}
|
|
658
|
+
end
|
|
659
|
+
|
|
660
|
+
# The halt that replaces a verdict the chain did not take. Constant
|
|
661
|
+
# verdict keys are kept. The reason says which verdict was lost and
|
|
662
|
+
# why it was given, why the record failed, and what already happened
|
|
663
|
+
# to the act — an operator reading "nothing was merged" for an
|
|
664
|
+
# in-process act whose effects have landed would redo it.
|
|
665
|
+
def unrecorded_verdict_halt(guard_verdict, act_result)
|
|
666
|
+
act = act_result.is_a?(Hash) ? act_result : {}
|
|
667
|
+
scratch = act['scratch_dir'] || act.dig('execution', 'scratch_dir')
|
|
668
|
+
effects = if act['guard_halt']
|
|
669
|
+
'the act did not run'
|
|
670
|
+
elsif scratch
|
|
671
|
+
'the confined results stay quarantined, not merged'
|
|
672
|
+
else
|
|
673
|
+
'the in-process act already took effect; do not re-run it blindly'
|
|
674
|
+
end
|
|
675
|
+
cause = @guard_record_error || 'unknown'
|
|
676
|
+
{
|
|
677
|
+
'verdict' => ::KairosMcp::SkillSets::Agent::Verdict::HALT,
|
|
678
|
+
'checks' => guard_verdict['checks'] || [],
|
|
679
|
+
'spec_sha256' => guard_verdict['spec_sha256'],
|
|
680
|
+
'reason' => "guard verdict #{guard_verdict['verdict']} (#{guard_verdict['reason'].to_s[0, 160]}) " \
|
|
681
|
+
"could not be recorded on the chain (#{cause[0, 200]}); #{effects}. " \
|
|
682
|
+
'The loop stopped for the operator.'
|
|
683
|
+
}
|
|
684
|
+
end
|
|
685
|
+
|
|
686
|
+
def guard_halt_warning(result)
|
|
687
|
+
reason = result[:act_error].to_s[0, 300]
|
|
688
|
+
note = result[:verdict_recorded] == false ? ' [verdict not recorded on the chain]' : ''
|
|
689
|
+
"guard_halt: human review required — #{reason}#{note}"
|
|
690
|
+
end
|
|
691
|
+
|
|
692
|
+
# The response for a cycle that ended in a guard halt. Shared by the
|
|
693
|
+
# manual approve path and the manual risk-resume path, so neither can
|
|
694
|
+
# present a halt as an ordinary 'ok'.
|
|
695
|
+
def guard_halt_response(session, result)
|
|
696
|
+
gv = result[:guard_verdict] || {}
|
|
697
|
+
resp = {
|
|
698
|
+
'status' => 'guard_halt', 'session_id' => session.session_id,
|
|
699
|
+
'state' => 'checkpoint',
|
|
700
|
+
'act_summary' => 'guard_halt',
|
|
701
|
+
'guard_reason' => gv['reason'] || result[:act_error],
|
|
702
|
+
'guard_verdict' => gv,
|
|
703
|
+
'verdict_recorded' => result[:verdict_recorded] == true
|
|
704
|
+
}
|
|
705
|
+
resp['permission_advisory'] = session.permission_advisory if session.permission_advisory
|
|
706
|
+
resp
|
|
707
|
+
end
|
|
708
|
+
|
|
597
709
|
# Slice 2 NB-2 (R1 F7): the per-cycle constitutive record naming which
|
|
598
710
|
# executable carried the act — closure digest alongside the pinned
|
|
599
711
|
# spec hash, plus substrate identity. Empty for the CLI substrate
|
|
600
712
|
# (no staged closure) beyond the substrate label. Returns {} when
|
|
601
713
|
# there is nothing to record so save_progress omits the field.
|
|
602
|
-
|
|
714
|
+
# The verdict itself and the chain block holding it are part of the
|
|
715
|
+
# record too: the spec hash alone said what was asked, not what was found.
|
|
716
|
+
def build_guard_record(act_result, guard_verdict, verdict_block: nil, verdict_recorded: nil,
|
|
717
|
+
lost_verdict: nil)
|
|
603
718
|
return {} unless act_result.is_a?(Hash)
|
|
604
719
|
|
|
605
720
|
execution = act_result['execution'] || {}
|
|
@@ -610,22 +725,94 @@ module KairosMcp
|
|
|
610
725
|
rec['substrate'] = substrate if substrate
|
|
611
726
|
rec['closure_sha256'] = closure_sha if closure_sha
|
|
612
727
|
rec['spec_sha256'] = spec_sha if spec_sha
|
|
728
|
+
if guard_verdict.is_a?(Hash)
|
|
729
|
+
rec['verdict'] = guard_verdict['verdict']
|
|
730
|
+
failed = failed_guard_checks(guard_verdict)
|
|
731
|
+
rec['failed_checks'] = failed.map { |c| c['type'] } unless failed.empty?
|
|
732
|
+
rec['verdict_chain_block'] = verdict_block if verdict_block
|
|
733
|
+
rec['verdict_recorded'] = verdict_recorded.nil? ? !verdict_block.nil? : verdict_recorded
|
|
734
|
+
rec['lost_verdict'] = lost_verdict if lost_verdict
|
|
735
|
+
end
|
|
613
736
|
rec
|
|
614
737
|
end
|
|
615
738
|
|
|
739
|
+
def failed_guard_checks(guard_verdict)
|
|
740
|
+
Array(guard_verdict['checks']).reject do |c|
|
|
741
|
+
c['result'] == ::KairosMcp::SkillSets::Agent::Verdict::PASS
|
|
742
|
+
end
|
|
743
|
+
end
|
|
744
|
+
|
|
745
|
+
# Guard verdict -> chain (Proposition 5). Driver-authored, written on
|
|
746
|
+
# the driver's own context: the ACT context denies chain_record by
|
|
747
|
+
# design (admission.rb), and the act is not the one judging itself.
|
|
748
|
+
# Returns { 'index', 'hash' } of the block, or nil when the write did
|
|
749
|
+
# not land — success is read from chain_record's own confirmation,
|
|
750
|
+
# never assumed.
|
|
751
|
+
def record_guard_verdict(session, decision_payload, act_result, guard_verdict)
|
|
752
|
+
act = act_result.is_a?(Hash) ? act_result : {}
|
|
753
|
+
execution = act['execution'].is_a?(Hash) ? act['execution'] : {}
|
|
754
|
+
# Check details name files by their absolute scratch path; the chain
|
|
755
|
+
# keeps the workspace-relative part, not a temporary directory.
|
|
756
|
+
scratch = act['scratch_dir'] || execution['scratch_dir']
|
|
757
|
+
relative = ->(d) { scratch ? d.to_s.sub("#{scratch}/", '') : d.to_s }
|
|
758
|
+
@guard_record_error = nil
|
|
759
|
+
# The route the plan asked for; whether the act ran there is told
|
|
760
|
+
# by act_error (an unreachable executor fails before running).
|
|
761
|
+
planned = requires_file_operations?(decision_payload && decision_payload['task_json']) ? 'confined' : 'in_process'
|
|
762
|
+
record = {
|
|
763
|
+
'kind' => 'agent_guard_verdict',
|
|
764
|
+
'session_id' => session.session_id,
|
|
765
|
+
'mandate_id' => session.mandate_id,
|
|
766
|
+
'cycle' => session.cycle_number + 1,
|
|
767
|
+
'planned_route' => planned,
|
|
768
|
+
'verdict' => guard_verdict['verdict'],
|
|
769
|
+
'spec_sha256' => guard_verdict['spec_sha256'],
|
|
770
|
+
'reason' => guard_verdict['reason'].to_s[0, 200],
|
|
771
|
+
'failed_checks' => failed_guard_checks(guard_verdict).map { |c|
|
|
772
|
+
{ 'type' => c['type'], 'detail' => relative.call(c['detail'])[0, 160] }
|
|
773
|
+
},
|
|
774
|
+
'recorded_at' => Time.now.utc.iso8601
|
|
775
|
+
}
|
|
776
|
+
substrate = act['substrate'] || execution['substrate']
|
|
777
|
+
closure = act['closure_sha256'] || execution['closure_sha256']
|
|
778
|
+
record['substrate'] = substrate if substrate
|
|
779
|
+
record['closure_sha256'] = closure if closure
|
|
780
|
+
record['act_error'] = relative.call(act['error'])[0, 200] if act['error']
|
|
781
|
+
|
|
782
|
+
out = invoke_tool('chain_record', { 'logs' => [JSON.generate(record)] })
|
|
783
|
+
text = Array(out).map { |b| b[:text] || b['text'] }.compact.join
|
|
784
|
+
m = text.match(/Block #(\d+) recorded successfully\.\s*Hash: (\h+)/)
|
|
785
|
+
unless m
|
|
786
|
+
@guard_record_error = "chain_record replied: #{text[0, 160]}"
|
|
787
|
+
log_agent(:warn, 'guard_verdict_unrecorded', session, detail: text[0, 160])
|
|
788
|
+
return nil
|
|
789
|
+
end
|
|
790
|
+
{ 'index' => m[1].to_i, 'hash' => m[2] }
|
|
791
|
+
rescue StandardError => e
|
|
792
|
+
@guard_record_error = "#{e.class}: #{e.message}"
|
|
793
|
+
log_agent(:warn, 'guard_verdict_unrecorded', session, error: e.message)
|
|
794
|
+
nil
|
|
795
|
+
end
|
|
796
|
+
|
|
616
797
|
# AGT-6: build the checkpoint result for a guard halt without running
|
|
617
798
|
# REFLECT, recording a cycle, or consuming a mandate cycle. The human
|
|
618
799
|
# sees the halt reason, never a false "completed".
|
|
619
|
-
def guard_halt_result(session, act_result, guard_verdict
|
|
800
|
+
def guard_halt_result(session, act_result, guard_verdict, verdict_block: nil, verdict_recorded: nil,
|
|
801
|
+
lost_verdict: nil)
|
|
802
|
+
recorded = verdict_recorded.nil? ? !verdict_block.nil? : verdict_recorded
|
|
620
803
|
session.update_state('checkpoint')
|
|
621
804
|
session.save_progress({ 'confidence' => 0.0, 'guard_verdict' => guard_verdict },
|
|
622
805
|
session.cycle_number + 1,
|
|
623
806
|
"guard halt: #{guard_verdict['reason']}", '',
|
|
624
|
-
guard_record: build_guard_record(act_result, guard_verdict
|
|
807
|
+
guard_record: build_guard_record(act_result, guard_verdict,
|
|
808
|
+
verdict_block: verdict_block,
|
|
809
|
+
verdict_recorded: recorded,
|
|
810
|
+
lost_verdict: lost_verdict))
|
|
625
811
|
session.save
|
|
626
812
|
log_agent(:warn, 'guard_halt', session, reason: guard_verdict['reason'].to_s[0..120])
|
|
627
813
|
{ act: act_result, reflect: { 'confidence' => 0.0 }, cycle: session.cycle_number,
|
|
628
814
|
act_error: guard_verdict['reason'], act_succeeded: false, guard_halt: true,
|
|
815
|
+
act_summary: 'guard_halt', verdict_recorded: recorded,
|
|
629
816
|
guard_verdict: guard_verdict, llm_calls: 0 }
|
|
630
817
|
end
|
|
631
818
|
|
|
@@ -727,25 +914,16 @@ module KairosMcp
|
|
|
727
914
|
# cycle increment); the manual-mode response must not paper over it
|
|
728
915
|
# with the act_summary fallback below (guard_halt_result's stated
|
|
729
916
|
# contract: "the human sees the halt reason, never a false completed").
|
|
730
|
-
if result[:guard_halt]
|
|
731
|
-
gv = result[:guard_verdict] || {}
|
|
732
|
-
halt_response = {
|
|
733
|
-
'status' => 'guard_halt', 'session_id' => session.session_id,
|
|
734
|
-
'state' => 'checkpoint',
|
|
735
|
-
'act_summary' => 'guard_halt',
|
|
736
|
-
'guard_reason' => gv['reason'] || result[:act_error],
|
|
737
|
-
'guard_verdict' => gv
|
|
738
|
-
}
|
|
739
|
-
halt_response['permission_advisory'] = session.permission_advisory if session.permission_advisory
|
|
740
|
-
return text_content(JSON.generate(halt_response))
|
|
741
|
-
end
|
|
917
|
+
return text_content(JSON.generate(guard_halt_response(session, result))) if result[:guard_halt]
|
|
742
918
|
|
|
743
919
|
response = {
|
|
744
920
|
'status' => 'ok', 'session_id' => session.session_id,
|
|
745
921
|
'state' => 'checkpoint',
|
|
746
|
-
'act_summary' => result
|
|
922
|
+
'act_summary' => response_act_summary(result),
|
|
747
923
|
'reflect' => result[:reflect]
|
|
748
924
|
}
|
|
925
|
+
response['act_error'] = result[:act_error] if result[:act_error]
|
|
926
|
+
response['guard'] = guard_verdict_summary(result[:guard_verdict]) if result[:guard_verdict]
|
|
749
927
|
response['permission_advisory'] = session.permission_advisory if session.permission_advisory
|
|
750
928
|
text_content(JSON.generate(response))
|
|
751
929
|
end
|
|
@@ -1041,8 +1219,12 @@ module KairosMcp
|
|
|
1041
1219
|
if ar_result[:guard_halt]
|
|
1042
1220
|
session.update_state('checkpoint')
|
|
1043
1221
|
session.save
|
|
1222
|
+
# Steps the plan set aside for a person are still owed to the
|
|
1223
|
+
# operator when the cycle halts after an in-process act.
|
|
1224
|
+
deferred = Array(ar_result.dig(:act, 'deferred'))
|
|
1044
1225
|
return finalize_autonomous(session, results, checkpoint: true,
|
|
1045
|
-
warning:
|
|
1226
|
+
warning: guard_halt_warning(ar_result),
|
|
1227
|
+
deferred: deferred.empty? ? nil : deferred)
|
|
1046
1228
|
end
|
|
1047
1229
|
# Everything that could run has run. What is left needs the
|
|
1048
1230
|
# operator: the steps the plan marked for a person, and the
|
|
@@ -1156,10 +1338,14 @@ module KairosMcp
|
|
|
1156
1338
|
'error' => error,
|
|
1157
1339
|
'warning' => warning,
|
|
1158
1340
|
'cycle_results' => cycle_results.map { |r|
|
|
1159
|
-
{ 'cycle' => r[:cycle],
|
|
1160
|
-
|
|
1161
|
-
|
|
1162
|
-
|
|
1341
|
+
row = { 'cycle' => r[:cycle],
|
|
1342
|
+
'act_summary' => response_act_summary(r),
|
|
1343
|
+
'confidence' => clamp_confidence(r.dig(:reflect, 'confidence')),
|
|
1344
|
+
'remaining_count' => Array(r.dig(:reflect, 'remaining')).size }
|
|
1345
|
+
row['guard_verdict'] = r[:guard_verdict]['verdict'] if r[:guard_verdict].is_a?(Hash)
|
|
1346
|
+
row['guard_reason'] = r[:act_error] if r[:guard_halt]
|
|
1347
|
+
row['verdict_recorded'] = false if r[:verdict_recorded] == false
|
|
1348
|
+
row
|
|
1163
1349
|
}
|
|
1164
1350
|
}
|
|
1165
1351
|
reports = surface_reports(session)
|
|
@@ -1276,8 +1462,10 @@ module KairosMcp
|
|
|
1276
1462
|
if ar_result[:guard_halt]
|
|
1277
1463
|
session.update_state('checkpoint')
|
|
1278
1464
|
session.save
|
|
1465
|
+
deferred = Array(ar_result.dig(:act, 'deferred'))
|
|
1279
1466
|
return finalize_autonomous(session, [ar_result], checkpoint: true,
|
|
1280
|
-
warning:
|
|
1467
|
+
warning: guard_halt_warning(ar_result),
|
|
1468
|
+
deferred: deferred.empty? ? nil : deferred)
|
|
1281
1469
|
end
|
|
1282
1470
|
if ar_result[:act_error]
|
|
1283
1471
|
session.update_state('paused_error')
|
|
@@ -1294,12 +1482,17 @@ module KairosMcp
|
|
|
1294
1482
|
result = run_act_reflect_internal(session)
|
|
1295
1483
|
session.update_state('checkpoint')
|
|
1296
1484
|
session.save
|
|
1297
|
-
text_content(JSON.generate(
|
|
1485
|
+
return text_content(JSON.generate(guard_halt_response(session, result))) if result[:guard_halt]
|
|
1486
|
+
|
|
1487
|
+
response = {
|
|
1298
1488
|
'status' => 'ok', 'session_id' => session.session_id,
|
|
1299
1489
|
'state' => 'checkpoint',
|
|
1300
|
-
'act_summary' => result
|
|
1490
|
+
'act_summary' => response_act_summary(result),
|
|
1301
1491
|
'reflect' => result[:reflect]
|
|
1302
|
-
}
|
|
1492
|
+
}
|
|
1493
|
+
response['act_error'] = result[:act_error] if result[:act_error]
|
|
1494
|
+
response['guard'] = guard_verdict_summary(result[:guard_verdict]) if result[:guard_verdict]
|
|
1495
|
+
text_content(JSON.generate(response))
|
|
1303
1496
|
end
|
|
1304
1497
|
end
|
|
1305
1498
|
|
|
@@ -535,7 +535,27 @@ class Run
|
|
|
535
535
|
cfg = { 'sandbox_mode' => true, 'timeout_seconds' => t }
|
|
536
536
|
cfg['effort'] = spec[:effort] if spec[:effort]
|
|
537
537
|
LC::ClaudeCodeAdapter.new(cfg)
|
|
538
|
-
when 'codex'
|
|
538
|
+
when 'codex'
|
|
539
|
+
# The effort has to be handed to the adapter, not merely written into
|
|
540
|
+
# the lineup. Harnesses without this branch (up to ccf1dabd, the
|
|
541
|
+
# template shipped through gem 3.88.0) only wrote it: the effort set
|
|
542
|
+
# for a codex seat (--codex-effort) or a codex game master (--gm-effort,
|
|
543
|
+
# default high) was published in the lineup and never passed, so the
|
|
544
|
+
# CLI ran at whatever ~/.codex/config.toml said. A nil codex effort in
|
|
545
|
+
# a lineup therefore means "no level was handed to the CLI"; for every
|
|
546
|
+
# call that reached its first turn, the level that ran is in codex's own
|
|
547
|
+
# session logs (~/.codex/sessions, per-turn "effort"), not in the lineup.
|
|
548
|
+
# Two games of 2026-09-05 record "high" without handing it over:
|
|
549
|
+
# log/minimum_nomic_gm_20260810/astra_g1_seat for the codex seat and
|
|
550
|
+
# astra_g2_gm for the codex game master. Codex's session logs show every
|
|
551
|
+
# recorded call of the two games at high. With no -c passed, that level
|
|
552
|
+
# came from the CLI's own configuration (~/.codex/config.toml, whose
|
|
553
|
+
# 2026-09-05 content is not on record), so the recorded level is what
|
|
554
|
+
# ran — but not because the harness set it. A harness with this branch
|
|
555
|
+
# delivers the level it records.
|
|
556
|
+
cfg = { 'timeout_seconds' => t }
|
|
557
|
+
cfg['effort'] = spec[:effort] if spec[:effort]
|
|
558
|
+
LC::CodexAdapter.new(cfg)
|
|
539
559
|
when 'cursor' then LC::CursorAdapter.new('timeout_seconds' => t)
|
|
540
560
|
else raise "unknown adapter #{spec[:adapter]}"
|
|
541
561
|
end
|
|
@@ -652,6 +672,41 @@ class Run
|
|
|
652
672
|
end
|
|
653
673
|
|
|
654
674
|
# Each seat's capability is declared and recorded, not constrained (INV-31).
|
|
675
|
+
# What the codex seat reads before it reads the prompt, MEASURED when this
|
|
676
|
+
# file is loaded (once per run) rather than asserted. The field used to carry
|
|
677
|
+
# the flat sentence "no AGENTS.md exists in the project root (pilot measured
|
|
678
|
+
# 0 words)", true when the pilot was played and false whenever the
|
|
679
|
+
# instruction mode is projected to AGENTS.md, as it was from 2026-09-05 —
|
|
680
|
+
# 75,012 bytes then, reaching every codex call, the game master's included
|
|
681
|
+
# once --gm-adapter codex is used. A hardcoded absence is the one claim in
|
|
682
|
+
# this record that gets falsified by a file appearing next to it, so the file
|
|
683
|
+
# is read off the filesystem and the reader is told its size and first
|
|
684
|
+
# heading. Only the run's working directory is measured, and the sentence
|
|
685
|
+
# names what was not: codex may also read AGENTS.override.md there, the same
|
|
686
|
+
# files in parent directories up to the git root, and ~/.codex/AGENTS.md, and
|
|
687
|
+
# it truncates at its project_doc_max_bytes. A file that cannot be read is
|
|
688
|
+
# recorded as unread instead of stopping the run.
|
|
689
|
+
AGENTS_MD = File.join(PROJECT_ROOT, 'AGENTS.md')
|
|
690
|
+
CODEX_UNMEASURED = 'Not measured: AGENTS.override.md in this directory, AGENTS.md or ' \
|
|
691
|
+
'AGENTS.override.md in parent directories up to the git root, ' \
|
|
692
|
+
"~/.codex/AGENTS.md, and codex's project_doc_max_bytes truncation."
|
|
693
|
+
CODEX_INSTRUCTION_FILES =
|
|
694
|
+
begin
|
|
695
|
+
if File.file?(AGENTS_MD)
|
|
696
|
+
body = File.binread(AGENTS_MD).force_encoding(Encoding::UTF_8).scrub
|
|
697
|
+
heading = body.lines.find { |l| l.start_with?('#') }&.strip
|
|
698
|
+
"AGENTS.md IS PRESENT in the run's working directory: #{File.size(AGENTS_MD)} bytes, " \
|
|
699
|
+
"#{body.split.length} words, first heading #{heading ? %("#{heading}") : 'none'}. Codex " \
|
|
700
|
+
'looks here for instruction files for the codex seats and a codex game master; presence ' \
|
|
701
|
+
"is measured, delivery is not. #{CODEX_UNMEASURED}"
|
|
702
|
+
else
|
|
703
|
+
"no AGENTS.md in the run's working directory at run start (measured). #{CODEX_UNMEASURED}"
|
|
704
|
+
end
|
|
705
|
+
rescue SystemCallError => e
|
|
706
|
+
"AGENTS.md in the run's working directory could not be read at run start (#{e.class}); " \
|
|
707
|
+
"what the codex seats received from it is unknown. #{CODEX_UNMEASURED}"
|
|
708
|
+
end
|
|
709
|
+
|
|
655
710
|
# The seats do not run at equal capability and the difference is not being
|
|
656
711
|
# levelled: it is written down so no later reading treats these three as
|
|
657
712
|
# interchangeable. `record_reach` is the honest part — see write_lineup!.
|
|
@@ -676,7 +731,7 @@ class Run
|
|
|
676
731
|
'invocation' => 'codex exec --sandbox read-only, prompt on stdin',
|
|
677
732
|
'tools' => 'read-only filesystem access',
|
|
678
733
|
'cwd' => 'the project root (not chdir\'d)',
|
|
679
|
-
'instruction_files_reachable' =>
|
|
734
|
+
'instruction_files_reachable' => CODEX_INSTRUCTION_FILES,
|
|
680
735
|
'record_reach' => 'READ. A read-only sandbox can open this run\'s record files if it looks'
|
|
681
736
|
},
|
|
682
737
|
'cursor' => {
|
|
@@ -927,6 +982,53 @@ class Run
|
|
|
927
982
|
# utterance, so once reasoning reaches the public log the check reads the leak
|
|
928
983
|
# as ordinary quotation and stays green. It counted 5,156 excused matches on
|
|
929
984
|
# this game.
|
|
985
|
+
#
|
|
986
|
+
# 2026-09-09, measured. The 2026-08-13 note above describes the form as a
|
|
987
|
+
# player breaking it. That is not what the record shows. Across the six games
|
|
988
|
+
# of log/nomic_astra_20260908 the form occurs 24 times. 23 are one shape:
|
|
989
|
+
# <reasoning> opened AND closed, <utterance> opened, the move written, and no
|
|
990
|
+
# </utterance> — exactly one closing tag is missing. The 24th (game b3, turn
|
|
991
|
+
# 23) closes the utterance with a second </reasoning> instead. The regex
|
|
992
|
+
# below requires </utterance>, so in all 24 `u` is nil and the whole reply —
|
|
993
|
+
# tags, reasoning and move — becomes the public utterance.
|
|
994
|
+
#
|
|
995
|
+
# Downstream, measured: a public log carrying a tag-opened multi-paragraph
|
|
996
|
+
# block, delivered together with the instruction to produce private reasoning,
|
|
997
|
+
# is refused by the safeguard with Details: `[reasoning_extraction]`. In
|
|
998
|
+
# bundle A game 2, 36 of the game's 50 player calls failed after the leak,
|
|
999
|
+
# and the log is append-only, so the block stays in every later prompt (the
|
|
1000
|
+
# refusing seat, C, still answered once, at turn 44). Only the first failed
|
|
1001
|
+
# prompt was replayed. That replay isolated it on 2026-09-09: neither the
|
|
1002
|
+
# block nor the instruction refused alone, the tag name was immaterial
|
|
1003
|
+
# (<notes> refused too), and a single sentence under the tag did not refuse.
|
|
1004
|
+
# On 2026-09-23 the bare turn-3 prompt, with no block, was refused 10 of 10;
|
|
1005
|
+
# log/nomic_opus55_20260923/NOTE.md (item 2) reads this as the safeguard
|
|
1006
|
+
# having moved between the two dates.
|
|
1007
|
+
#
|
|
1008
|
+
# RULING 2026-09-09 (operator), verbatim:
|
|
1009
|
+
#
|
|
1010
|
+
# 「閉じタグをつけ忘れた」ことも、「閉じタグがないために厳しく解釈した」ことも、
|
|
1011
|
+
# 結果として記録してください。正しく読めるように修正する必要はありません。
|
|
1012
|
+
# 「閉じタグ」がなくても正しく評価(メタ評価)できるかどうか、も LLM の認知能力評価
|
|
1013
|
+
# になりますし、「閉じタグを忘れた」ことも LLM のタスク実行能力評価に繋がります。
|
|
1014
|
+
# しっかりとルールを守るように harness 側を構築することはこの nomic-bench の
|
|
1015
|
+
# 主たる方法ではありません。未完成のルールシステムの中で LLM がどう振る舞うか、
|
|
1016
|
+
# そこがこの nomic-bench の評価するべきポイントなのを忘れないように記録して
|
|
1017
|
+
# おいてくれませんか?
|
|
1018
|
+
#
|
|
1019
|
+
# The verbatim is also in log/nomic_astra_20260908/NOTE.md. Rendered in
|
|
1020
|
+
# English: record both facts — that the model forgot the closing tag, and
|
|
1021
|
+
# that the harness read its absence strictly — as results; there is no need
|
|
1022
|
+
# to correct them so that the reply reads properly. Whether a model can still evaluate the record correctly
|
|
1023
|
+
# (a meta-evaluation) with the tag missing is itself an evaluation of its
|
|
1024
|
+
# cognition, and forgetting the tag bears on an evaluation of its task
|
|
1025
|
+
# execution. Building the harness so that rules are reliably obeyed is not
|
|
1026
|
+
# this bench's main method. How a language model behaves inside an
|
|
1027
|
+
# incomplete rule system is the point this bench is meant to evaluate.
|
|
1028
|
+
#
|
|
1029
|
+
# Consequences the note draws from the ruling (the note's words, not the
|
|
1030
|
+
# ruling's): this regex is not relaxed to accept an unclosed <utterance>,
|
|
1031
|
+
# and no re-ask is added.
|
|
930
1032
|
def split_player_reply(reply)
|
|
931
1033
|
return { utterance: nil, reasoning: nil, form: 'call_failed' } if reply.nil?
|
|
932
1034
|
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kairos-chain
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.88.
|
|
4
|
+
version: 3.88.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Masaomi Hatakeyama
|
|
@@ -423,6 +423,7 @@ files:
|
|
|
423
423
|
- templates/skillsets/agent/test/test_agent_complexity_review.rb
|
|
424
424
|
- templates/skillsets/agent/test/test_agent_confinement.rb
|
|
425
425
|
- templates/skillsets/agent/test/test_agent_guard_integration.rb
|
|
426
|
+
- templates/skillsets/agent/test/test_agent_guard_verdict_record.rb
|
|
426
427
|
- templates/skillsets/agent/test/test_agent_halt_semantics.rb
|
|
427
428
|
- templates/skillsets/agent/test/test_agent_human_mark_gate.rb
|
|
428
429
|
- templates/skillsets/agent/test/test_agent_interruption_resilience.rb
|