kairos-chain 3.76.0 → 3.77.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +54 -0
- data/lib/kairos_mcp/version.rb +1 -1
- data/templates/skillsets/agent/config/agent.yml +13 -2
- data/templates/skillsets/agent/lib/agent/mandate_adapter.rb +26 -2
- data/templates/skillsets/agent/skillset.json +2 -1
- data/templates/skillsets/agent/test/test_agent_complexity_review.rb +1 -1
- data/templates/skillsets/agent/test/test_agent_halt_semantics.rb +2 -2
- data/templates/skillsets/agent/test/test_agent_human_mark_gate.rb +478 -0
- data/templates/skillsets/agent/test/test_agent_m2.rb +1 -1
- data/templates/skillsets/agent/test/test_agent_m3.rb +1 -1
- data/templates/skillsets/agent/test/test_agent_m4.rb +1 -1
- data/templates/skillsets/agent/tools/agent_start.rb +8 -2
- data/templates/skillsets/agent/tools/agent_step.rb +287 -13
- data/templates/skillsets/agent/tools/operator_report.rb +130 -0
- data/templates/skillsets/autoexec/tools/autoexec_run.rb +69 -1
- data/templates/skillsets/autonomos/lib/autonomos/mandate.rb +38 -8
- data/templates/skillsets/kairos_hook_projector/hooks/readable_gate.rb +312 -104
- data/templates/skillsets/kairos_hook_projector/test/mutation_check_readable_gate.rb +220 -44
- data/templates/skillsets/kairos_hook_projector/test/test_readable_gate.rb +1031 -146
- data/templates/skillsets/minimum_nomic/README.md +7 -0
- data/templates/skillsets/minimum_nomic/bin/propose_metric.rb +17 -3
- data/templates/skillsets/minimum_nomic/skillset.json +1 -1
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ca968a084e8868b9cbeaa3283850dd7e56095a503d661ce6e74158ec7555f0e3
|
|
4
|
+
data.tar.gz: 7a410b2edb98cd9bf02c8396f11d1d411e9b190456ea87402789d9a95aea5f16
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 5cea432521cb70bcc7ed1f354bb0a4abe1165d16a25ed185d8b3c46af5db0d00d57c459737e3a52f290a356d29356033cce8e6ef4c56e7526438eb6f7eba30d3
|
|
7
|
+
data.tar.gz: cf536e54e866a943d95d0209741c1a6658e6df62421f9a5e0e082983e98025057aa4e4dfc04ad08a4f96e7acfd9a982e69bde253d6f3986302325d9dd7ab65f3
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,60 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
This project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [3.77.0] - 2026-08-26
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **`operator_report` — the agent SkillSet's first tool for handing a
|
|
12
|
+
deliverable to a human.** Three runs produced documents no one could receive;
|
|
13
|
+
the loop concluded "there is no way to deliver this" and it was right. The
|
|
14
|
+
tool writes the report body to a file under `.kairos/log/agent_reports/` and
|
|
15
|
+
returns only the path, because the executor truncates tool output at 500
|
|
16
|
+
characters. It is registered `low` in the risk table — an unlisted tool would
|
|
17
|
+
be marked by norm (d) and the report itself would wait for human
|
|
18
|
+
confirmation. Two norms ship with it: (e) an irreversible step is marked and
|
|
19
|
+
deferred, not a reason to halt the plan; (f) a delivery tool exists — do not
|
|
20
|
+
conclude there is no way to hand the result over.
|
|
21
|
+
|
|
22
|
+
- **`test_agent_human_mark_gate.rb`** — 39 tests, 87 assertions, covering the
|
|
23
|
+
deferred-mark path, halt-kind classification, and the cumulative cycle
|
|
24
|
+
ledger.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
|
|
28
|
+
- **Seven fixes surfaced by running the loop, not by reviewing it.** The goal
|
|
29
|
+
text never reached the DECIDE prompt — the loop was judging without its goal;
|
|
30
|
+
it is now included, truncated at 6,000 characters with the cut declared in
|
|
31
|
+
the text. Halt kinds were hardcoded to one kind, so tool errors were recorded
|
|
32
|
+
as "waiting for human"; the kind now reads from each step's own status, and
|
|
33
|
+
the seven-site refactor a prior design round estimated for this turned out
|
|
34
|
+
unnecessary. A plan halted at its first human-marked step; `human_mark_mode:
|
|
35
|
+
'defer'` sets marked steps and their dependents aside, runs the rest, and
|
|
36
|
+
returns the deferred list at the cycle's end (the default remains `halt`).
|
|
37
|
+
The completed-cycle count was per call; the ledger now accumulates across the
|
|
38
|
+
run. Norm (d) pulled read-only tools into confirmation waits; it now names
|
|
39
|
+
writing, recording, sending, and deleting tools only, and its counter is
|
|
40
|
+
renamed `unlisted_writing_tool_unmarked` with the old name's counts folded
|
|
41
|
+
in. Report titles doubled when the body already opened with a heading; they
|
|
42
|
+
no longer do.
|
|
43
|
+
|
|
44
|
+
### Removed
|
|
45
|
+
|
|
46
|
+
- **The 300-second wall clock and `max_cycles`.** The clock was shorter than
|
|
47
|
+
one measured cycle (823 seconds for a 13-step cycle); both caps are gone.
|
|
48
|
+
What still stops a run: the call budget (`max_total_llm_calls`, default 60 —
|
|
49
|
+
about 15 cycles at the measured 4 calls per cycle), two consecutive errors,
|
|
50
|
+
loop detection, self-declared completion at confidence ≥ 0.9, and the
|
|
51
|
+
`checkpoint_every` return, where continuing is decided by whether the
|
|
52
|
+
operator calls the next `agent_step`. The uncapped run that validated this
|
|
53
|
+
went three cycles: the first drafted a priority list, the second found and
|
|
54
|
+
corrected six errors in it, the third declined to write a third version,
|
|
55
|
+
declared the second canonical, and returned four open items — zero norm
|
|
56
|
+
violations, confidence never reaching the self-completion threshold. Mutation
|
|
57
|
+
testing covers the session's first half (11 mutants, 11 killed); the latter
|
|
58
|
+
eight fixes ship unmutated by operator decision — this release exists to find
|
|
59
|
+
their defects in use.
|
|
60
|
+
|
|
7
61
|
## [3.76.0] - 2026-08-22
|
|
8
62
|
|
|
9
63
|
### Changed
|
data/lib/kairos_mcp/version.rb
CHANGED
|
@@ -30,6 +30,10 @@ tool_blacklist:
|
|
|
30
30
|
- "skills_rollback"
|
|
31
31
|
- "skills_promote"
|
|
32
32
|
- "instructions_update"
|
|
33
|
+
# The norms the agent is held to live in L1. A run with risk_budget: medium
|
|
34
|
+
# would otherwise reach knowledge_update (medium in TOOL_RISK) and let the
|
|
35
|
+
# constrained side rewrite what constrains it.
|
|
36
|
+
- "knowledge_update"
|
|
33
37
|
- "token_manage"
|
|
34
38
|
- "system_upgrade"
|
|
35
39
|
- "chain_import"
|
|
@@ -82,8 +86,15 @@ orient_tools_extra: []
|
|
|
82
86
|
|
|
83
87
|
# Autonomous mode limits
|
|
84
88
|
autonomous:
|
|
85
|
-
|
|
86
|
-
|
|
89
|
+
# The cost bound, and the one that actually stops a runaway. At the ~4 calls
|
|
90
|
+
# per cycle measured on 2026-08-26 this is about 15 cycles.
|
|
91
|
+
max_total_llm_calls: 60
|
|
92
|
+
# Wall-clock bound, off. Blank or 0 means no bound: the loop runs until it is
|
|
93
|
+
# done rather than until the clock says so. It never interrupted a running
|
|
94
|
+
# cycle — it only refused to start the next — so at 300 against a measured
|
|
95
|
+
# 823-second cycle it just made max_cycles above 1 unreachable. Withdrawn
|
|
96
|
+
# 2026-08-26; put a number back only if a run is seen to go on too long.
|
|
97
|
+
max_duration_seconds:
|
|
87
98
|
min_cycles_before_exit: 2 # confidence exit disabled for first N cycles
|
|
88
99
|
confidence_exit_threshold: 0.9 # minimum confidence for early exit
|
|
89
100
|
|
|
@@ -6,13 +6,37 @@ module KairosMcp
|
|
|
6
6
|
# Bridges agent structures to Autonomos::Mandate API shapes.
|
|
7
7
|
# Input: string keys (from JSON.parse). Output: symbol keys (for Mandate API).
|
|
8
8
|
module MandateAdapter
|
|
9
|
+
# Tools whose presence routes the whole plan to the agent_execute
|
|
10
|
+
# subcontractor instead of in-process autoexec.
|
|
11
|
+
#
|
|
12
|
+
# Defined here rather than in agent_step because the risk gate and the
|
|
13
|
+
# ACT router must agree on the route. If they disagree, a plan can be
|
|
14
|
+
# granted the human-mark exemption below and then run under the
|
|
15
|
+
# subcontractor, which formats steps as prose and never reads the mark
|
|
16
|
+
# — the marked step would be delegated rather than halted on.
|
|
17
|
+
FILE_TOOL_NAMES = %w[Edit Write Read Bash file_edit file_write file_read].freeze
|
|
18
|
+
|
|
19
|
+
def self.routes_to_subcontractor?(task_json)
|
|
20
|
+
steps = task_json && task_json['steps']
|
|
21
|
+
Array(steps).any? { |s| FILE_TOOL_NAMES.include?(s['tool_name']) }
|
|
22
|
+
end
|
|
23
|
+
|
|
9
24
|
# Convert decision_payload to Mandate-compatible proposal
|
|
10
25
|
# for Mandate.risk_exceeds_budget? and Mandate.loop_detected?
|
|
26
|
+
#
|
|
27
|
+
# enforce_human_marks declares that this caller halts before a marked
|
|
28
|
+
# step at execution time. It lives inside autoexec_task, beside the
|
|
29
|
+
# steps it qualifies, because risk_exceeds_budget? reads that hash and a
|
|
30
|
+
# declaration written elsewhere than it is read is the whole defect.
|
|
11
31
|
def self.to_mandate_proposal(decision_payload)
|
|
32
|
+
task_json = decision_payload['task_json']
|
|
12
33
|
{
|
|
13
34
|
autoexec_task: {
|
|
14
|
-
|
|
15
|
-
|
|
35
|
+
enforce_human_marks: !routes_to_subcontractor?(task_json),
|
|
36
|
+
steps: Array(task_json && task_json['steps']).map { |s|
|
|
37
|
+
{ risk: s['risk'] || 'low',
|
|
38
|
+
tool_name: s['tool_name'],
|
|
39
|
+
requires_human_cognition: s['requires_human_cognition'] == true }
|
|
16
40
|
}
|
|
17
41
|
},
|
|
18
42
|
selected_gap: {
|
|
@@ -14,7 +14,8 @@
|
|
|
14
14
|
"KairosMcp::SkillSets::Agent::Tools::AgentStep",
|
|
15
15
|
"KairosMcp::SkillSets::Agent::Tools::AgentStatus",
|
|
16
16
|
"KairosMcp::SkillSets::Agent::Tools::AgentStop",
|
|
17
|
-
"KairosMcp::SkillSets::Agent::Tools::AgentWait"
|
|
17
|
+
"KairosMcp::SkillSets::Agent::Tools::AgentWait",
|
|
18
|
+
"KairosMcp::SkillSets::Agent::Tools::OperatorReport"
|
|
18
19
|
],
|
|
19
20
|
"config_files": ["config/agent.yml"],
|
|
20
21
|
"knowledge_dirs": ["knowledge/agent_guide"],
|
|
@@ -69,7 +69,7 @@ module Autonomos
|
|
|
69
69
|
end
|
|
70
70
|
end
|
|
71
71
|
|
|
72
|
-
require File.expand_path('
|
|
72
|
+
require File.expand_path('../autonomos/lib/autonomos/mandate',
|
|
73
73
|
File.dirname(__dir__))
|
|
74
74
|
|
|
75
75
|
Session = KairosMcp::SkillSets::Agent::Session
|
|
@@ -67,10 +67,10 @@ module Autonomos
|
|
|
67
67
|
end
|
|
68
68
|
end
|
|
69
69
|
|
|
70
|
-
require File.expand_path('
|
|
70
|
+
require File.expand_path('../autonomos/lib/autonomos/mandate',
|
|
71
71
|
File.dirname(__dir__))
|
|
72
72
|
# The autonomous loop reaches Ooda for goal loading and COMPLEX_KEYWORDS.
|
|
73
|
-
require File.expand_path('
|
|
73
|
+
require File.expand_path('../autonomos/lib/autonomos/ooda',
|
|
74
74
|
File.dirname(__dir__))
|
|
75
75
|
|
|
76
76
|
Session = KairosMcp::SkillSets::Agent::Session
|
|
@@ -0,0 +1,478 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# Acceptance criteria for the "norms first" slice of
|
|
5
|
+
# docs/drafts/agent_judgment_norms_loop_v0.9.md §5.
|
|
6
|
+
#
|
|
7
|
+
# 1 a marked step is exempt from the risk count only under the declaration
|
|
8
|
+
# 2 a plan bound for the subcontractor route gets no exemption
|
|
9
|
+
# 3 the autonomos setup plan is still stopped at both budgets
|
|
10
|
+
# 7 knowledge_update is denied by config, not by the risk gate
|
|
11
|
+
# 8 run metrics carry attempted cycles as the denominator
|
|
12
|
+
# 9 norms (c) and (d) are counted by machine; (a) and (b) are not
|
|
13
|
+
#
|
|
14
|
+
# Reads the SHIPPED template copies, not the instance copies under .kairos.
|
|
15
|
+
# Usage: ruby test_agent_human_mark_gate.rb
|
|
16
|
+
|
|
17
|
+
require 'minitest/autorun'
|
|
18
|
+
require 'json'
|
|
19
|
+
require 'yaml'
|
|
20
|
+
require 'fileutils'
|
|
21
|
+
require 'tmpdir'
|
|
22
|
+
|
|
23
|
+
$test_kairos_dir = Dir.mktmpdir('agent_mark_gate_test')
|
|
24
|
+
|
|
25
|
+
module Autoexec
|
|
26
|
+
def self.loaded?
|
|
27
|
+
true
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
module KairosMcp
|
|
32
|
+
def self.data_dir
|
|
33
|
+
$test_kairos_dir
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
module Tools
|
|
37
|
+
class BaseTool
|
|
38
|
+
def text_content(text)
|
|
39
|
+
text
|
|
40
|
+
end
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# The parent, not just mandate.rb: Mandate.save resolves its path through
|
|
46
|
+
# Autonomos.storage_path, which lives there.
|
|
47
|
+
require File.expand_path('../autonomos/lib/autonomos', File.dirname(__dir__))
|
|
48
|
+
require_relative '../lib/agent/mandate_adapter'
|
|
49
|
+
|
|
50
|
+
$LOAD_PATH.unshift File.expand_path('../../../lib', __dir__)
|
|
51
|
+
$LOAD_PATH.unshift File.expand_path('../lib', __dir__)
|
|
52
|
+
$LOAD_PATH.unshift File.expand_path('../../autoexec/lib', __dir__)
|
|
53
|
+
require 'kairos_mcp/invocation_context'
|
|
54
|
+
require 'kairos_mcp/tools/base_tool'
|
|
55
|
+
require 'kairos_mcp/tool_registry'
|
|
56
|
+
require_relative '../lib/agent'
|
|
57
|
+
require_relative '../tools/agent_step'
|
|
58
|
+
require File.expand_path('../autoexec/lib/autoexec', File.dirname(__dir__))
|
|
59
|
+
require File.expand_path('../autoexec/tools/autoexec_run', File.dirname(__dir__))
|
|
60
|
+
|
|
61
|
+
Mandate = Autonomos::Mandate
|
|
62
|
+
Adapter = KairosMcp::SkillSets::Agent::MandateAdapter
|
|
63
|
+
StepTool = KairosMcp::SkillSets::Agent::Tools::AgentStep
|
|
64
|
+
RunTool = KairosMcp::SkillSets::Autoexec::Tools::AutoexecRun
|
|
65
|
+
require_relative '../tools/operator_report'
|
|
66
|
+
ReportTool = KairosMcp::SkillSets::Agent::Tools::OperatorReport
|
|
67
|
+
|
|
68
|
+
def payload(steps, task_id: 'tsk_1', summary: 'gap')
|
|
69
|
+
{ 'summary' => summary,
|
|
70
|
+
'task_json' => { 'task_id' => task_id, 'steps' => steps } }
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def step(tool, risk: 'low', marked: false, id: 's1')
|
|
74
|
+
{ 'step_id' => id, 'tool_name' => tool, 'risk' => risk,
|
|
75
|
+
'requires_human_cognition' => marked }
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
# --- 1: the exemption, and its limit ----------------------------------------
|
|
79
|
+
class TestMarkedStepExemption < Minitest::Test
|
|
80
|
+
# A plan of nothing but marked high steps must clear budget low. This is the
|
|
81
|
+
# 2026-08-22 case: the model marked the step honestly and lost the plan.
|
|
82
|
+
def test_marked_only_plan_clears_low_budget
|
|
83
|
+
p = Adapter.to_mandate_proposal(
|
|
84
|
+
payload([{ 'step_id' => 's1', 'tool_name' => '', 'risk' => 'high',
|
|
85
|
+
'requires_human_cognition' => true }])
|
|
86
|
+
)
|
|
87
|
+
assert_equal true, p[:autoexec_task][:enforce_human_marks]
|
|
88
|
+
refute Mandate.risk_exceeds_budget?(p, 'low')
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# ...and one unmarked over-budget step in the same plan brings it back.
|
|
92
|
+
# An implementation that exempts the whole plan when any step is marked
|
|
93
|
+
# passes the first assertion and fails here.
|
|
94
|
+
def test_one_unmarked_over_budget_step_still_exceeds
|
|
95
|
+
p = Adapter.to_mandate_proposal(
|
|
96
|
+
payload([{ 'step_id' => 's1', 'tool_name' => '', 'risk' => 'high',
|
|
97
|
+
'requires_human_cognition' => true },
|
|
98
|
+
step('safe_file_write', id: 's2')])
|
|
99
|
+
)
|
|
100
|
+
assert Mandate.risk_exceeds_budget?(p, 'low')
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
# The flag is read strictly, matching TaskDsl (task_dsl.rb:117). A truthy
|
|
104
|
+
# string must not buy the exemption.
|
|
105
|
+
def test_string_false_does_not_mark
|
|
106
|
+
p = Adapter.to_mandate_proposal(
|
|
107
|
+
payload([{ 'step_id' => 's1', 'tool_name' => '', 'risk' => 'high',
|
|
108
|
+
'requires_human_cognition' => 'false' }])
|
|
109
|
+
)
|
|
110
|
+
assert Mandate.risk_exceeds_budget?(p, 'low')
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
# The adapter normalises the flag, so the case above never reaches the gate
|
|
114
|
+
# through that path. The gate must not lean on the adapter having done it:
|
|
115
|
+
# any other caller can hand it a truthy non-true value.
|
|
116
|
+
def test_gate_reads_the_mark_strictly_on_a_raw_proposal
|
|
117
|
+
raw = { autoexec_task: { enforce_human_marks: true,
|
|
118
|
+
steps: [{ risk: 'high', tool_name: '',
|
|
119
|
+
requires_human_cognition: 'false' }] },
|
|
120
|
+
selected_gap: { description: 'x' } }
|
|
121
|
+
assert Mandate.risk_exceeds_budget?(raw, 'low')
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# Without the declaration the mark buys nothing, whoever built the hash.
|
|
125
|
+
def test_no_declaration_means_no_exemption
|
|
126
|
+
raw = { autoexec_task: { steps: [{ risk: 'high', tool_name: '',
|
|
127
|
+
requires_human_cognition: true }] },
|
|
128
|
+
selected_gap: { description: 'x' } }
|
|
129
|
+
assert Mandate.risk_exceeds_budget?(raw, 'low')
|
|
130
|
+
end
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
# --- 2: the subcontractor route gets no exemption ----------------------------
|
|
134
|
+
class TestSubcontractorRouteHasNoExemption < Minitest::Test
|
|
135
|
+
# format_steps_as_instructions never reads the mark, so a marked delete
|
|
136
|
+
# would be handed to the subcontractor as prose. The plan must stay refused.
|
|
137
|
+
def test_marked_delete_beside_unmarked_file_write_is_refused
|
|
138
|
+
p = Adapter.to_mandate_proposal(
|
|
139
|
+
payload([step('safe_file_delete', risk: 'high', marked: true, id: 's1'),
|
|
140
|
+
step('file_write', id: 's2')])
|
|
141
|
+
)
|
|
142
|
+
assert_equal false, p[:autoexec_task][:enforce_human_marks]
|
|
143
|
+
assert Mandate.risk_exceeds_budget?(p, 'low')
|
|
144
|
+
assert Mandate.risk_exceeds_budget?(p, 'medium')
|
|
145
|
+
end
|
|
146
|
+
|
|
147
|
+
def test_route_predicate_matches_every_file_tool_name
|
|
148
|
+
%w[Edit Write Read Bash file_edit file_write file_read].each do |tool|
|
|
149
|
+
assert Adapter.routes_to_subcontractor?({ 'steps' => [step(tool)] }),
|
|
150
|
+
"#{tool} should route to the subcontractor"
|
|
151
|
+
end
|
|
152
|
+
refute Adapter.routes_to_subcontractor?({ 'steps' => [step('safe_file_read')] })
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
def test_missing_task_json_is_not_a_subcontractor_route
|
|
156
|
+
refute Adapter.routes_to_subcontractor?(nil)
|
|
157
|
+
assert_equal true, Adapter.to_mandate_proposal({ 'summary' => 's' })[:autoexec_task][:enforce_human_marks]
|
|
158
|
+
end
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# --- 3: the autonomos setup plan is still stopped ----------------------------
|
|
162
|
+
class TestAutonomosSetupPlanStillStops < Minitest::Test
|
|
163
|
+
# The shape ooda.rb builds for a high-priority gap (ooda.rb:85-107): no
|
|
164
|
+
# tool_name at all, the implement step at the gap's risk, and marked when the
|
|
165
|
+
# gap is setup. Autonomos builds its own proposal and never goes through the
|
|
166
|
+
# adapter, so no declaration is present and the mark buys nothing.
|
|
167
|
+
def ooda_shaped_setup_proposal
|
|
168
|
+
{ autoexec_task: {
|
|
169
|
+
steps: [
|
|
170
|
+
{ step_id: 'analyze', risk: 'low', requires_human_cognition: false },
|
|
171
|
+
{ step_id: 'implement', risk: 'high', requires_human_cognition: true },
|
|
172
|
+
{ step_id: 'verify', risk: 'low', requires_human_cognition: false }
|
|
173
|
+
]
|
|
174
|
+
},
|
|
175
|
+
selected_gap: { description: 'setup' } }
|
|
176
|
+
end
|
|
177
|
+
|
|
178
|
+
def test_setup_plan_stops_at_both_budgets
|
|
179
|
+
assert Mandate.risk_exceeds_budget?(ooda_shaped_setup_proposal, 'low')
|
|
180
|
+
assert Mandate.risk_exceeds_budget?(ooda_shaped_setup_proposal, 'medium')
|
|
181
|
+
end
|
|
182
|
+
|
|
183
|
+
# The policy at ooda.rb:58-61 — a high-priority gap always needs human
|
|
184
|
+
# confirmation — survives only because autonomos does not build its proposal
|
|
185
|
+
# through the adapter. Wiring it up would grant the implement step the
|
|
186
|
+
# exemption and retract that policy silently, so guard the wiring itself.
|
|
187
|
+
def test_autonomos_loop_does_not_build_proposals_through_the_adapter
|
|
188
|
+
src = File.read(
|
|
189
|
+
File.expand_path('../autonomos/tools/autonomos_loop.rb', File.dirname(__dir__))
|
|
190
|
+
)
|
|
191
|
+
refute_match(/MandateAdapter/, src)
|
|
192
|
+
end
|
|
193
|
+
end
|
|
194
|
+
|
|
195
|
+
# --- 7: knowledge_update is denied by config --------------------------------
|
|
196
|
+
class TestNormEditingToolIsDenied < Minitest::Test
|
|
197
|
+
def agent_yml
|
|
198
|
+
YAML.load_file(File.expand_path('../config/agent.yml', __dir__))
|
|
199
|
+
end
|
|
200
|
+
|
|
201
|
+
# The gate cannot carry this one: knowledge_update is medium, so a run at
|
|
202
|
+
# budget medium passes it. The blacklist is the only thing standing between
|
|
203
|
+
# the agent and the norms that bind it.
|
|
204
|
+
def test_gate_alone_would_let_the_norm_editor_through
|
|
205
|
+
p = Adapter.to_mandate_proposal(payload([step('knowledge_update', risk: 'medium')]))
|
|
206
|
+
refute Mandate.risk_exceeds_budget?(p, 'medium')
|
|
207
|
+
end
|
|
208
|
+
|
|
209
|
+
def test_blacklist_denies_it
|
|
210
|
+
assert_includes agent_yml['tool_blacklist'], 'knowledge_update'
|
|
211
|
+
end
|
|
212
|
+
|
|
213
|
+
# Already present before this slice; asserted so a later edit cannot drop
|
|
214
|
+
# them while adding the new entry.
|
|
215
|
+
def test_blacklist_still_denies_the_other_two_norm_editors
|
|
216
|
+
assert_includes agent_yml['tool_blacklist'], 'instructions_update'
|
|
217
|
+
assert_includes agent_yml['tool_blacklist'], 'skills_promote'
|
|
218
|
+
end
|
|
219
|
+
end
|
|
220
|
+
|
|
221
|
+
# --- 8: the exit tally must be able to add up --------------------------------
|
|
222
|
+
class TestExitReasonKey < Minitest::Test
|
|
223
|
+
def key(reason)
|
|
224
|
+
StepTool.new(nil, registry: nil).send(:exit_reason_key, reason)
|
|
225
|
+
end
|
|
226
|
+
|
|
227
|
+
# The defect the first run exposed (2026-08-26): the halt reason carries the
|
|
228
|
+
# step id in its prose, so a whole-string key gives every halt its own bucket.
|
|
229
|
+
def test_halts_at_different_steps_share_one_key
|
|
230
|
+
a = key('human_cognition_halt at step s2: Review the step, then re-run')
|
|
231
|
+
b = key('human_cognition_halt at step s7: Review the step, then re-run')
|
|
232
|
+
assert_equal 'human_cognition_halt', a
|
|
233
|
+
assert_equal a, b
|
|
234
|
+
end
|
|
235
|
+
|
|
236
|
+
# The same shape with a colon rather than a space — the other prose reason in
|
|
237
|
+
# the loop. Naming only the case that was observed leaves this one open.
|
|
238
|
+
def test_colon_form_is_folded_too
|
|
239
|
+
assert_equal 'guard_halt', key('guard_halt: human review required')
|
|
240
|
+
end
|
|
241
|
+
|
|
242
|
+
# Bare identifiers must survive untouched, or the fold silently renames the
|
|
243
|
+
# exits that were already countable.
|
|
244
|
+
def test_bare_reasons_are_unchanged
|
|
245
|
+
%w[max_cycles_reached goal_achieved loop_detected risk_exceeded timeout
|
|
246
|
+
llm_budget_exceeded act_failed review_rejected review_max_retries
|
|
247
|
+
goal_content_changed l0_requires_external_review checkpoint error].each do |r|
|
|
248
|
+
assert_equal r, key(r)
|
|
249
|
+
end
|
|
250
|
+
end
|
|
251
|
+
|
|
252
|
+
# A reason that does not begin with an identifier must land in one bounded
|
|
253
|
+
# bucket rather than becoming a key of its own.
|
|
254
|
+
def test_unrecognised_reasons_land_in_one_bucket
|
|
255
|
+
assert_equal 'unclassified', key('Review the step')
|
|
256
|
+
assert_equal 'unclassified', key('')
|
|
257
|
+
assert_equal 'unclassified', key(nil)
|
|
258
|
+
end
|
|
259
|
+
|
|
260
|
+
# Folding on write is not enough. A mandate written by the code that shipped
|
|
261
|
+
# before this fix carries whole-sentence keys, and a run that resumes it must
|
|
262
|
+
# not leave the old key sitting beside the folded one, counting the same exit
|
|
263
|
+
# twice. Drives the real accumulator against a real mandate.
|
|
264
|
+
FakeSession = Struct.new(:mandate_id, :cycle_number)
|
|
265
|
+
|
|
266
|
+
def test_a_sentence_key_left_by_an_earlier_run_is_folded_on_read
|
|
267
|
+
m = Mandate.create(goal_name: 'g', goal_hash: 'h', max_cycles: 3,
|
|
268
|
+
checkpoint_every: 3, risk_budget: 'low')
|
|
269
|
+
m[:exits] = { 'human_cognition_halt at step s2: Review the step' => 1 }
|
|
270
|
+
Mandate.save(m[:mandate_id], m)
|
|
271
|
+
|
|
272
|
+
metrics = StepTool.new(nil, registry: nil).send(
|
|
273
|
+
:accumulate_run_metrics,
|
|
274
|
+
FakeSession.new(m[:mandate_id], 1),
|
|
275
|
+
'human_cognition_halt at step s7: Review the step', []
|
|
276
|
+
)
|
|
277
|
+
refute_nil metrics, 'the accumulator swallowed an error'
|
|
278
|
+
assert_equal({ 'human_cognition_halt' => 2 }, metrics['exits'])
|
|
279
|
+
end
|
|
280
|
+
end
|
|
281
|
+
|
|
282
|
+
# --- the halt kind, the goal in the prompt, the run-wide cycle count ---------
|
|
283
|
+
class TestHaltKind < Minitest::Test
|
|
284
|
+
def kind(results, at)
|
|
285
|
+
RunTool.new(nil, registry: nil).send(:halt_kind, results, at)
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
# Six paths reach the stop and the reason used to name only the first. The
|
|
289
|
+
# live run on 2026-08-26 stopped on a tool error and was filed as waiting for
|
|
290
|
+
# a person.
|
|
291
|
+
def test_a_tool_error_is_not_a_person
|
|
292
|
+
assert_equal 'step_failed',
|
|
293
|
+
kind([{ step_id: :s4, status: 'failed', error: 'pm_item rejected the update' }], :s4)
|
|
294
|
+
end
|
|
295
|
+
|
|
296
|
+
def test_the_human_path_leaves_no_row_and_is_read_from_that
|
|
297
|
+
assert_equal 'human_cognition', kind([], :s2)
|
|
298
|
+
end
|
|
299
|
+
|
|
300
|
+
def test_the_other_kinds_keep_their_own_names
|
|
301
|
+
assert_equal 'tool_missing',
|
|
302
|
+
kind([{ step_id: :s1, status: 'failed', error: "Tool 'x' not found in registry" }], :s1)
|
|
303
|
+
assert_equal 'policy_denied', kind([{ step_id: :s1, status: 'policy_denied' }], :s1)
|
|
304
|
+
assert_equal 'blocked_on_delegated', kind([{ step_id: :s1, status: 'blocked' }], :s1)
|
|
305
|
+
end
|
|
306
|
+
end
|
|
307
|
+
|
|
308
|
+
class TestGoalReachesDecide < Minitest::Test
|
|
309
|
+
FakeSession = Struct.new(:mandate_id, :cycle_number, :goal_name)
|
|
310
|
+
|
|
311
|
+
def tool_with_goal(text)
|
|
312
|
+
t = StepTool.new(nil, registry: nil)
|
|
313
|
+
t.define_singleton_method(:load_goal_content) { |_| text }
|
|
314
|
+
t
|
|
315
|
+
end
|
|
316
|
+
|
|
317
|
+
# The goal text reached OBSERVE and stopped there, so a goal saying "read
|
|
318
|
+
# only" produced a plan that wrote. Nothing disobeyed; DECIDE never saw it.
|
|
319
|
+
def test_the_prohibition_is_in_the_prompt
|
|
320
|
+
t = tool_with_goal("# GOAL\n禁止事項: ファイルを作成しない")
|
|
321
|
+
t.define_singleton_method(:build_tool_catalog) { |_| 'resource_read' }
|
|
322
|
+
prompt = t.send(:build_decide_prompt, FakeSession.new('m', 0, 'g'), { 'content' => 'analysis' })
|
|
323
|
+
assert_includes prompt, '禁止事項: ファイルを作成しない'
|
|
324
|
+
assert_includes prompt, 'analysis'
|
|
325
|
+
end
|
|
326
|
+
|
|
327
|
+
def test_a_long_goal_is_cut_and_says_so
|
|
328
|
+
excerpt = tool_with_goal('あ' * 9000).send(:goal_excerpt, FakeSession.new('m', 0, 'g'))
|
|
329
|
+
assert_includes excerpt, 'truncated'
|
|
330
|
+
assert_operator excerpt.length, :<, 9000
|
|
331
|
+
end
|
|
332
|
+
|
|
333
|
+
def test_no_goal_means_no_section
|
|
334
|
+
t = tool_with_goal(nil)
|
|
335
|
+
t.define_singleton_method(:build_tool_catalog) { |_| 'resource_read' }
|
|
336
|
+
prompt = t.send(:build_decide_prompt, FakeSession.new('m', 0, 'g'), { 'content' => 'analysis' })
|
|
337
|
+
refute_includes prompt, 'The goal, verbatim'
|
|
338
|
+
end
|
|
339
|
+
end
|
|
340
|
+
|
|
341
|
+
class TestUnlimitedCycles < Minitest::Test
|
|
342
|
+
# Withdrawn 2026-08-26. A cycle count fixed in advance is a guess about how
|
|
343
|
+
# long the work takes; the operator already gets the decision back at every
|
|
344
|
+
# checkpoint.
|
|
345
|
+
def test_a_mandate_can_be_created_without_a_cycle_cap
|
|
346
|
+
m = Mandate.create(goal_name: 'g', goal_hash: 'h', max_cycles: nil,
|
|
347
|
+
checkpoint_every: 3, risk_budget: 'low')
|
|
348
|
+
assert_nil m[:max_cycles]
|
|
349
|
+
end
|
|
350
|
+
|
|
351
|
+
def test_an_uncapped_mandate_never_terminates_on_cycle_count
|
|
352
|
+
m = { max_cycles: nil, cycles_completed: 500, consecutive_errors: 0 }
|
|
353
|
+
assert_nil Mandate.check_termination(m)
|
|
354
|
+
end
|
|
355
|
+
|
|
356
|
+
# The other stoppers stay. Removing the cap must not remove the error floor.
|
|
357
|
+
def test_the_error_floor_still_stops_an_uncapped_run
|
|
358
|
+
m = { max_cycles: nil, cycles_completed: 500, consecutive_errors: 2 }
|
|
359
|
+
assert_equal 'error_threshold', Mandate.check_termination(m)
|
|
360
|
+
end
|
|
361
|
+
|
|
362
|
+
def test_a_cap_still_works_when_one_is_given
|
|
363
|
+
m = { max_cycles: 3, cycles_completed: 3, consecutive_errors: 0 }
|
|
364
|
+
assert_equal 'max_cycles_reached', Mandate.check_termination(m)
|
|
365
|
+
assert_raises(ArgumentError) do
|
|
366
|
+
Mandate.create(goal_name: 'g', goal_hash: 'h', max_cycles: 99,
|
|
367
|
+
checkpoint_every: 3, risk_budget: 'low')
|
|
368
|
+
end
|
|
369
|
+
end
|
|
370
|
+
|
|
371
|
+
# checkpoint_every is what hands the decision back, so it stays bounded even
|
|
372
|
+
# when the cycle count is not.
|
|
373
|
+
def test_checkpoint_every_is_still_bounded_without_a_cap
|
|
374
|
+
assert_raises(ArgumentError) do
|
|
375
|
+
Mandate.create(goal_name: 'g', goal_hash: 'h', max_cycles: nil,
|
|
376
|
+
checkpoint_every: 9, risk_budget: 'low')
|
|
377
|
+
end
|
|
378
|
+
end
|
|
379
|
+
end
|
|
380
|
+
|
|
381
|
+
class TestOperatorReportBody < Minitest::Test
|
|
382
|
+
def body(title, content)
|
|
383
|
+
ReportTool.new(nil, registry: nil).send(:body, title, content)
|
|
384
|
+
end
|
|
385
|
+
|
|
386
|
+
# The first report written, 2026-08-26, opened with two headings: the model
|
|
387
|
+
# wrote one and the tool prepended another.
|
|
388
|
+
def test_no_second_heading_when_the_text_already_has_one
|
|
389
|
+
assert_equal "# 今週の順位\n\n本文", body('今週の優先順位', "# 今週の順位\n\n本文")
|
|
390
|
+
end
|
|
391
|
+
|
|
392
|
+
def test_the_title_becomes_the_heading_when_there_is_none
|
|
393
|
+
assert_equal "# 今週の順位\n\n本文", body('今週の順位', '本文')
|
|
394
|
+
end
|
|
395
|
+
|
|
396
|
+
def test_no_title_means_the_text_is_left_alone
|
|
397
|
+
assert_equal '本文', body('', '本文')
|
|
398
|
+
end
|
|
399
|
+
end
|
|
400
|
+
|
|
401
|
+
class TestUnlistedToolCount < Minitest::Test
|
|
402
|
+
FakeSession = Struct.new(:mandate_id, :cycle_number)
|
|
403
|
+
|
|
404
|
+
def count_for(steps)
|
|
405
|
+
m = Mandate.create(goal_name: 'g', goal_hash: 'h', max_cycles: 3,
|
|
406
|
+
checkpoint_every: 3, risk_budget: 'low')
|
|
407
|
+
tool = StepTool.new(nil, registry: nil)
|
|
408
|
+
tool.send(:observe_norms, FakeSession.new(m[:mandate_id], 1),
|
|
409
|
+
{ 'task_id' => 't1', 'steps' => steps })
|
|
410
|
+
Mandate.load(m[:mandate_id])[:norm_breaks][:unlisted_writing_tool_unmarked].to_i
|
|
411
|
+
end
|
|
412
|
+
|
|
413
|
+
# The 2026-08-26 run counted four read-only queries as breaches. Reading is
|
|
414
|
+
# not changing, and marking those would have stopped the plan for nothing.
|
|
415
|
+
def test_read_only_unlisted_tools_are_not_counted
|
|
416
|
+
steps = %w[pm_digest pm_query safe_git_status].map.with_index { |t, i|
|
|
417
|
+
{ 'step_id' => "s#{i}", 'tool_name' => t, 'risk' => 'low' }
|
|
418
|
+
}
|
|
419
|
+
assert_equal 0, count_for(steps)
|
|
420
|
+
end
|
|
421
|
+
|
|
422
|
+
def test_an_unlisted_tool_the_model_called_medium_is_counted
|
|
423
|
+
assert_equal 1, count_for([{ 'step_id' => 's1', 'tool_name' => 'pm_item', 'risk' => 'medium' }])
|
|
424
|
+
end
|
|
425
|
+
|
|
426
|
+
def test_marking_it_settles_the_norm
|
|
427
|
+
assert_equal 0, count_for([{ 'step_id' => 's1', 'tool_name' => 'pm_item',
|
|
428
|
+
'risk' => 'medium', 'requires_human_cognition' => true }])
|
|
429
|
+
end
|
|
430
|
+
|
|
431
|
+
# A tool in the table is the machine's judgement, not the model's, so it is
|
|
432
|
+
# outside this norm whatever risk the model assigned.
|
|
433
|
+
def test_a_listed_tool_is_not_counted
|
|
434
|
+
assert_equal 0, count_for([{ 'step_id' => 's1', 'tool_name' => 'chain_record', 'risk' => 'high' }])
|
|
435
|
+
end
|
|
436
|
+
end
|
|
437
|
+
|
|
438
|
+
class TestCompletedCyclesSpanTheRun < Minitest::Test
|
|
439
|
+
FakeSession = Struct.new(:mandate_id, :cycle_number)
|
|
440
|
+
|
|
441
|
+
# Reported 1 completed against 2 attempted on the second call, because one
|
|
442
|
+
# side counted the run and the other counted the call.
|
|
443
|
+
def test_the_count_accumulates_across_calls
|
|
444
|
+
m = Mandate.create(goal_name: 'g', goal_hash: 'h', max_cycles: 3,
|
|
445
|
+
checkpoint_every: 3, risk_budget: 'low')
|
|
446
|
+
tool = StepTool.new(nil, registry: nil)
|
|
447
|
+
tool.send(:accumulate_run_metrics, FakeSession.new(m[:mandate_id], 1), 'checkpoint', [{ cycle: 1 }])
|
|
448
|
+
second = tool.send(:accumulate_run_metrics, FakeSession.new(m[:mandate_id], 2), 'checkpoint', [{ cycle: 2 }])
|
|
449
|
+
assert_equal 2, second['attempted_cycles']
|
|
450
|
+
assert_equal 2, second['completed_cycles']
|
|
451
|
+
end
|
|
452
|
+
end
|
|
453
|
+
|
|
454
|
+
# --- 9 (partial): what the risk table does and does not cover ----------------
|
|
455
|
+
class TestRiskTableCoverage < Minitest::Test
|
|
456
|
+
# Norm (d) exists because the table is thin. If the table ever grows to cover
|
|
457
|
+
# the registry, the norm loses its reason and this test should be revisited.
|
|
458
|
+
# 28 since operator_report was added on 2026-08-26: a tool the plan must be
|
|
459
|
+
# able to call without marking it has to be in the table, because the norms
|
|
460
|
+
# tell the model to mark anything that is not.
|
|
461
|
+
def test_table_is_thinner_than_the_registry
|
|
462
|
+
assert_equal 28, Mandate::TOOL_RISK.size
|
|
463
|
+
assert_equal 'low', Mandate::TOOL_RISK['operator_report']
|
|
464
|
+
assert_equal 3, Mandate::TOOL_RISK.count { |_, r| r == 'high' }
|
|
465
|
+
%w[safe_file_delete safe_git_push].each do |tool|
|
|
466
|
+
assert_equal 'high', Mandate::TOOL_RISK[tool]
|
|
467
|
+
end
|
|
468
|
+
end
|
|
469
|
+
|
|
470
|
+
# high exceeds every budget there is, so the two tools above are the only
|
|
471
|
+
# ones the mechanism refuses outright.
|
|
472
|
+
def test_high_exceeds_every_budget
|
|
473
|
+
Mandate::RISK_BUDGETS.each do |budget|
|
|
474
|
+
p = Adapter.to_mandate_proposal(payload([step('safe_file_delete', risk: 'high')]))
|
|
475
|
+
assert Mandate.risk_exceeds_budget?(p, budget), "high should exceed #{budget}"
|
|
476
|
+
end
|
|
477
|
+
end
|
|
478
|
+
end
|
|
@@ -64,7 +64,7 @@ module Autonomos
|
|
|
64
64
|
end
|
|
65
65
|
|
|
66
66
|
# Load Mandate for testing
|
|
67
|
-
require File.expand_path('
|
|
67
|
+
require File.expand_path('../autonomos/lib/autonomos/mandate',
|
|
68
68
|
File.dirname(__dir__))
|
|
69
69
|
|
|
70
70
|
Session = KairosMcp::SkillSets::Agent::Session
|