kairos-chain 3.66.0 → 3.68.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +27 -0
- data/bin/kairos-chain +116 -0
- data/lib/kairos_mcp/version.rb +1 -1
- data/templates/skillsets/kairos_hook_projector/lib/readable_gate_setup.rb +112 -0
- data/templates/skillsets/kairos_hook_projector/test/test_readable_gate_setup.rb +136 -0
- data/templates/skillsets/minimum_nomic/README.md +150 -0
- data/templates/skillsets/minimum_nomic/bin/audit_scores.rb +217 -0
- data/templates/skillsets/minimum_nomic/bin/check_gm.rb +270 -0
- data/templates/skillsets/minimum_nomic/bin/cross_model.rb +126 -0
- data/templates/skillsets/minimum_nomic/bin/reanalyse.rb +147 -0
- data/templates/skillsets/minimum_nomic/bin/rules_initial.json +73 -0
- data/templates/skillsets/minimum_nomic/bin/run_gm.rb +978 -0
- data/templates/skillsets/minimum_nomic/skillset.json +13 -0
- metadata +11 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7ac682d1d3fc16af582fb7eadccc08c263b9a2161805de89859e0c0d036c694a
|
|
4
|
+
data.tar.gz: 5d5ebdcac2a80196a1ed4bdd322b69b5de9a027c23f384d315ab7ab5a5119e5f
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 12b7c204a2e70e581d6746452e5faa64c991c8f07536d1531f3aebbae45dadcd783ccd75a736bd00bacecf92ffb00cd5dbb6cb1cba285fb8447d82c9899ca1d1
|
|
7
|
+
data.tar.gz: 63188706bfebce9144966aa795f26b875102ac66696e89100fd90e3b5fbca09705910a790cb0bf284dc20270a290e7a413cf7cb32413ec09fb55f2183f0ef991
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,33 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
This project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [3.67.0] - 2026-08-15
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **The Minimum Nomic bench ships.** Three language models play a self-amending
|
|
12
|
+
game with nine changeable rules, no goal and no termination condition, while a
|
|
13
|
+
fourth decides only who speaks next. It lived at `bench/minimum_nomic/` in the
|
|
14
|
+
development checkout, which the gemspec never walked, so nobody outside this
|
|
15
|
+
repository could reproduce a run. It is now the `minimum_nomic` SkillSet under
|
|
16
|
+
`templates/skillsets/`, with the five scripts in `bin/`.
|
|
17
|
+
|
|
18
|
+
The single thing that kept it out was a hard-coded `../../.kairos/` path to the
|
|
19
|
+
`llm_client` adapters, which resolved only in the development checkout.
|
|
20
|
+
`llm_client` is now resolved as a **sibling SkillSet**
|
|
21
|
+
(`../../llm_client/lib/llm_client`), the same relative path in the gem's
|
|
22
|
+
templates and in a projected instance. When it is absent the run aborts naming
|
|
23
|
+
both directories it searched, rather than failing inside a `require`.
|
|
24
|
+
|
|
25
|
+
`bin/cross_model.rb` is new: it analyses a stored game with one named model at
|
|
26
|
+
one named reasoning effort, instead of the panel recorded in the game's own
|
|
27
|
+
line-up. Without it, a comparison between model generations reuses both stored
|
|
28
|
+
panels and confounds generation with effort — the stored games ran
|
|
29
|
+
`claude-opus-4-6` at medium and `claude-opus-5` at high.
|
|
30
|
+
|
|
31
|
+
Not yet demonstrated: no game has been played from an installed gem. Until one
|
|
32
|
+
has, treat distribution as untested.
|
|
33
|
+
|
|
7
34
|
## [3.65.0] - 2026-08-13
|
|
8
35
|
|
|
9
36
|
### Changed
|
data/bin/kairos-chain
CHANGED
|
@@ -406,6 +406,122 @@ when 'upgrade'
|
|
|
406
406
|
end
|
|
407
407
|
exit
|
|
408
408
|
|
|
409
|
+
when 'gate'
|
|
410
|
+
# Readable-gate enablement in one call. The judgement lives in the
|
|
411
|
+
# kairos_hook_projector SkillSet, which is not a core SkillSet: this loads it
|
|
412
|
+
# from the instance and refuses when it is absent, the shape `attestation
|
|
413
|
+
# scan` already uses for synoptis. Loading it from the gem's templates instead
|
|
414
|
+
# would run a different copy than the one installed here.
|
|
415
|
+
ARGV.shift
|
|
416
|
+
|
|
417
|
+
action = ARGV.shift
|
|
418
|
+
if action.nil? || %w[-h --help help].include?(action)
|
|
419
|
+
puts <<~HELP
|
|
420
|
+
Usage: kairos-chain gate enable <mode> [--section "§ Heading"] [--apply]
|
|
421
|
+
|
|
422
|
+
Add the readable gate to an instruction mode: write the declaration
|
|
423
|
+
from the catalogue, then install it into .claude/settings.json.
|
|
424
|
+
|
|
425
|
+
Without --apply it proposes and writes nothing.
|
|
426
|
+
|
|
427
|
+
It does not read the mode body and decides nothing about it. The
|
|
428
|
+
section and the thresholds come from the catalogue and are the mode's
|
|
429
|
+
own to edit afterwards, in the declaration this prints. `blocking`
|
|
430
|
+
starts false, which is what the catalogue ships: run report-only,
|
|
431
|
+
read the log, then set it true and re-run with --apply.
|
|
432
|
+
|
|
433
|
+
A verdict of OPEN_QUESTIONS after a successful install is not a
|
|
434
|
+
failure — it means some other section of the mode states a limit with
|
|
435
|
+
no recorded decision. Record it under not_gated in the declaration.
|
|
436
|
+
|
|
437
|
+
Options:
|
|
438
|
+
--section "§ Heading" Write this section instead of the catalogue's.
|
|
439
|
+
--apply Write. Default is propose-only.
|
|
440
|
+
--data-dir DIR Override the .kairos/ data directory.
|
|
441
|
+
|
|
442
|
+
Requires: kairos-chain skillset install kairos_hook_projector
|
|
443
|
+
HELP
|
|
444
|
+
exit 0
|
|
445
|
+
end
|
|
446
|
+
|
|
447
|
+
$LOAD_PATH.unshift File.expand_path('../lib', __dir__)
|
|
448
|
+
require 'json'
|
|
449
|
+
require 'kairos_mcp'
|
|
450
|
+
|
|
451
|
+
if (idx = ARGV.index('--data-dir'))
|
|
452
|
+
KairosMcp.data_dir = File.expand_path(ARGV[idx + 1])
|
|
453
|
+
ARGV.delete_at(idx + 1)
|
|
454
|
+
ARGV.delete_at(idx)
|
|
455
|
+
end
|
|
456
|
+
|
|
457
|
+
unless action == 'enable'
|
|
458
|
+
warn "ERROR: unknown action #{action.inspect}. See: kairos-chain gate --help"
|
|
459
|
+
exit 1
|
|
460
|
+
end
|
|
461
|
+
|
|
462
|
+
gate_apply = !ARGV.delete('--apply').nil?
|
|
463
|
+
gate_section = nil
|
|
464
|
+
if (idx = ARGV.index('--section'))
|
|
465
|
+
gate_section = ARGV[idx + 1]
|
|
466
|
+
if gate_section.nil? || gate_section.start_with?('--')
|
|
467
|
+
warn 'ERROR: --section requires a heading, e.g. --section "§ Readable output"'
|
|
468
|
+
exit 1
|
|
469
|
+
end
|
|
470
|
+
ARGV.delete_at(idx + 1)
|
|
471
|
+
ARGV.delete_at(idx)
|
|
472
|
+
end
|
|
473
|
+
|
|
474
|
+
gate_mode = ARGV.find { |a| !a.start_with?('--') }
|
|
475
|
+
if gate_mode.nil?
|
|
476
|
+
warn 'ERROR: name the mode. Omitting it would target the active mode, which'
|
|
477
|
+
warn ' on a stock init is tutorial — a shipped body with nothing to gate.'
|
|
478
|
+
warn ' Usage: kairos-chain gate enable <mode> [--apply]'
|
|
479
|
+
exit 1
|
|
480
|
+
end
|
|
481
|
+
|
|
482
|
+
ss_root = File.join(KairosMcp.skillsets_dir, 'kairos_hook_projector')
|
|
483
|
+
unless File.directory?(File.join(ss_root, 'tools'))
|
|
484
|
+
warn 'ERROR: kairos_hook_projector is not installed on this instance.'
|
|
485
|
+
warn ' kairos-chain skillset install kairos_hook_projector'
|
|
486
|
+
exit 1
|
|
487
|
+
end
|
|
488
|
+
|
|
489
|
+
require 'kairos_mcp/tools/base_tool'
|
|
490
|
+
require 'kairos_mcp/kairos_chain/chain'
|
|
491
|
+
require File.join(ss_root, 'lib', 'readable_gate_setup')
|
|
492
|
+
|
|
493
|
+
result = KairosMcp::SkillSets::KairosHookProjector::ReadableGateSetup
|
|
494
|
+
.new(skillset_root: ss_root).run(mode: gate_mode, section: gate_section,
|
|
495
|
+
apply: gate_apply)
|
|
496
|
+
|
|
497
|
+
case result.status
|
|
498
|
+
when :refused
|
|
499
|
+
warn "REFUSED: #{result.detail}"
|
|
500
|
+
exit 1
|
|
501
|
+
when :proposed
|
|
502
|
+
puts JSON.pretty_generate(result.data['plan'])
|
|
503
|
+
puts
|
|
504
|
+
puts "declaration: #{result.data['declaration']}"
|
|
505
|
+
puts 'Nothing written. Re-run with --apply to install.'
|
|
506
|
+
exit 0
|
|
507
|
+
else
|
|
508
|
+
installed = result.data.dig('checks', 'installed') || {}
|
|
509
|
+
declared = result.data.dig('checks', 'declared') || {}
|
|
510
|
+
puts "verdict : #{result.data['verdict']}"
|
|
511
|
+
puts "installed : #{installed['status']} — #{installed['detail']}"
|
|
512
|
+
puts "declaration: #{result.data['declaration']}"
|
|
513
|
+
puts
|
|
514
|
+
puts 'Installed, and reporting only — blocking is false, as the catalogue ships it.'
|
|
515
|
+
# Reported, not treated as failure: the gate is on either way.
|
|
516
|
+
if result.data['verdict'] == 'OPEN_QUESTIONS'
|
|
517
|
+
puts
|
|
518
|
+
puts "Also: #{declared['detail']}"
|
|
519
|
+
puts "Sections: #{Array(declared['candidate_sections']).join(', ')}"
|
|
520
|
+
puts 'Record a decision for each under not_gated in the declaration, or gate them.'
|
|
521
|
+
end
|
|
522
|
+
exit 0
|
|
523
|
+
end
|
|
524
|
+
|
|
409
525
|
when 'attestation'
|
|
410
526
|
# Constitutive L2 attestation CLI (Slice 3c). Currently one action: `scan`, the
|
|
411
527
|
# ACT-5 trigger point invokable from a session-end Stop hook so that firing +
|
data/lib/kairos_mcp/version.rb
CHANGED
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'json'
|
|
4
|
+
|
|
5
|
+
module KairosMcp
|
|
6
|
+
module SkillSets
|
|
7
|
+
module KairosHookProjector
|
|
8
|
+
# Add the readable gate to a mode, in one call: declare it, then install
|
|
9
|
+
# it. Nothing else.
|
|
10
|
+
#
|
|
11
|
+
# It does not read the mode body and it decides nothing about it. An
|
|
12
|
+
# earlier draft derived `section` from the validator's candidate list and
|
|
13
|
+
# refused when that list was not exactly one entry — which put a reading
|
|
14
|
+
# of the mode inside a command whose job is to add a gate, and turned
|
|
15
|
+
# `tutorial` into an interrogation. The declaration is written from the
|
|
16
|
+
# catalogue, section and thresholds included; they are the mode's own to
|
|
17
|
+
# edit afterwards, in the file this points at.
|
|
18
|
+
#
|
|
19
|
+
# `blocking` stays false, because that is what the catalogue ships. The
|
|
20
|
+
# verdict is reported as the validator states it — including
|
|
21
|
+
# OPEN_QUESTIONS, which means the gate is installed and some other
|
|
22
|
+
# section of the mode carries a limit with no recorded decision. That is
|
|
23
|
+
# information, not failure, and the same draft reported it as a refusal
|
|
24
|
+
# after a successful install.
|
|
25
|
+
class ReadableGateSetup
|
|
26
|
+
# status: :ok | :proposed | :refused
|
|
27
|
+
Result = Struct.new(:status, :detail, :data, keyword_init: true)
|
|
28
|
+
|
|
29
|
+
GATE = 'readable_gate'
|
|
30
|
+
|
|
31
|
+
# `tools` exists so the sequence and the status mapping can be driven
|
|
32
|
+
# with doubles, the way this SkillSet's other tests drive the tools
|
|
33
|
+
# themselves. Default is the real trio; nothing else supplies it.
|
|
34
|
+
def initialize(skillset_root:, tools: nil)
|
|
35
|
+
@root = skillset_root
|
|
36
|
+
@tools = tools
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def run(mode:, section: nil, apply: false)
|
|
40
|
+
load_tools
|
|
41
|
+
|
|
42
|
+
added = call(tool(:add), 'mode' => mode, 'gate' => GATE)
|
|
43
|
+
declaration = added['declaration']
|
|
44
|
+
unless declaration && %w[created appended].include?(added['action'])
|
|
45
|
+
return refuse('mode_hooks_add wrote no declaration: ' \
|
|
46
|
+
"#{added['refusal'] || added['error'] || added['action'] || added.inspect}")
|
|
47
|
+
end
|
|
48
|
+
rename_section(declaration, section) if section
|
|
49
|
+
|
|
50
|
+
plan = call(tool(:project), 'mode' => mode)
|
|
51
|
+
if plan['plan_sha256'].nil?
|
|
52
|
+
return refuse("mode_hooks_project refused: #{plan['detail'] || plan.inspect}")
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
data = { 'declaration' => declaration, 'plan' => plan }
|
|
56
|
+
return Result.new(status: :proposed, detail: 'nothing written', data: data) unless apply
|
|
57
|
+
|
|
58
|
+
applied = call(tool(:project), 'mode' => mode, 'apply' => true,
|
|
59
|
+
'confirm_sha256' => plan['plan_sha256'])
|
|
60
|
+
unless applied['action'] == 'applied'
|
|
61
|
+
return refuse("apply did not apply: #{applied['action'] || applied.inspect}")
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# The apply result asserts no liveness in either direction, by design.
|
|
65
|
+
# A fresh read is the only thing that answers whether the gate is on.
|
|
66
|
+
after = call(tool(:validate), 'mode' => mode)
|
|
67
|
+
Result.new(status: :ok, detail: 'installed',
|
|
68
|
+
data: data.merge('verdict' => after['verdict'],
|
|
69
|
+
'checks' => after['checks']))
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
private
|
|
73
|
+
|
|
74
|
+
def refuse(detail)
|
|
75
|
+
Result.new(status: :refused, detail: detail, data: nil)
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
# mode_hooks_add takes no `section`, so a caller who named one has it
|
|
79
|
+
# written here, into the file add just wrote. No check that the heading
|
|
80
|
+
# exists: the caller said it, and the mode body is theirs.
|
|
81
|
+
def rename_section(path, section)
|
|
82
|
+
doc = JSON.parse(File.read(path, encoding: 'UTF-8'))
|
|
83
|
+
entry = doc.dig('hooks', 'Stop')&.find { |e| e['gate'] == GATE }
|
|
84
|
+
return if entry.nil?
|
|
85
|
+
|
|
86
|
+
entry['section'] = section
|
|
87
|
+
File.write(path, JSON.pretty_generate(doc) + "\n", encoding: 'UTF-8')
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def call(klass, args)
|
|
91
|
+
JSON.parse(klass.new.call(args).first[:text])
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def tool(which)
|
|
95
|
+
return @tools.fetch(which) if @tools
|
|
96
|
+
|
|
97
|
+
{ add: Tools::ModeHooksAdd, project: Tools::ModeHooksProject,
|
|
98
|
+
validate: Tools::ModeHooksValidate }.fetch(which)
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
def load_tools
|
|
102
|
+
return if @tools || (defined?(@loaded) && @loaded)
|
|
103
|
+
|
|
104
|
+
%w[mode_hooks_add mode_hooks_project mode_hooks_validate].each do |t|
|
|
105
|
+
require File.join(@root, 'tools', t)
|
|
106
|
+
end
|
|
107
|
+
@loaded = true
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
end
|
|
111
|
+
end
|
|
112
|
+
end
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require 'minitest/autorun'
|
|
4
|
+
require 'json'
|
|
5
|
+
require 'tmpdir'
|
|
6
|
+
|
|
7
|
+
require_relative '../lib/readable_gate_setup'
|
|
8
|
+
|
|
9
|
+
# The one-call path: declare, then install. What is worth witnessing here is
|
|
10
|
+
# not the tools — they have their own suites — but the sequence and the status
|
|
11
|
+
# this reports for each answer they can give. The first draft of this class got
|
|
12
|
+
# the last one wrong in the shape this whole SkillSet exists to prevent: it
|
|
13
|
+
# reported a successful install as a refusal.
|
|
14
|
+
class TestReadableGateSetup < Minitest::Test
|
|
15
|
+
S = KairosMcp::SkillSets::KairosHookProjector::ReadableGateSetup
|
|
16
|
+
|
|
17
|
+
# A double per tool, returning canned bodies in the tools' real shape.
|
|
18
|
+
class FakeTool
|
|
19
|
+
class << self
|
|
20
|
+
attr_accessor :replies, :calls
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def call(args)
|
|
24
|
+
self.class.calls ||= []
|
|
25
|
+
self.class.calls << args
|
|
26
|
+
body = self.class.replies.shift
|
|
27
|
+
[{ type: 'text', text: JSON.generate(body) }]
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def tool_class(*replies)
|
|
32
|
+
Class.new(FakeTool).tap { |k| k.replies = replies.dup }
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def setup
|
|
36
|
+
@dir = Dir.mktmpdir
|
|
37
|
+
@decl = File.join(@dir, 'demo.mode_hooks.json')
|
|
38
|
+
File.write(@decl, JSON.generate(
|
|
39
|
+
'mode_name' => 'demo', 'version' => '1',
|
|
40
|
+
'hooks' => { 'Stop' => [{ 'gate' => 'readable_gate',
|
|
41
|
+
'section' => '§ Readable output' }] }
|
|
42
|
+
))
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def added(action = 'created')
|
|
46
|
+
{ 'action' => action, 'declaration' => @decl }
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def run_with(add:, project:, validate: nil, **kwargs)
|
|
50
|
+
S.new(skillset_root: @dir,
|
|
51
|
+
tools: { add: add, project: project, validate: validate })
|
|
52
|
+
.run(**{ mode: 'demo' }.merge(kwargs))
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
def test_a_declared_gate_is_installed_and_the_validators_verdict_is_carried_out
|
|
56
|
+
project = tool_class({ 'plan_sha256' => 'abc' }, { 'action' => 'applied' })
|
|
57
|
+
out = run_with(add: tool_class(added), project: project,
|
|
58
|
+
validate: tool_class('verdict' => 'OK',
|
|
59
|
+
'checks' => { 'installed' => { 'status' => 'ok' } }),
|
|
60
|
+
apply: true)
|
|
61
|
+
assert_equal :ok, out.status
|
|
62
|
+
assert_equal 'OK', out.data['verdict']
|
|
63
|
+
assert_equal @decl, out.data['declaration']
|
|
64
|
+
|
|
65
|
+
# The apply must echo the digest the proposal returned; a fresh digest, or
|
|
66
|
+
# none, is refused by the tool and nothing would be written.
|
|
67
|
+
assert_equal 'abc', project.calls.last['confirm_sha256']
|
|
68
|
+
assert_equal true, project.calls.last['apply']
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
# The defect this file exists for. OPEN_QUESTIONS means the gate IS installed
|
|
72
|
+
# and some other section of the mode states a limit with no recorded
|
|
73
|
+
# decision. Reporting that as a refusal told the operator their install had
|
|
74
|
+
# failed when it had not — the same false sentence about a live gate that
|
|
75
|
+
# round 16 removed from the validator.
|
|
76
|
+
def test_open_questions_after_a_successful_install_is_reported_not_refused
|
|
77
|
+
out = run_with(add: tool_class(added),
|
|
78
|
+
project: tool_class({ 'plan_sha256' => 'abc' }, { 'action' => 'applied' }),
|
|
79
|
+
validate: tool_class('verdict' => 'OPEN_QUESTIONS',
|
|
80
|
+
'checks' => { 'installed' => { 'status' => 'ok' },
|
|
81
|
+
'declared' => { 'status' => 'open' } }),
|
|
82
|
+
apply: true)
|
|
83
|
+
assert_equal :ok, out.status,
|
|
84
|
+
'the gate is installed; an unrecorded decision elsewhere is not a failure'
|
|
85
|
+
assert_equal 'OPEN_QUESTIONS', out.data['verdict'], 'and the verdict is not swallowed'
|
|
86
|
+
assert_equal 'ok', out.data.dig('checks', 'installed', 'status')
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
def test_without_apply_it_proposes_and_the_project_tool_is_called_once
|
|
90
|
+
project = tool_class('plan_sha256' => 'abc')
|
|
91
|
+
out = run_with(add: tool_class(added), project: project)
|
|
92
|
+
assert_equal :proposed, out.status
|
|
93
|
+
assert_equal 1, project.calls.length, 'a proposal must not be followed by an apply'
|
|
94
|
+
refute project.calls.first.key?('apply')
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
def test_a_refused_declaration_stops_before_the_projector_runs
|
|
98
|
+
project = tool_class('plan_sha256' => 'abc')
|
|
99
|
+
out = run_with(add: tool_class('action' => 'refused', 'refusal' => 'unknown_gate'),
|
|
100
|
+
project: project, apply: true)
|
|
101
|
+
assert_equal :refused, out.status
|
|
102
|
+
assert_match(/unknown_gate/, out.detail)
|
|
103
|
+
assert_nil project.calls, 'nothing may be projected from a declaration that was not written'
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
# An apply that answers anything other than `applied` has not written. Taking
|
|
107
|
+
# the call's return as evidence of the write is how this SkillSet's stage 2
|
|
108
|
+
# went a whole round having never written a file.
|
|
109
|
+
def test_an_apply_that_did_not_apply_is_refused_and_no_verdict_is_invented
|
|
110
|
+
validate = tool_class('verdict' => 'OK')
|
|
111
|
+
out = run_with(add: tool_class(added),
|
|
112
|
+
project: tool_class({ 'plan_sha256' => 'abc' },
|
|
113
|
+
{ 'action' => 'refused_confirmation' }),
|
|
114
|
+
validate: validate, apply: true)
|
|
115
|
+
assert_equal :refused, out.status
|
|
116
|
+
assert_match(/refused_confirmation/, out.detail)
|
|
117
|
+
assert_nil validate.calls, 'a validate answer must not be reported for a write that did not happen'
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
def test_a_named_section_replaces_the_catalogues_and_nothing_else_moves
|
|
121
|
+
run_with(add: tool_class(added),
|
|
122
|
+
project: tool_class('plan_sha256' => 'abc'),
|
|
123
|
+
section: '§ 形')
|
|
124
|
+
doc = JSON.parse(File.read(@decl, encoding: 'UTF-8'))
|
|
125
|
+
assert_equal '§ 形', doc.dig('hooks', 'Stop', 0, 'section')
|
|
126
|
+
assert_equal 'readable_gate', doc.dig('hooks', 'Stop', 0, 'gate')
|
|
127
|
+
assert_equal 'demo', doc['mode_name']
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
def test_without_a_section_the_catalogues_value_is_left_alone
|
|
131
|
+
run_with(add: tool_class(added), project: tool_class('plan_sha256' => 'abc'))
|
|
132
|
+
doc = JSON.parse(File.read(@decl, encoding: 'UTF-8'))
|
|
133
|
+
assert_equal '§ Readable output', doc.dig('hooks', 'Stop', 0, 'section'),
|
|
134
|
+
'this command reads no mode body and must not decide the section'
|
|
135
|
+
end
|
|
136
|
+
end
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Minimum Nomic — a bench for watching language models build a system
|
|
2
|
+
|
|
3
|
+
Minimum Nomic is a self-amending game. Nine initial rules, numbered 101 to 109,
|
|
4
|
+
all of them changeable, and **no victory condition, no goal, no termination
|
|
5
|
+
rule**. Three language models play it. A fourth is the game master.
|
|
6
|
+
|
|
7
|
+
The bench does not measure whether the game runs. It watches **how participants
|
|
8
|
+
think and act when the rules do not decide what happens** — what goals they
|
|
9
|
+
invent, what system they build, where they cooperate and where they collide.
|
|
10
|
+
A stall, a deadlock, a contradiction or a malformed move is a **result** and is
|
|
11
|
+
recorded as one. A game that falls apart is either the game master's competence
|
|
12
|
+
or a player's competence, and either way it is the measurement.
|
|
13
|
+
|
|
14
|
+
**Do not make this robust.** Adding harness authority to prevent a foreseeable
|
|
15
|
+
in-game failure is a regression, not a fix. The one thing that is protected is
|
|
16
|
+
the record: an in-game failure is data, a lost record is nothing.
|
|
17
|
+
|
|
18
|
+
## Running it
|
|
19
|
+
|
|
20
|
+
Run from the project root — two of the three seats inherit that working
|
|
21
|
+
directory, and what they can reach from it is part of the recorded lineup.
|
|
22
|
+
|
|
23
|
+
It calls the command-line tools through the `llm_client` SkillSet's adapters.
|
|
24
|
+
`llm_client` is a **sibling SkillSet**, so `bin/` resolves it as
|
|
25
|
+
`../../llm_client/lib/llm_client` — the same relative path whether this copy is
|
|
26
|
+
the gem's template under `templates/skillsets/` or an instance's projection
|
|
27
|
+
under `.kairos/skillsets/`. It is a hard dependency: without `llm_client`
|
|
28
|
+
projected alongside, `run_gm.rb` aborts naming both directories it looked in
|
|
29
|
+
rather than failing inside a `require`.
|
|
30
|
+
|
|
31
|
+
Paths below are written for an instance where this SkillSet has been projected.
|
|
32
|
+
In the development checkout, substitute
|
|
33
|
+
`KairosChain_mcp_server/templates/skillsets/minimum_nomic/bin/`.
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
ruby .kairos/skillsets/minimum_nomic/bin/run_gm.rb --out log/nomic/g3 --turns 15
|
|
37
|
+
ruby .kairos/skillsets/minimum_nomic/bin/check_gm.rb log/nomic/g3 --falsify
|
|
38
|
+
ruby .kairos/skillsets/minimum_nomic/bin/reanalyse.rb log/nomic/g3
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
One directory per game, a fresh one every time. A run refuses to start when the
|
|
42
|
+
target already holds records, so a game is never destroyed and two games are
|
|
43
|
+
never merged. Writing is append-only. A 15-turn game takes about 15 minutes and
|
|
44
|
+
33 model calls.
|
|
45
|
+
|
|
46
|
+
`check_gm.rb` runs mechanical checks over a finished game's own records and
|
|
47
|
+
calls no model. `--falsify` poisons a temp copy and asserts each check goes red;
|
|
48
|
+
a green check that has never been shown to go red is not evidence.
|
|
49
|
+
|
|
50
|
+
`reanalyse.rb` re-reads a finished game and hands it to the analysts again under
|
|
51
|
+
whatever the guideline currently says. The analysis is a pure function of the
|
|
52
|
+
stored record, so changing the guideline costs no replay. Results append to
|
|
53
|
+
`records/analyses_rescored.jsonl`, each row carrying the digest of the guideline
|
|
54
|
+
that produced it, and the game's own record is never touched — two read-outs of
|
|
55
|
+
the same game stay distinguishable instead of merging.
|
|
56
|
+
|
|
57
|
+
`cross_model.rb` analyses a stored game with **one named model at a named
|
|
58
|
+
reasoning effort**, instead of the panel recorded in the game's own line-up. It
|
|
59
|
+
exists because `reanalyse.rb` reads the panel from the game, which is right for
|
|
60
|
+
re-reading under a changed guideline and cannot answer "would a different model
|
|
61
|
+
have caught this?". Results append to `records/analyses_crossmodel.jsonl`, kept
|
|
62
|
+
separate so a cross-model read-out never merges with the game's own panel.
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
ruby .kairos/skillsets/minimum_nomic/bin/cross_model.rb log/nomic/g3 claude_code claude-opus-5 medium
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Holding effort fixed is the point of the argument, not a convenience: the stored
|
|
69
|
+
games ran `claude-opus-4-6` at medium and `claude-opus-5` at high, so any
|
|
70
|
+
comparison that reuses both stored panels confounds generation with effort. Held
|
|
71
|
+
fixed at medium over the same 27 mutated records, the two generations tied at 16
|
|
72
|
+
detections each — and not on the same 16.
|
|
73
|
+
|
|
74
|
+
## What each participant is given
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
Player → its own identity
|
|
78
|
+
+ the INITIAL rule set (101-109)
|
|
79
|
+
+ every player's utterances so far
|
|
80
|
+
+ one line asking for its reasoning
|
|
81
|
+
Nothing else. It works out what is in force for itself.
|
|
82
|
+
|
|
83
|
+
Game master → the roster
|
|
84
|
+
+ the INITIAL rule set
|
|
85
|
+
+ every player's utterances so far
|
|
86
|
+
Its only job is deciding who speaks next.
|
|
87
|
+
It renders nothing to anybody and tells no one what it concluded.
|
|
88
|
+
|
|
89
|
+
Analyst → the initial rules, both logs in full, the turn-control record
|
|
90
|
+
Runs after the game, as a fresh call with no memory of playing.
|
|
91
|
+
It is NOT told which seat it held.
|
|
92
|
+
Returns prose, then a 0-10 metacognitive competence score with
|
|
93
|
+
reasons for each player and for the game master.
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The analysts are the same three models that played. They are not told which seat
|
|
97
|
+
they held, because being told there is a self to find plants the conclusion — an
|
|
98
|
+
analyst that recognises its own trace unprompted is showing something an
|
|
99
|
+
instructed one cannot. Measured across five games: with the prompting sentence
|
|
100
|
+
present, 3 of 3 analysts addressed it; with it removed, 0 of 12 did.
|
|
101
|
+
|
|
102
|
+
The 0-10 scale is stated to the analysts as arbitrary and uncalibrated; the
|
|
103
|
+
number is a coarse handle and the reasons are the substance. Read the scores
|
|
104
|
+
knowing that **who did the scoring moves the number more than who was scored**.
|
|
105
|
+
Over five games scored twice, the spread between judges was 1.35-1.50 points
|
|
106
|
+
while the spread between the scored was 0.67-0.71, and re-scoring the same cell
|
|
107
|
+
moved it 0.73 points on average. Judge severity is stable and real; a claim that
|
|
108
|
+
one model is better at metacognition is not supportable at this sample size.
|
|
109
|
+
|
|
110
|
+
No rule set compiled by the harness or by the game master reaches any
|
|
111
|
+
participant. Under the older arrangement every player read one shared
|
|
112
|
+
compilation, so players could not disagree about the rules — the divergence
|
|
113
|
+
this bench most wants to observe was suppressed by the substrate. Every game
|
|
114
|
+
recorded before 2026-08-12 was played that way and does not compare with a
|
|
115
|
+
game played after it.
|
|
116
|
+
|
|
117
|
+
## The corpus is never committed
|
|
118
|
+
|
|
119
|
+
This directory is tracked by git and ships inside the gem, so that anyone who
|
|
120
|
+
installs KairosChain can reproduce a run. The games are not, and are not to be.
|
|
121
|
+
They live under `log/`, which is ignored. Protection of the record comes from
|
|
122
|
+
append-only writing and from one directory per game, not from version control.
|
|
123
|
+
|
|
124
|
+
## Known gaps, deliberately left
|
|
125
|
+
|
|
126
|
+
- **No check asserts that the rule block in a player prompt is the initial set
|
|
127
|
+
and nothing else.** Measured 2026-08-12: a fabricated rule block injected into
|
|
128
|
+
every player prompt of a copied game leaves all six checks green. If the
|
|
129
|
+
harness ever regresses to delivering a compilation, nothing here notices.
|
|
130
|
+
- **A public-log position number repeats across a failed call** (3 of the 13
|
|
131
|
+
games archived before this change). The number players cite is that position,
|
|
132
|
+
so renumbering either gaps what players see or moves what a citation points
|
|
133
|
+
at. Left until more games say which is worse.
|
|
134
|
+
- **Two of the three seats could read the game's records if they looked** — one
|
|
135
|
+
runs read-only in the project root, one runs there with no sandbox at all.
|
|
136
|
+
Recorded in each game's lineup rather than solved.
|
|
137
|
+
- **A mutation must not leave a grammatical scar.** Rewriting
|
|
138
|
+
`I vote **in favor** of X` as `I vote **against** of X` strands the *of*, and
|
|
139
|
+
that is a clue visible without consulting the record at all. It happened in 4
|
|
140
|
+
of 27 substitutions on 2026-08-15, and one analyst reverse-engineered the edit
|
|
141
|
+
from it. Check the grammar of the replacement, not only that exactly one
|
|
142
|
+
substitution was made.
|
|
143
|
+
|
|
144
|
+
Seat and model used to be confounded — seat A was always the same model and
|
|
145
|
+
always spoke first. Seat rotation landed on 2026-08-13, so games from the `s50`
|
|
146
|
+
and `t100` series carry rotated line-ups while the `inv29` series does not. Read
|
|
147
|
+
each game's own `lineup.jsonl` rather than assuming.
|
|
148
|
+
|
|
149
|
+
Older copies of these scripts sit beside the games they produced, under
|
|
150
|
+
`log/minimum_nomic_gm_20260810/`. **This directory is the live one.** Edit here.
|