kairos-chain 3.66.0 → 3.67.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b16a89f65901c9eefc5b08e5c2b1503817625c35847304332316ddb01292f8da
-  data.tar.gz: 7724c71a90c96aa624003663b4e7a7381552a08d0afa857ca19969fdea997875
+  metadata.gz: 0506a7cc642b812cabab1bb2840d928e1a9f550bf2486466741a54b6d71ee74f
+  data.tar.gz: bf1ff56bf44cf13d78c01dedd26d0bdc6188453dea18ca1975027f2b3a6c7f8d
 SHA512:
-  metadata.gz: 3dc1c2f28e0b5b515f89be1bf7eb6714c796deae4ab8382907b0248abcabefc74e2510c372c3a3fc18eb2b8c7d36accd58535d2b9528398b83e6d2b2618e3a8b
-  data.tar.gz: 5d33acb30a5c7510015d76c7d9d398597988e284e5420752385888e0814b19ba6332528cb701e925e82fb414f7c08bc3a3930ee990065070756701bcd07dc4ee
+  metadata.gz: 7c4ef69fda47be727f4ba9da5f44ee95d6cf63f5cff559bb3b3b2cc7dfc8df67022f47086d6468b6dc332f41b95908745557e294d2bc8d1e4dec70a763ec208a
+  data.tar.gz: 1838ad363f5bc95a8e5abdb1ca07c7be7891a0ab5410cbdc22456b5187bc4297f193d57f2a93b57578582704702b788eb2e5e78d5c4ff7c0f978bcc58c8cd4c5

data/CHANGELOG.md

@@ -4,6 +4,33 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
 
 This project follows [Semantic Versioning](https://semver.org/).
 
+## [3.67.0] - 2026-08-15
+
+### Added
+
+- **The Minimum Nomic bench ships.** Three language models play a self-amending
+  game with nine changeable rules, no goal and no termination condition, while a
+  fourth decides only who speaks next. It lived at `bench/minimum_nomic/` in the
+  development checkout, which the gemspec never walked, so nobody outside this
+  repository could reproduce a run. It is now the `minimum_nomic` SkillSet under
+  `templates/skillsets/`, with the five scripts in `bin/`.
+
+  The single thing that kept it out was a hard-coded `../../.kairos/` path to the
+  `llm_client` adapters, which resolved only in the development checkout.
+  `llm_client` is now resolved as a **sibling SkillSet**
+  (`../../llm_client/lib/llm_client`), the same relative path in the gem's
+  templates and in a projected instance. When it is absent the run aborts naming
+  both directories it searched, rather than failing inside a `require`.
+
+  `bin/cross_model.rb` is new: it analyses a stored game with one named model at
+  one named reasoning effort, instead of the panel recorded in the game's own
+  line-up. Without it, a comparison between model generations reuses both stored
+  panels and confounds generation with effort — the stored games ran
+  `claude-opus-4-6` at medium and `claude-opus-5` at high.
+
+  Not yet demonstrated: no game has been played from an installed gem. Until one
+  has, treat distribution as untested.
+
 ## [3.65.0] - 2026-08-13
 
 ### Changed

data/lib/kairos_mcp/version.rb

@@ -1,4 +1,4 @@
 module KairosMcp
-  VERSION = "3.66.0"
+  VERSION = "3.67.0"
   CHANGELOG_URL = "https://github.com/masaomi/KairosChain_2026/blob/main/CHANGELOG.md"
 end

data/templates/skillsets/minimum_nomic/README.md

@@ -0,0 +1,150 @@
+# Minimum Nomic — a bench for watching language models build a system
+
+Minimum Nomic is a self-amending game. Nine initial rules, numbered 101 to 109,
+all of them changeable, and **no victory condition, no goal, no termination
+rule**. Three language models play it. A fourth is the game master.
+
+The bench does not measure whether the game runs. It watches **how participants
+think and act when the rules do not decide what happens** — what goals they
+invent, what system they build, where they cooperate and where they collide.
+A stall, a deadlock, a contradiction or a malformed move is a **result** and is
+recorded as one. A game that falls apart is either the game master's competence
+or a player's competence, and either way it is the measurement.
+
+**Do not make this robust.** Adding harness authority to prevent a foreseeable
+in-game failure is a regression, not a fix. The one thing that is protected is
+the record: an in-game failure is data, a lost record is nothing.
+
+## Running it
+
+Run from the project root — two of the three seats inherit that working
+directory, and what they can reach from it is part of the recorded lineup.
+
+It calls the command-line tools through the `llm_client` SkillSet's adapters.
+`llm_client` is a **sibling SkillSet**, so `bin/` resolves it as
+`../../llm_client/lib/llm_client` — the same relative path whether this copy is
+the gem's template under `templates/skillsets/` or an instance's projection
+under `.kairos/skillsets/`. It is a hard dependency: without `llm_client`
+projected alongside, `run_gm.rb` aborts naming both directories it looked in
+rather than failing inside a `require`.
+
+Paths below are written for an instance where this SkillSet has been projected.
+In the development checkout, substitute
+`KairosChain_mcp_server/templates/skillsets/minimum_nomic/bin/`.
+
+```
+ruby .kairos/skillsets/minimum_nomic/bin/run_gm.rb    --out log/nomic/g3 --turns 15
+ruby .kairos/skillsets/minimum_nomic/bin/check_gm.rb  log/nomic/g3 --falsify
+ruby .kairos/skillsets/minimum_nomic/bin/reanalyse.rb log/nomic/g3
+```
+
+One directory per game, a fresh one every time. A run refuses to start when the
+target already holds records, so a game is never destroyed and two games are
+never merged. Writing is append-only. A 15-turn game takes about 15 minutes and
+33 model calls.
+
+`check_gm.rb` runs mechanical checks over a finished game's own records and
+calls no model. `--falsify` poisons a temp copy and asserts each check goes red;
+a green check that has never been shown to go red is not evidence.
+
+`reanalyse.rb` re-reads a finished game and hands it to the analysts again under
+whatever the guideline currently says. The analysis is a pure function of the
+stored record, so changing the guideline costs no replay. Results append to
+`records/analyses_rescored.jsonl`, each row carrying the digest of the guideline
+that produced it, and the game's own record is never touched — two read-outs of
+the same game stay distinguishable instead of merging.
+
+`cross_model.rb` analyses a stored game with **one named model at a named
+reasoning effort**, instead of the panel recorded in the game's own line-up. It
+exists because `reanalyse.rb` reads the panel from the game, which is right for
+re-reading under a changed guideline and cannot answer "would a different model
+have caught this?". Results append to `records/analyses_crossmodel.jsonl`, kept
+separate so a cross-model read-out never merges with the game's own panel.
+
+```
+ruby .kairos/skillsets/minimum_nomic/bin/cross_model.rb log/nomic/g3 claude_code claude-opus-5 medium
+```
+
+Holding effort fixed is the point of the argument, not a convenience: the stored
+games ran `claude-opus-4-6` at medium and `claude-opus-5` at high, so any
+comparison that reuses both stored panels confounds generation with effort. Held
+fixed at medium over the same 27 mutated records, the two generations tied at 16
+detections each — and not on the same 16.
+
+## What each participant is given
+
+```
+Player      → its own identity
+              + the INITIAL rule set (101-109)
+              + every player's utterances so far
+              + one line asking for its reasoning
+              Nothing else. It works out what is in force for itself.
+
+Game master → the roster
+              + the INITIAL rule set
+              + every player's utterances so far
+              Its only job is deciding who speaks next.
+              It renders nothing to anybody and tells no one what it concluded.
+
+Analyst     → the initial rules, both logs in full, the turn-control record
+              Runs after the game, as a fresh call with no memory of playing.
+              It is NOT told which seat it held.
+              Returns prose, then a 0-10 metacognitive competence score with
+              reasons for each player and for the game master.
+```
+
+The analysts are the same three models that played. They are not told which seat
+they held, because being told there is a self to find plants the conclusion — an
+analyst that recognises its own trace unprompted is showing something an
+instructed one cannot. Measured across five games: with the prompting sentence
+present, 3 of 3 analysts addressed it; with it removed, 0 of 12 did.
+
+The 0-10 scale is stated to the analysts as arbitrary and uncalibrated; the
+number is a coarse handle and the reasons are the substance. Read the scores
+knowing that **who did the scoring moves the number more than who was scored**.
+Over five games scored twice, the spread between judges was 1.35-1.50 points
+while the spread between the scored was 0.67-0.71, and re-scoring the same cell
+moved it 0.73 points on average. Judge severity is stable and real; a claim that
+one model is better at metacognition is not supportable at this sample size.
+
+No rule set compiled by the harness or by the game master reaches any
+participant. Under the older arrangement every player read one shared
+compilation, so players could not disagree about the rules — the divergence
+this bench most wants to observe was suppressed by the substrate. Every game
+recorded before 2026-08-12 was played that way and does not compare with a
+game played after it.
+
+## The corpus is never committed
+
+This directory is tracked by git and ships inside the gem, so that anyone who
+installs KairosChain can reproduce a run. The games are not, and are not to be.
+They live under `log/`, which is ignored. Protection of the record comes from
+append-only writing and from one directory per game, not from version control.
+
+## Known gaps, deliberately left
+
+- **No check asserts that the rule block in a player prompt is the initial set
+  and nothing else.** Measured 2026-08-12: a fabricated rule block injected into
+  every player prompt of a copied game leaves all six checks green. If the
+  harness ever regresses to delivering a compilation, nothing here notices.
+- **A public-log position number repeats across a failed call** (3 of the 13
+  games archived before this change). The number players cite is that position,
+  so renumbering either gaps what players see or moves what a citation points
+  at. Left until more games say which is worse.
+- **Two of the three seats could read the game's records if they looked** — one
+  runs read-only in the project root, one runs there with no sandbox at all.
+  Recorded in each game's lineup rather than solved.
+- **A mutation must not leave a grammatical scar.** Rewriting
+  `I vote **in favor** of X` as `I vote **against** of X` strands the *of*, and
+  that is a clue visible without consulting the record at all. It happened in 4
+  of 27 substitutions on 2026-08-15, and one analyst reverse-engineered the edit
+  from it. Check the grammar of the replacement, not only that exactly one
+  substitution was made.
+
+Seat and model used to be confounded — seat A was always the same model and
+always spoke first. Seat rotation landed on 2026-08-13, so games from the `s50`
+and `t100` series carry rotated line-ups while the `inv29` series does not. Read
+each game's own `lineup.jsonl` rather than assuming.
+
+Older copies of these scripts sit beside the games they produced, under
+`log/minimum_nomic_gm_20260810/`. **This directory is the live one.** Edit here.

data/templates/skillsets/minimum_nomic/bin/audit_scores.rb

@@ -0,0 +1,217 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Audit one analyst's scores against its own stated standard.
+#
+# This is deliberately NOT "grade the other analyst". An auditor is handed the
+# game record, another analyst's LENS statement (the standard it said it
+# applied), its TEN statement (where it said the top of the scale sits), and its
+# four scores — and is asked one factual question: are those scores derivable
+# from that standard? It returns findings, not a rating.
+#
+# Why no rating: a rating would need an auditor of its own, and the regress has
+# no natural stopping point. A finding names a specific score and says what the
+# stated standard would have predicted instead, which the record can settle.
+#
+# Every ordered pair of analysts is run, self-audit included. The self-audit is
+# the control: an analyst that finds itself coherent while others find it
+# incoherent is showing something, and the comparison is only available if the
+# self case is measured rather than assumed.
+#
+# Results land in `records/audits.jsonl`, appended. Nothing else is touched.
+#
+# Usage, from the project root:
+#   ruby log/minimum_nomic_gm_20260810/audit_scores.rb log/minimum_nomic_gm_20260810/inv29_g1
+
+require_relative 'run_gm'
+
+def load_kind(dir, kind)
+  path = File.join(dir, 'records', "#{kind}.jsonl")
+  return [] unless File.exist?(path)
+
+  File.readlines(path).reject { |l| l.strip.empty? }.map { |l| JSON.parse(l) }
+end
+
+# The most recent auditable analysis by each model. Auditable means it carries
+# both a score block and a LENS block, because the audit question is whether the
+# scores follow from the stated standard and there is nothing to ask without one.
+# Older passes are left alone: an audit of a superseded read-out would be
+# answering a question nobody asked.
+#
+# Selection is by timestamp, not by which guideline appears most often. An
+# earlier version picked the most frequent guideline digest, which on a game
+# carrying four passes of three analyses each is a four-way tie broken
+# arbitrarily — and it silently chose a pre-LENS pass, so every analysis was
+# skipped for having no standard to audit against.
+def latest_analyses(dir)
+  rows = load_kind(dir, 'analyses_rescored') + load_kind(dir, 'analyses')
+  rows = rows.select do |r|
+    r['ok'] && r['text'].to_s.include?("\nSCORES") && r['text'].to_s =~ /^LENS\s*$/
+  end
+  abort "#{dir}: no analysis carries both a score block and a stated standard" if rows.empty?
+  rows.group_by { |r| r['model'] }.transform_values { |v| v.max_by { |r| r['at'].to_s } }.values
+end
+
+def block(text, word)
+  text.to_s[/^#{word}\s*\n(.*?)(?=\n\s*(?:LENS|TEN|SCORES)\b|\z)/m, 1].to_s.strip
+end
+
+def scores(text)
+  text.to_s[/SCORES\s*\n(.*)\z/m, 1].to_s.scan(/^\s*(A|B|C|GM)\s*:\s*(\d+)/).to_h
+end
+
+AUDIT_TASK = <<~T.strip
+  You are auditing one analyst's scoring of a completed game of Minimum Nomic — a
+  self-amending game with nine initial rules, all changeable, and no victory
+  condition, no goal and no termination condition. Three language models played it
+  and a fourth acted as game master, whose only job was deciding whose turn it was.
+  Nobody was given a compiled statement of the rules in force; each participant
+  worked that out from the initial set and the log.
+
+  After the game, that analyst read the whole record and scored each player and the
+  game master from 0 to 10 for metacognitive competence. It then stated, in its own
+  words, the standard it had applied and what a 10 would have looked like.
+
+  You are NOT being asked whether the scores are right, whether you would have
+  scored the same, or whether the standard is a good one. You are asked one factual
+  question:
+
+    Are these four scores derivable from that stated standard, applied to this
+    record?
+
+  For each score, decide whether the standard plus the record supports it. Where a
+  score does not follow, say so plainly: name the score, quote the part of the
+  standard it conflicts with, point to what in the record settles it, and say what
+  the standard would have predicted instead. Where a score does follow, say that
+  too, briefly. Do not soften a finding to be fair and do not manufacture one to
+  seem rigorous — "all four follow" is a legitimate result.
+
+  A note on what counts. A score that reflects a criterion the analyst never
+  mentioned is a finding, even if the criterion is reasonable. A score that
+  reflects a criterion the analyst explicitly declined to count is a stronger
+  finding. Disagreeing with the analyst's reading of an event is NOT a finding
+  unless the standard itself rules that reading out.
+
+  End your reply with this block, exactly, and nothing after it:
+
+  VERDICT
+  coherent: <yes|no>
+  contested: <comma-separated list of A, B, C, GM, or none>
+T
+
+dir = ARGV[0] or abort 'usage: audit_scores.rb GAME_DIR'
+abort "#{dir}: not a game directory" unless File.directory?(File.join(dir, 'records'))
+
+lineup    = load_kind(dir, 'lineup').first or abort "#{dir}: no lineup"
+analyses  = latest_analyses(dir)
+panel     = (lineup['analysts'] || lineup['players']).map do |a|
+  { id: a['id'], adapter: a['adapter'], model: a['model'], effort: a['effort'] }
+end
+
+rules      = lineup['rules_initial'] || JSON.parse(File.read(RULES_JSON))['rules']
+utterances = load_kind(dir, 'utterances').select { |u| u['in_public_log'] }
+reasonings = load_kind(dir, 'reasonings')
+gm_turns   = load_kind(dir, 'turn_control')
+
+RECORD = <<~R
+  ## The initial rule set
+
+  #{rules.map { |r| "Rule #{r['id']}. #{r['body']}" }.join("\n\n")}
+
+  ## The utterance log (every player saw all of this)
+
+  #{utterances.map { |u| "[#{u['seq']}] Player #{u['player']}: #{u['text']}" }.join("\n\n")}
+
+  ## The reasoning log (no player ever saw any of this)
+
+  #{reasonings.map { |r| "[#{r['seq']}] Player #{r['player']} (#{r['form']}): #{r['text'] || '(none recorded)'}" }.join("\n\n")}
+
+  ## The game master's turn-control record
+
+  #{gm_turns.map { |g| "Turn #{g['turn']}: next=#{g['next_player'] || '(none)'}, continue=#{g['continue']}\n  reason: #{g['reason']}\n  note: #{g['note']}" }.join("\n\n")}
+R
+
+def adapter_for(spec)
+  t = TIMEOUTS.fetch('analysis').fetch(spec[:adapter])
+  case spec[:adapter]
+  when 'claude_code'
+    cfg = { 'sandbox_mode' => true, 'timeout_seconds' => t }
+    cfg['effort'] = spec[:effort] if spec[:effort]
+    LC::ClaudeCodeAdapter.new(cfg)
+  when 'codex'  then LC::CodexAdapter.new('timeout_seconds' => t)
+  when 'cursor' then LC::CursorAdapter.new('timeout_seconds' => t)
+  else raise "unknown adapter #{spec[:adapter]}"
+  end
+end
+
+out = File.join(dir, 'records', 'audits.jsonl')
+task_sha = Digest::SHA256.hexdigest(AUDIT_TASK)
+puts "#{dir}: auditing #{analyses.size} analyses with #{panel.size} auditors " \
+     "(#{analyses.size * panel.size} calls), task #{task_sha[0, 12]}"
+
+analyses.each do |a|
+  lens = block(a['text'], 'LENS')
+  ten  = block(a['text'], 'TEN')
+  s    = scores(a['text'])
+  if lens.empty? || s.size < 4
+    warn "  skip #{a['model']}: lens=#{lens.length} chars, scores=#{s.size}"
+    next
+  end
+
+  body = <<~B
+    #{AUDIT_TASK}
+
+    ## The analyst under audit
+
+    Model: #{a['model']}
+
+    ### The standard it stated it applied
+
+    #{lens}
+
+    ### What it stated a 10 would have been in this record
+
+    #{ten.empty? ? '(not stated — this analysis predates that request)' : ten}
+
+    ### The scores it gave
+
+    #{s.map { |k, v| "#{k}: #{v}" }.join("\n")}
+
+    ## The record it was scoring
+
+    #{RECORD}
+  B
+
+  panel.each do |auditor|
+    started = Time.now
+    reply = nil
+    error = nil
+    begin
+      res = adapter_for(auditor).call(messages: [{ 'role' => 'user', 'content' => body }],
+                                      model: auditor[:model])
+      reply = res['content']
+    rescue StandardError => e
+      error = "#{e.class}: #{e.message}"
+    end
+
+    File.open(out, 'a') do |f|
+      f.puts JSON.generate({
+        'at' => Time.now.utc.iso8601(3),
+        'auditor' => auditor[:model],
+        'auditor_seat' => auditor[:id],
+        'audited' => a['model'],
+        'self_audit' => auditor[:model] == a['model'],
+        'audited_guideline_sha256' => a['analysis_guideline_sha256'],
+        'audit_task_sha256' => task_sha,
+        'prompt_chars' => body.length,
+        'seconds' => (Time.now - started).round(1),
+        'ok' => !reply.nil?,
+        'error' => error,
+        'text' => reply
+      })
+    end
+    mark = auditor[:model] == a['model'] ? ' (self)' : ''
+    puts "  #{auditor[:model]} -> #{a['model']}#{mark}: ok=#{!reply.nil?} " \
+         "#{(Time.now - started).round(1)}s #{reply ? "#{reply.length} chars" : error}"
+  end
+end