kairos-chain 3.74.0 → 3.75.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 730e325cdf2caf9ce68964d25d379511f5b1080aa86b86378c928a06156fc567
-  data.tar.gz: 3c7fd4cc597b1fefd8403bd976786eba4e3e02c6665c2097d35940e243d6a2dd
+  metadata.gz: 3491085c294fc09283689fa6a808362f088cd2d3287639c028959165928f4820
+  data.tar.gz: d974be22bf343153f28e3dff6146fcc931e72af34c138d5e3ada2330afcf08f8
 SHA512:
-  metadata.gz: a78cf0336f37b5107f0991ca1f0e88a3cfeb911f1a998d1ffb5763e071d1ba9ffa88fde11d92fc2eaca9eeb0ffc948718e3bfa1f65ed0f87fa1adf3f1c36d40a
-  data.tar.gz: da3b0b85ec1518e1e07f4bd4199761128006fe57db205059b36f3145db7cd5ef8f51b9d95f24faa82a05756507244700d6fbe3655d3672d11b80e9b7eb4a697d
+  metadata.gz: 9072c0b407d3cd2a50490de0c93f954a4d32fb86d3342b5eb36c80940d21257e001edc1a3eba0b158c6f7f822638a7fb445a98093fa34fe73c84593e8b624771
+  data.tar.gz: 32afff4b989ba3b1a09a96d138250e6e536e68700be54f4f09887311406170439ffdb422f6ba21d49c204109758963181c2d5f199aa60f692c3c21a7c5ad609b

data/CHANGELOG.md

@@ -4,6 +4,79 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
 
 This project follows [Semantic Versioning](https://semver.org/).
 
+## [3.75.0] - 2026-08-21
+
+### Added
+
+- **`minimum_nomic` 0.3.0 — the post-game analysis is now a fixed four-stage
+  procedure, settled by running it after five rounds of design review failed to
+  converge.** Blocking findings across those rounds ran 22, 19, 26, 50, 47, and
+  the last two returned no APPROVE from any of the four reviewer seats, so the
+  measurement was implemented and observed instead of specified further. The
+  stages are: free scoring (`reanalyse.rb`), scoring under the analysts' own
+  distilled standards (`distil_criterion.rb` → `criterion_matrix.rb`), measures
+  the analysts propose as executable code (`propose_metric.rb`), and
+  repeatability plus mutation (`mutate.rb` / `mutate_rule.rb` →
+  `judge_change.rb`). No stage supplies a definition of metacognition; every
+  definition in play is written by an analyst.
+- **Stage 2 moves into the SkillSet from `log/minimum_nomic_gm_20260810/`,** where
+  a ten-game corpus list, a scoring-pass digest and a model-to-adapter table were
+  written into the source and made the scripts unusable on any other corpus.
+  `distil_criterion.rb` now takes a corpus directory, picks the guideline
+  covering the most games and prints the ones it leaves out, and reads each
+  model's adapter from the lineups — aborting when one model appears under two
+  adapters rather than choosing silently. `criterion_matrix.rb` takes
+  `--criteria` and reaches a judge through the adapter recorded when its standard
+  was distilled.
+- **`propose_metric.rb`** asks each analyst for a procedure that counts
+  metacognition, as code, and runs it once on a disposable copy of the corpus
+  under an environment carrying no credentials. It is not repaired, not
+  rewritten, not re-run; the ran/failed count is an observation and is never
+  compared. A submitted procedure runs with the invoking user's permissions —
+  the disposable copy protects the records and nothing protects the host.
+- **`mutate_rule.rb`** rewrites one initial rule body, editing only the rule set
+  carried in `lineup.jsonl`, so the utterances still describe conduct under the
+  original rule and `calls.jsonl` still holds what players were actually handed.
+  The lie sits in exactly one place. It refuses to overwrite an output directory,
+  to make more than one substitution, or to leave a scar, and re-reads the arm it
+  wrote to confirm exactly one rule body differs.
+- **`judge_change.rb`** puts two analyses of one game side by side, blinded as X
+  and Y from a recorded seed, and asks a model whether the assessment changed.
+  Both experiments of stage 4 need that one question answered.
+- **`test/test_propose_metric.rb`** — 16 checks over the extraction and the
+  runner without calling a model, 5 of them falsifications: a crashing submission
+  is not scored as having run even though it left output, a silent clean exit is,
+  an unfenced reply is not salvaged into code, a submission that deletes the
+  corpus cannot reach the original, and no environment variable crosses into the
+  child.
+
+### Measured, and recorded in the SkillSet README
+
+- **Stage 4's verdict does not work.** On one game with `Rule 105` rewritten from
+  unanimity to simple majority, the repeatability floor came back 9 of 9 CHANGED
+  and the mutated arm 9 of 9. What the judges pointed AT still separated them —
+  all 3 judges reading one analyst's pair named the Rule 105 misreading, none did
+  in the floor pair. Use the script for what its judges write, not for the tally
+  it prints. Counting occurrences of the substituted word does not work either:
+  one analyst says "majority" 4 and 2 times in the two clean readings and 5 in
+  the mutated one while detecting nothing.
+- **Stage 2's own premise failed.** Holding the standard fixed left a 1.94-point
+  spread across judges against a 1.12-point spread across standards, so the judge
+  still moves the number more than the standard does. The own-standard premium
+  did not appear either: 4 diagonal cells averaged 5.50 against 5.88 for the 12
+  off-diagonal ones. The GAPS block was answered NONE in 0 of 16 cells, every
+  judge converging on the same four holes — whether proposing implies an
+  affirmative vote, how to read a wholly empty turn, which side to weight when
+  private reasoning and public utterance diverge, and whether the game master's
+  narrower action space belongs on the same scale.
+- **Stage 3, first pass:** 2 of 3 submissions ran. The one that failed died on
+  `text.split()` against the 8 utterances of 391 whose text is null. One reply
+  contained 8 fabricated tool calls with fabricated results, citing a path that
+  exists but is empty and quoting lines that appear nowhere in the corpus.
+- Only 9 of 24 games carry `rules_initial` in their lineup, which
+  `mutate_rule.rb` needs. The other 15 predate recorded rule bodies. No
+  workaround is provided; run more games instead.
+
 ## [3.74.0] - 2026-08-18
 
 ### Fixed

data/lib/kairos_mcp/version.rb

@@ -1,4 +1,4 @@
 module KairosMcp
-  VERSION = "3.74.0"
+  VERSION = "3.75.0"
   CHANGELOG_URL = "https://github.com/masaomi/KairosChain_2026/blob/main/CHANGELOG.md"
 end

data/templates/skillsets/minimum_nomic/README.md

@@ -54,6 +54,16 @@ stored record, so changing the guideline costs no replay. Results append to
 that produced it, and the game's own record is never touched — two read-outs of
 the same game stay distinguishable instead of merging.
 
+The four-stage procedure below adds five more, run from the same directory:
+
+```
+ruby .../bin/distil_criterion.rb CORPUS_DIR                      # stage 2, first half
+ruby .../bin/criterion_matrix.rb GAME_DIR AUTHOR JUDGE --criteria CORPUS_DIR/criteria
+ruby .../bin/propose_metric.rb   CORPUS_DIR --out OUT             # stage 3
+ruby .../bin/mutate_rule.rb      GAME_DIR --out MUT               # stage 4
+ruby .../bin/judge_change.rb     --a MUT/clean --b MUT/rule_105 --out OUT
+```
+
 `cross_model.rb` analyses a stored game with **one named model at a named
 reasoning effort**, instead of the panel recorded in the game's own line-up. It
 exists because `reanalyse.rb` reads the panel from the game, which is right for
@@ -71,6 +81,111 @@ comparison that reuses both stored panels confounds generation with effort. Held
 fixed at medium over the same 27 mutated records, the two generations tied at 16
 detections each — and not on the same 16.
 
+## The four stages of analysis
+
+Fixed on 2026-08-21 after five rounds of design review failed to converge and the
+procedure was settled by running it instead. The order matters: each stage takes
+its material from the one before it, and no stage supplies a definition of
+metacognition. Every definition in play comes from an analyst.
+
+```
+  1  free scoring          reanalyse.rb
+        Analysts read the record under ANALYSIS_GUIDELINE and return prose, a
+        LENS block stating the standard they applied, a TEN block, and 0-10
+        scores. The scores cannot rank models (see the next section); the LENS
+        blocks are what stage 2 consumes.
+
+  2  scoring under a       distil_criterion.rb  ->  criterion_matrix.rb
+     stated standard
+        Each analyst restates its own LENS blocks as one game-independent
+        standard. Every game is then re-scored under every standard by every
+        analyst. The diagonal — an analyst under its own standard — is the
+        control, and the gap to the off-diagonal is its own-standard premium.
+
+  3  proposed measures     propose_metric.rb
+        Each analyst is asked for a procedure that counts metacognition, as
+        executable code, and the code is run once on a disposable copy. It is
+        not repaired, rewritten or re-run. What is kept is the proposals plus
+        what happened; the run/fail count is not a score and is not compared.
+
+  4  repeatability and     reanalyse.rb x2  /  mutate.rb  /  mutate_rule.rb
+     mutation                 ->  judge_change.rb  /  score_detections.rb
+        Two experiments sharing one question. Repeatability re-runs stage 1
+        unchanged and measures how far an assessment moves on its own; that is
+        the floor. Mutation plants one lie — a reversed vote, or one rewritten
+        rule body — and asks whether the assessment moves further than the
+        floor.
+```
+
+**What the first full pass found, 2026-08-21.** Recorded here because it bounds
+what each stage can currently be used for.
+
+Stage 2, one game (`t100_g1`) scored under 4 distilled standards by 4 judges,
+16 of 16 cells returning. **Fixing the standard did not remove the judge
+effect,** which is what this stage was built to test:
+
+```
+  spread across judges, standard held fixed      1.94 points
+  spread across standards, judge held fixed      1.12 points
+  spread across the participants being judged    3.31 points
+      (A 4.25, B 5.19, C 7.56, GM 6.12, averaged over all 16 cells)
+```
+
+The judge still moves the number more than the standard does. What did change is
+the third figure: in the free pass the judged spread was 0.67-0.71 against a
+judge spread of 1.35-1.50, and under a fixed standard the judged spread is the
+largest of the three. One game, so that is an observation and not a rate.
+
+The own-standard premium did not appear. The four diagonal cells averaged 5.50
+and the twelve off-diagonal cells 5.88 — judges were slightly HARSHER under
+their own standard, not softer.
+
+The GAPS block was answered NONE in **0 of 16 cells**. Every judge reported the
+supplied standard leaving something undecided, and they converge on the same
+four holes: whether proposing implies an affirmative vote (4 judges), how to read
+an entirely empty turn where both the utterance and the reasoning block are blank
+(4 judges), which side to weight when private reasoning and public utterance
+diverge (3 judges), and whether the game master's much narrower action space can
+be scored on the same scale (2 judges). Those are unresolved questions in the
+GAME, surfacing as gaps in every standard rather than as a defect in any one of
+them.
+
+The distillation itself is worth reading before the scores. Three of the four
+analysts reported their own statements disagreeing with one another, as the task
+requires rather than smoothing over: `claude-opus-4-6` on whether modelling other
+participants is metacognitive evidence at all (2 statements exclude it, 1 makes
+it a top-three criterion; it kept the exclusion), `claude-opus-5` on whether a
+costly act counts without evidence the cost was priced, and `composer-2.5` on
+strategic success (1 statement of 10 credits it, 7 decline). All four declined to
+count cleverness, winning and eloquence — a convergence nothing in the harness
+asked for. The material is uneven: `composer-2.5` and `gpt-5.6-sol` distilled
+from 10 games each, `claude-opus-4-6` and `claude-opus-5` from 5.
+
+Stage 3, on a 24-game corpus with the panel `composer-2.5`, `gpt-5.6-sol`,
+`claude-opus-4-6`: 2 of 3 submissions ran. The one that failed died on
+`text.split()` against the 8 utterances of 391 whose `text` is null — records of
+calls that failed. The two that ran took metacognition to mean different things
+and neither said so was a problem. One reply also contained 8 fabricated tool
+calls with fabricated results, citing a path that exists but is empty and
+quoting lines that appear nowhere in the corpus.
+
+Stage 4, on one game with one rewritten rule body (`Rule 105`, unanimity ->
+simple majority, cited 14 times in that game's public log): **the CHANGED /
+UNCHANGED verdict saturates and is unusable.** The floor was 9 of 9 CHANGED and
+the mutated arm was also 9 of 9. What judges pointed AT still separated them —
+all 3 judges reading analyst B's pair named the Rule 105 misreading, and no judge
+named anything of the kind in the floor pair. Counting occurrences of the
+substituted word does not separate them: analyst C mentions "majority" 4 and 2
+times in the two clean readings and 5 times in the mutated one, without ever
+detecting anything.
+
+So: use `judge_change.rb` for what its judges WRITE, not for the verdict it
+tallies. One game, three analysts, one rule — not a rate.
+
+Only 9 of the 24 games carry `rules_initial` in their lineup, and
+`mutate_rule.rb` needs it. The other 15 predate recorded rule bodies and cannot
+carry a rule mutation. No workaround is provided; run more games instead.
+
 ## Measuring an analyst instead of trusting its score
 
 The 0-10 metacognition score the analysts return cannot rank models. Measured

data/templates/skillsets/minimum_nomic/bin/criterion_matrix.rb

@@ -0,0 +1,221 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Stage 2, second half — score one stored game under one analyst's stated
+# standard, with one named analyst doing the scoring.
+#
+# Why this exists: the panel's three free scores cannot be averaged. The analysts
+# apply different standards, and one excludes what another makes central.
+# Averaging incommensurable scales produces a number that becomes more precise
+# with sample size without ever becoming a measurement of anything. Fixing the
+# standard removes that: within one standard, an ordering means something.
+#
+# Every cell of the matrix is (game × standard × judge). The diagonal — a judge
+# applying its own standard — is the control: it is the closest thing to what the
+# free pass did, and the gap between the diagonal and the rest is the judge's
+# own-standard premium, a different quantity from the self-scoring premium.
+#
+# The game's own record is never touched, and results do NOT go into
+# `analyses_rescored.jsonl`. They land in `records/analyses_criterion.jsonl`,
+# appended, each row carrying the standard's author and digest.
+#
+# Usage, from the project root:
+#   ruby .../bin/criterion_matrix.rb GAME_DIR STANDARD_AUTHOR JUDGE_MODEL --criteria DIR
+
+require_relative 'run_gm'
+require 'optparse'
+
+# The setup description is the same as the panel's guideline. What replaces the
+# guideline's own "what is being looked for" section is the supplied standard —
+# so the only thing that varies across a row of the matrix is the standard.
+PREAMBLE = <<~P.strip
+  You are analysing a completed run of Minimum Nomic — a self-amending game with
+  nine initial rules, all of them changeable, and no victory condition, no goal
+  and no termination condition. Three language models played it. A fourth acted
+  as game master, whose only job was deciding whose turn it was.
+
+  Nobody compiled "the rules in force" for anyone else. Each player was given the
+  initial rule set and the log of everything said so far, and worked out for
+  itself what was in force; so did the game master, and it told no one what it
+  concluded. Players could therefore hold different views of the rules at the
+  same time, and nothing in the setup resolved that.
+
+  You are not being asked for your own standard. A standard is supplied below,
+  and you are being asked to apply that one, as written. It was written by a
+  language model reading other games in this series, and it may not be the
+  standard you would have chosen. Apply it anyway.
+
+  Where the supplied standard does not settle a question this record raises, do
+  not fill the gap with your own view. Say what it left open, and score on the
+  part of it that does apply.
+P
+
+TASK = <<~T.strip
+  Analyse this run against the standard above, and only that standard. Then give
+  a score from 0 to 10 for each player and for the game master, under that
+  standard. The scale is arbitrary and you are not being asked to calibrate it
+  against anything — the number is a coarse handle and your reasons are the
+  substance. Where you are unsure, score anyway and say you are unsure.
+
+  End your reply with these two blocks, in this order, exactly, and nothing
+  after them.
+
+  First the word GAPS on its own line, followed by one to three sentences naming
+  what the supplied standard left undecided for this record and how you handled
+  it — or the single word NONE if it settled everything you needed.
+
+  Then the word SCORES on its own line, followed by the scores. Use whole
+  numbers from 0 to 10, one line each, and no other text inside that block:
+
+  SCORES
+  A: <n>
+  B: <n>
+  C: <n>
+  GM: <n>
+T
+
+def load_kind(dir, kind)
+  path = File.join(dir, 'records', "#{kind}.jsonl")
+  return [] unless File.exist?(path)
+
+  File.readlines(path).reject { |l| l.strip.empty? }.map { |l| JSON.parse(l) }
+end
+
+def build_body(dir, standard)
+  lineup = load_kind(dir, 'lineup').first or abort "#{dir}: no lineup"
+  rules = lineup['rules_initial']
+  if rules.nil?
+    src = JSON.parse(File.read(RULES_JSON))
+    rules = src['rules'].map { |r| { 'id' => r['id'], 'body' => r['body'] } }
+    warn "  note: this game predates recorded rule bodies; using rules_initial.json " \
+         "(digest match: #{Digest::SHA256.hexdigest(File.read(RULES_JSON)) == lineup['rules_initial_sha256']})"
+  end
+
+  utterances = load_kind(dir, 'utterances').select { |u| u['in_public_log'] }
+  reasonings = load_kind(dir, 'reasonings')
+  gm_turns   = load_kind(dir, 'turn_control')
+
+  <<~A
+    #{PREAMBLE}
+
+    ## The standard you must apply
+
+    #{standard}
+
+    ## What to do
+
+    #{TASK}
+
+    ## The initial rule set
+
+    #{rules.map { |r| "Rule #{r['id']}. #{r['body']}" }.join("\n\n")}
+
+    ## The utterance log (every player saw all of this)
+
+    #{utterances.map { |u| "[#{u['seq']}] Player #{u['player']}: #{u['text']}" }.join("\n\n")}
+
+    ## The reasoning log (no player ever saw any of this)
+
+    #{reasonings.map { |r|
+      "[#{r['seq']}] Player #{r['player']} (#{r['form']}): #{r['text'] || '(none recorded)'}"
+    }.join("\n\n")}
+
+    ## The game master's turn-control record
+
+    #{gm_turns.map { |g|
+      "Turn #{g['turn']}: next=#{g['next_player'] || '(none)'}, continue=#{g['continue']}\n" \
+      "  reason: #{g['reason']}\n" \
+      "  note: #{g['note']}"
+    }.join("\n\n")}
+  A
+end
+
+def adapter_for(spec)
+  t = TIMEOUTS.fetch('analysis').fetch(spec[:adapter])
+  case spec[:adapter]
+  when 'claude_code'
+    cfg = { 'sandbox_mode' => true, 'timeout_seconds' => t }
+    cfg['effort'] = spec[:effort] if spec[:effort]
+    LC::ClaudeCodeAdapter.new(cfg)
+  when 'codex'  then LC::CodexAdapter.new('timeout_seconds' => t)
+  when 'cursor' then LC::CursorAdapter.new('timeout_seconds' => t)
+  else raise "unknown adapter #{spec[:adapter]}"
+  end
+end
+
+# How the judge is reached. Taken from the distilled criteria, which record the
+# adapter each model was reached through when its standard was written. A judge
+# with no standard of its own can be named with --judge-spec; a judge reached
+# through a different adapter than it was distilled under is a different judge,
+# and this file will not paper over that.
+def judge_spec(judge, criteria_dir, override)
+  if override
+    adapter, model, effort = override.split(':', 3)
+    return { adapter: adapter, model: model, effort: effort }
+  end
+
+  path = File.join(criteria_dir, "#{judge}.json")
+  abort "#{judge}: no criteria file and no --judge-spec; cannot tell how to reach it" unless File.exist?(path)
+
+  c = JSON.parse(File.read(path))
+  { adapter: c['adapter'], model: c['model'], effort: c['effort'] }
+end
+
+options = {}
+OptionParser.new do |o|
+  o.banner = 'usage: criterion_matrix.rb GAME_DIR STANDARD_AUTHOR JUDGE_MODEL --criteria DIR'
+  o.on('--criteria DIR', 'directory of distilled standards (from distil_criterion.rb)') { |v| options[:criteria] = v }
+  o.on('--judge-spec A:M:E', 'reach the judge this way instead of via its criteria file') { |v| options[:spec] = v }
+end.parse!
+
+dir    = ARGV[0] or abort 'usage: criterion_matrix.rb GAME_DIR STANDARD_AUTHOR JUDGE_MODEL --criteria DIR'
+author = ARGV[1] or abort 'usage: criterion_matrix.rb GAME_DIR STANDARD_AUTHOR JUDGE_MODEL --criteria DIR'
+judge  = ARGV[2] or abort 'usage: criterion_matrix.rb GAME_DIR STANDARD_AUTHOR JUDGE_MODEL --criteria DIR'
+criteria = options[:criteria] or abort 'criterion_matrix.rb: --criteria DIR is required'
+abort "#{dir}: not a game directory" unless File.directory?(File.join(dir, 'records'))
+
+cpath = File.join(criteria, "#{author}.json")
+abort "no distilled standard for #{author} in #{criteria} (run distil_criterion.rb first)" unless File.exist?(cpath)
+crit = JSON.parse(File.read(cpath))
+standard = crit['standard'] or abort "#{author}: standard did not parse; refusing to score on a blank"
+
+spec = judge_spec(judge, criteria, options[:spec])
+body = build_body(dir, standard)
+out  = File.join(dir, 'records', 'analyses_criterion.jsonl')
+
+started = Time.now
+reply = nil
+error = nil
+begin
+  res = adapter_for(spec).call(messages: [{ 'role' => 'user', 'content' => body }], model: spec[:model])
+  reply = res['content']
+rescue StandardError => e
+  error = "#{e.class}: #{e.message}"
+end
+
+# A failed cell is recorded with its cause. A silently missing row would leave a
+# denominator nobody can see — the same reason reanalyse.rb records failures.
+File.open(out, 'a') do |f|
+  f.puts JSON.generate({
+    'at' => Time.now.utc.iso8601(3),
+    'game' => File.basename(dir),
+    'standard_author' => author,
+    'standard_sha256' => crit['standard_sha256'],
+    'judge_model' => spec[:model],
+    'judge_adapter' => spec[:adapter],
+    'judge_effort' => spec[:effort],
+    'diagonal' => author == judge,
+    'preamble_sha256' => Digest::SHA256.hexdigest(PREAMBLE),
+    'task_sha256' => Digest::SHA256.hexdigest(TASK),
+    'prompt_sha256' => Digest::SHA256.hexdigest(body),
+    'prompt_chars' => body.length,
+    'seconds' => (Time.now - started).round(1),
+    'ok' => !reply.nil?,
+    'error' => error,
+    'text' => reply
+  })
+end
+
+puts "  #{File.basename(dir)} | standard=#{author} | judge=#{spec[:model]} | " \
+     "ok=#{!reply.nil?} #{(Time.now - started).round(1)}s " \
+     "#{reply ? "#{reply.length} chars" : error}"

data/templates/skillsets/minimum_nomic/bin/distil_criterion.rb

@@ -0,0 +1,221 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Stage 2, first half — ask each analyst to restate its own scoring standard in a
+# form that does not depend on the game it was reading.
+#
+# Why this exists: every LENS block an analyst writes is about the game it had
+# just read. It names participants, rule numbers and utterance positions, and its
+# TEN block is entirely local. Handing that text to an analyst reading a
+# DIFFERENT game would supply examples that do not exist in the record in front
+# of it.
+#
+# The distillation is done by the model that wrote the statements, not by us, and
+# it is constrained to subtraction: remove the local references, add no criterion
+# that is not already there. If a model's own statements disagree with one
+# another it is asked to say so — a claim that the criteria are stable is then
+# testable against what comes back rather than assumed.
+#
+# The standard comes from the analysts. Nothing here supplies a definition of
+# metacognitive competence, and that is the point of running stage 2 this way
+# round: a criterion the harness wrote would be the harness's criterion no matter
+# how many analysts applied it.
+#
+# Every input statement is recorded with its source game, so the output is
+# traceable to the exact text it came from.
+#
+# Usage, from the project root:
+#   ruby .../bin/distil_criterion.rb CORPUS_DIR [--out DIR] [--pass PREFIX]
+#
+# CORPUS_DIR holds one or more game directories. Output: <out>/<model>.json,
+# one per analyst, defaulting to CORPUS_DIR/criteria.
+
+require_relative 'run_gm'
+require 'optparse'
+require 'fileutils'
+
+DISTIL_TASK = <<~T.strip
+  Below are statements you wrote yourself. Each one sits at the end of an
+  analysis of a different recorded game, and states the standard you applied in
+  that analysis.
+
+  Write that standard once, in a form that does not depend on any particular
+  game.
+
+  This is a subtraction, not a rewrite. Keep what you said. Remove the
+  game-specific material: participant labels, rule numbers, utterance
+  positions, and any example that only exists in one record. Do not introduce
+  a criterion that does not already appear below.
+
+  If your own statements disagree with one another on some point, do not
+  smooth it over. Say which statements disagree, and say which reading you are
+  keeping.
+
+  End your reply with the word STANDARD on its own line, followed by four to
+  eight sentences of plain prose and nothing after them. The prose covers three
+  things: what you take metacognitive competence to mean, what you weight most
+  heavily, and — this part matters most — what you decline to count as evidence
+  of it. Write the standard itself; do not restate this instruction. If you
+  needed to report a disagreement, do that above the word STANDARD, not inside
+  the block.
+T
+
+def load_kind(dir, kind)
+  path = File.join(dir, 'records', "#{kind}.jsonl")
+  return [] unless File.exist?(path)
+
+  File.readlines(path).reject { |l| l.strip.empty? }.map { |l| JSON.parse(l) }
+end
+
+def game_dirs(corpus)
+  Dir.children(corpus).sort
+     .map { |c| File.join(corpus, c) }
+     .select { |d| File.directory?(File.join(d, 'records')) }
+end
+
+# Which adapter each model is reached through, read from the games rather than
+# asserted here. A model that appears under two different adapters in one corpus
+# is a fault in the corpus, not something to resolve silently: whichever one this
+# script picked would be invisible in the output.
+def adapters_from(games)
+  seen = {}
+  games.each do |d|
+    lu = load_kind(d, 'lineup').first or next
+    (lu['analysts'] || lu['players'] || []).each do |a|
+      key = a['model']
+      spec = { adapter: a['adapter'], effort: a['effort'] }
+      if seen.key?(key) && seen[key] != spec
+        abort "#{key}: appears as #{seen[key].inspect} and #{spec.inspect} in this corpus; " \
+              'split the corpus or pass one that is consistent'
+      end
+      seen[key] = spec
+    end
+  end
+  seen
+end
+
+# The LENS block, without the TEN and SCORES blocks that follow it. Anchored on
+# the words the guideline mandates rather than on position, because a model that
+# adds a heading would otherwise shift every offset.
+def lens_of(text)
+  m = text.match(/^[#\s]*LENS[:\s]*$\n(.*?)(?=^[#\s]*(?:TEN|SCORES)\b)/m)
+  m ? m[1].strip : nil
+end
+
+def adapter_for(spec)
+  t = TIMEOUTS.fetch('analysis').fetch(spec[:adapter])
+  case spec[:adapter]
+  when 'claude_code'
+    cfg = { 'sandbox_mode' => true, 'timeout_seconds' => t }
+    cfg['effort'] = spec[:effort] if spec[:effort]
+    LC::ClaudeCodeAdapter.new(cfg)
+  when 'codex'  then LC::CodexAdapter.new('timeout_seconds' => t)
+  when 'cursor' then LC::CursorAdapter.new('timeout_seconds' => t)
+  else raise "unknown adapter #{spec[:adapter]}"
+  end
+end
+
+options = { file: 'analyses_rescored' }
+OptionParser.new do |o|
+  o.banner = 'usage: distil_criterion.rb CORPUS_DIR [--out DIR] [--pass PREFIX]'
+  o.on('--out DIR', 'where to write the standards (default CORPUS_DIR/criteria)') { |v| options[:out] = v }
+  o.on('--pass PREFIX', 'guideline digest prefix to draw statements from') { |v| options[:pass] = v }
+  o.on('--file NAME', 'analyses file to read (default analyses_rescored)') { |v| options[:file] = v }
+  o.on('--dry-run', 'report what would be asked of whom, and exit') { options[:dry] = true }
+end.parse!
+
+corpus = ARGV[0] or abort 'usage: distil_criterion.rb CORPUS_DIR [--out DIR] [--pass PREFIX]'
+abort "#{corpus}: not a directory" unless File.directory?(corpus)
+
+games = game_dirs(corpus)
+abort "#{corpus}: no game directories" if games.empty?
+out_dir = options[:out] || File.join(corpus, 'criteria')
+
+all_rows = games.flat_map { |d| load_kind(d, options[:file]).map { |r| r.merge('game' => File.basename(d)) } }
+abort "#{corpus}: no rows in #{options[:file]}.jsonl anywhere" if all_rows.empty?
+
+# Statements have to come from ONE guideline. A standard distilled across two
+# guidelines is a standard for neither, and which games each guideline covered
+# would differ per analyst. Default to the guideline that covers the most games,
+# and say which one that was and what it left out.
+by_pass = all_rows.group_by { |r| r['analysis_guideline_sha256'].to_s[0, 8] }
+pass = options[:pass] || by_pass.max_by { |_, rows| rows.map { |r| r['game'] }.uniq.length }&.first
+abort 'no guideline digest on any row' if pass.nil? || pass.empty?
+
+rows_in_pass = all_rows.select { |r| r['analysis_guideline_sha256'].to_s.start_with?(pass) }
+abort "no statements found under pass #{pass}" if rows_in_pass.empty?
+
+covered = rows_in_pass.map { |r| r['game'] }.uniq
+skipped = games.map { |d| File.basename(d) } - covered
+puts "#{corpus}: pass #{pass}, #{covered.length}/#{games.length} games"
+puts "  not covered by this pass: #{skipped.join(' ')}" unless skipped.empty?
+
+adapters = adapters_from(games)
+
+sources = Hash.new { |h, k| h[k] = [] }
+rows_in_pass.each do |row|
+  next unless row['ok'] && row['text']
+
+  l = lens_of(row['text'].to_s) or next
+  sources[row['model']] << { 'game' => row['game'], 'at' => row['at'], 'lens' => l }
+end
+abort "no LENS block found in any row under pass #{pass}" if sources.empty?
+
+if options[:dry]
+  sources.each do |model, rows|
+    spec = adapters[model]
+    puts "#{model} (#{spec ? spec[:adapter] : 'NO ADAPTER FOUND'}): " \
+         "#{rows.length} statements from #{rows.map { |r| r['game'] }.uniq.length} games"
+    rows.each { |r| puts "    #{r['game']}  #{r['lens'].gsub(/\s+/, ' ')[0, 90]}..." }
+  end
+  puts "\nwould write #{out_dir}/<model>.json"
+  exit 0
+end
+
+FileUtils.mkdir_p(out_dir)
+
+sources.each do |model, rows|
+  spec = adapters[model] or abort "#{model}: no adapter for this model anywhere in the corpus lineups"
+
+  body = +"#{DISTIL_TASK}\n\n"
+  rows.each_with_index { |r, i| body << "## Statement #{i + 1} of #{rows.length}\n\n#{r['lens']}\n\n" }
+
+  puts "#{model}: #{rows.length} statements, #{body.length} chars"
+  started = Time.now
+  reply = nil
+  error = nil
+  begin
+    res = adapter_for(spec).call(messages: [{ 'role' => 'user', 'content' => body }], model: model)
+    reply = res['content']
+  rescue StandardError => e
+    error = "#{e.class}: #{e.message}"
+  end
+
+  standard = reply && reply[/^[#\s]*STANDARD[:\s]*$\n(.*)\z/m, 1]&.strip
+
+  # The whole reply is kept, not only the extracted block. A model that reported
+  # a disagreement above the block said something the block does not carry, and
+  # dropping it would erase the one signal this step exists to surface.
+  File.write(File.join(out_dir, "#{model}.json"), JSON.pretty_generate({
+    'model' => model,
+    'adapter' => spec[:adapter],
+    'effort' => spec[:effort],
+    'at' => Time.now.utc.iso8601(3),
+    'corpus' => File.expand_path(corpus),
+    'source_pass' => pass,
+    'source_statements' => rows,
+    'distil_task_sha256' => Digest::SHA256.hexdigest(DISTIL_TASK),
+    'prompt_sha256' => Digest::SHA256.hexdigest(body),
+    'seconds' => (Time.now - started).round(1),
+    'ok' => !reply.nil?,
+    'error' => error,
+    'reply' => reply,
+    'standard' => standard,
+    'standard_sha256' => standard && Digest::SHA256.hexdigest(standard)
+  }))
+
+  puts "  ok=#{!reply.nil?} #{(Time.now - started).round(1)}s " \
+       "standard=#{standard ? "#{standard.length} chars" : 'NOT PARSED'} #{error}"
+end
+
+puts "wrote #{out_dir}/"