kairos-chain 3.55.0 → 3.56.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2291c491a077a0cfdce3d77bdd3709e8e6278fc9eb656a02f3de5508e4f66580
-  data.tar.gz: 96fa06b0195df3167b3113c54125a093f31f4bed10955c044a2ef218fe6d9278
+  metadata.gz: 185d6af51fd316124bc72eabbfc3f6b280fee6a2beeda91ed2b690781bb6c26f
+  data.tar.gz: 83951da8550d67d22b67a62153c71a923c4d016a59e94c86d07062a5aac4fedb
 SHA512:
-  metadata.gz: 72064bd0e1bde0516e8dd2b90adcbb9a85f3d079f71507b9f77210985e2f995fc9a1e8af066eee19835dda664a707f09ae8f6420ecfb6135dec62aa92449b756
-  data.tar.gz: 2553da41e8590b33c49f8e0d3526d37ac80bc8f742bab79af45cb5798c7a425ad978e66099a47454f9ca1ade3cbce310a798b7a9d1400eedac7a781d3cfdd745
+  metadata.gz: 14288ed7e74a7a276f68c94292935804830f1d302272ac5ff2e1ba11865ac00cf99acb3d8d93382a2fa0c878e3c0df1ddb4f2b9dfbc742f387386d32c0a6e1da
+  data.tar.gz: cdc967af9cd5962d175787d22cdfd63c881be35aefd6d2dcfba0afd14714482674465cf11df138f646214c37b2800163b982546da5381367c8826d9a74b3b7f4

data/CHANGELOG.md

@@ -4,6 +4,41 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
 
 This project follows [Semantic Versioning](https://semver.org/).
 
+## [3.56.0] - 2026-07-31
+
+### Changed
+
+- **`multi_llm_review` SkillSet 0.7.0 — verdict reading hardened across both
+  observer paths** (review rounds R10–R15, frozen by operator declaration
+  2026-07-31 after three consecutive rounds with zero deployment-grounded
+  findings).
+  - The verdict vocabulary is one table (`VerdictVocabulary::WORDS`); the
+    prose-search and whole-value patterns are built from it, differing only
+    in anchoring and separator strictness. The rebuilt regexes are
+    source-identical to the previous literals.
+  - Verdict *reading* is `stated` (whole-value, anchored) on every path.
+    `VerdictVocabulary.classify`, `PersonaAssembly.normalize_verdict`, and
+    the dead constants `VERDICT_PATTERNS` / `ALLOWED_VERDICTS` /
+    `*_ALIASES` are deleted; their absence is pinned by tests.
+  - A persona verdict field that is not a verdict is **refused at
+    submission** (`PersonaAssembly.validate!` raises ArgumentError) instead
+    of being word-searched (≤ R12) or defaulted to REVISE (R13). The collect
+    tool validates before consuming, so a refused submission leaves the
+    pending token collectable and the corrected submission carries the vote.
+  - Declared verdicts are carried in canonical case (`extract_verdict`
+    merges the admitted form), so a lower-case declaration can no longer sit
+    in the denominator without counting.
+  - Test suite grew 401 → 478 runs / 1676 assertions, including a
+    mutation-survivor file distilled from three generative sweeps.
+- **`llm_client` SkillSet 0.2.0 — CLI envelope diagnostics preserved.**
+  `claude_code_adapter` now picks `model_observed` by output tokens rather
+  than hash order, and passes `model_usage`, `api_error_status`,
+  `fast_mode_state` and `terminal_reason` through to the response. Four
+  model-divergence incidents (a slot requesting opus-4-6 answered by
+  haiku-4-5) were undiagnosable because these fields were discarded; the
+  divergence itself was confirmed real (the divergent reply cited
+  identifiers that exist nowhere in the reviewed code).
+
 ## [3.55.0] - 2026-07-27
 
 ### Added

data/lib/kairos_mcp/version.rb

@@ -1,4 +1,4 @@
 module KairosMcp
-  VERSION = "3.55.0"
+  VERSION = "3.56.0"
   CHANGELOG_URL = "https://github.com/masaomi/KairosChain_2026/blob/main/CHANGELOG.md"
 end

data/templates/knowledge/multi_llm_review_workflow/multi_llm_review_workflow.md

@@ -1,7 +1,7 @@
 ---
 name: multi_llm_review_workflow
 description: "Multi-LLM review methodology and execution — workflow pattern, CLI tooling, consensus analysis, Persona Assembly. Applicable to design, implementation, documentation, or any artifact."
-version: "3.7.0"
+version: "3.8.0"
 tags:
   - workflow
   - review
@@ -29,6 +29,51 @@ This skill covers:
 For **WHO** (which LLM is good at what), see: `multi_llm_reviewer_evaluation`
 For **development lifecycle** (design → implement → verify), see: `design_to_implementation_workflow`
 
+## Step -1 — Pre-declared review spec (loop hygiene)
+
+> Validation scope: these rules were derived from one non-converging loop
+> (multi_llm_review R10–R15, 2026-07) where they took the P0 count from 10 to
+> 2 in two rounds and closed the loop in three. They are instance practice
+> until reproduced on a second, non-self-referential subject; treat the
+> numbers below as one loop's evidence, not a law.
+
+Before dispatching round 1 — and again whenever the review TARGET changes —
+write a review spec and declare it frozen for the round:
+
+1. **Pre-declare the pass condition** (per `loop_validation`: spec before
+   judgement, fail-closed). State what APPROVE requires. A loop whose target
+   drifted (e.g. from an implementation to the instrument that measures it)
+   without a re-declared spec is structurally non-converging: a 300-claim
+   artifact at any realistic per-claim error rate yields double-digit
+   findings every round regardless of quality.
+2. **Split target from appendix.** Only the shipping deliverable is
+   P0-eligible. Instruments, sweep logs, classification tables are appendix:
+   findings against them are advisory and go to the queue. This is what
+   collapsed the claim surface from ~350 to ~30.
+3. **Cap fixes per round (≤5)** and write one line per fix: *what this fix
+   newly claims* (values pinned, ranges narrowed, failure visibility
+   changed). A fix that cannot state its new claims is doing more than the
+   finding asked.
+4. **Pre-flight falsifier.** Before dispatch, one agent whose only job is to
+   refute every factual claim in the spec and artifact — especially numbers
+   and "X does not exist" claims. In this loop it caught real errors before
+   every single dispatch (3 + 1 + 0 refuted across R13–R15); rounds without
+   it had returned the same errors as P0s.
+5. **Check threshold reachability before dispatching.** Compute the maximum
+   achievable approve count from live slots; if the threshold is
+   unreachable, declare the exhaustion path up front (the frozen design's
+   own closing: findings exhausted → operator freeze declaration) instead of
+   discovering it at collect.
+6. **Reference originals by path + sha256; do not transcribe.** Reviewers
+   read the repository; the artifact carries the manifest. Transcription
+   errors are undetectable and 100KB+ pastes rot.
+
+Corollaries observed in the same loop: fix the *class*, and fix every copy —
+a corrected lib comment whose refuted twin survives in a test file costs a
+full round. When an author writes history into comments, the falsifier must
+check the cited records; two of the loop's P0s were numbers copied from the
+wrong document.
+
 ## Step 0 — Load reviewer characteristics (mandatory)
 
 **Before invoking any reviewer**, fetch `multi_llm_reviewer_evaluation` via
@@ -878,26 +923,102 @@ after five consecutive non-substantive returns.
 
 ### Substance and the denominator
 
-A reply that carries a verdict and nothing else does not count as a review. It
-leaves the denominator instead of being counted as agreement — otherwise it
-raises the agreement everyone else must reach while contributing none of its own,
-which is the failure mode that retired Fable 5 from the roster.
+A reply counts toward the denominator when it **carries a verdict** and **says
+something beyond it**. Both conditions, separately checked; a reply failing
+either leaves the denominator rather than being counted, because otherwise it
+raises the agreement everyone else must reach while contributing none of its own.
+
+The two failures look nothing alike, and conflating them is why this rule was
+rebuilt four times:
+
+| Reply | Carries a verdict? | Says something? | Outcome |
+|---|---|---|---|
+| `APPROVE` | yes | no | `insubstantial` |
+| `APPROVE. P0` | yes | no (a severity names no defect) | `insubstantial` |
+| `I'll review this and give my assessment.` | **no** | yes | `no_verdict` |
+| `競合状態あり` | no (no verdict word) | yes | `no_verdict` |
+| `REJECT: P0 key logged in plaintext` | yes | yes | counts |
+| `NO-GO` + findings | yes (see the vocabulary below) | yes | counts as REJECT |
+
+The second row is the shape that retired Fable 5 — long enough to pass any
+substance rule, and stating no judgement. Three rebuilds of the substance rule
+missed it because they were all looking at the wrong half. A reply that states
+no verdict used to be counted as a conservative `REVISE`, which blocked
+convergence on a judgement its author never gave.
 
 - The test is mechanical and has no setting: **does anything remain once the
   verdict word is removed?** No model is asked to judge whether a review is good.
 - It applies to every observer equally, the persona team included. A persona
   submission whose findings are blank leaves the denominator exactly as a
-  subprocess reply would.
+  subprocess reply would — and so does a structured reply from any other slot.
+  A JSON reply is judged by **the words it carries, whatever keys they live
+  under**, not by the text it renders to and not by keys named in advance.
+  Judging it by rendered text counted its own key names as content: the residue
+  of `{"overall_verdict": "APPROVE", "findings": [], "reasoning": ""}` is three
+  words of schema and no review. Naming the keys instead threw away real
+  reviews: a REJECT whose findings used `description` was recorded as an empty
+  submission. **Answer in whatever shape you like** — nothing here requires a
+  particular key.
+- **Open your reply with your verdict.** A structured submission states its
+  verdict as a field; a free-text reply states it in an `**Overall Verdict**`
+  header **on the first line, before anything else**. A header anywhere else is
+  not read as yours — it may be a sample you are discussing — so quoting verdict
+  headers further down is safe and needs no escaping.
+
+  Three rounds tried to tell a stated header from a displayed one by reading the
+  text more cleverly, and each attempt opened a hole in the direction that
+  passes: round 4 anchored the search to the start of a line, and a line inside
+  a fence starts a line; round 6 excluded fenced and blockquoted regions, and a
+  four-backtick fence closes on the inner three-backtick line, taking the rest
+  of the reply — including the real verdict — with it. A quotation and a
+  statement are the same characters, so no amount of reading decides between
+  them. Position does, because a quotation cannot be first unless the reviewer
+  chose to open with one.
+
+  A reply that opens with a preamble falls to a last-resort word scan over the
+  whole text. That is a guess, and it reads quotations along with everything
+  else — but it checks REJECT first, so it cannot turn a stated rejection into
+  a recorded approval.
+- **The words that count as a verdict are one list, the same for every
+  observer.** `APPROVE / PASS / ACCEPT / LGTM / SHIP IT`,
+  `REJECT / FAIL / BLOCK / NO-GO / NACK / DENY / VETO`,
+  `REVISE / CHANGES REQUIRED / NEEDS WORK / NEEDS REVISION / REWORK`. Where a
+  reply carries more than one, the blocking reading wins (REJECT, then REVISE,
+  then APPROVE) — a reply saying two of them has qualified one judgement, not
+  stated two. Japanese verdict words are deliberately **not** in this list
+  (author decision, 2026-07-28): a Japanese review counts on its findings, but
+  its judgement must be stated in one of the words above.
 - It is **not** a quality bar, and must not be tuned into one. `競合状態あり` is a
   review. So is `REJECT: P0 private key logged in plaintext`. Earlier versions of
   this rule measured length and, at one setting, discarded Japanese reviews
   entirely — reviewers answer in the artifact's language, and this project's
   artifacts are largely Japanese.
-- What leaves the denominator is recorded with the reason: `skip_reason` reads
-  `insubstantial` or `transport`, per reviewer and again in
-  `denominator_composition`. A slot that answered emptily and a slot that never
-  answered are both absent from the numerator and must never be confused in the
-  record.
+- What leaves the denominator is recorded with the reason, per reviewer and
+  again in `denominator_composition`. A slot that answered emptily and a slot
+  that never answered are both absent from the numerator and must never be
+  confused in the record, so `skip_reason` distinguishes: `insubstantial` (a
+  reply that said nothing beyond its verdict), `no_verdict` (a reply that stated
+  no judgement), `transport` (a call that was attempted and failed),
+  `not_dispatched` or the dispatcher's own wording such as `dispatch_timeout`
+  (a slot this system declined to run), and `declined` (a submission that stated
+  SKIP for itself — reachable only by a caller that declares it, which nothing
+  in this repository currently does). Only a token-shaped reason is carried
+  through; anything else becomes `not_dispatched`, so a traceback or a sentence
+  cannot land in a field documented as a small vocabulary. Nothing is defaulted:
+  a row with no reason omits the field, because a default here states a cause
+  the record does not know.
+
+- **The escalation record is filled in only where it is wholly absent.** Pending
+  state written before escalation existed carries no such field, and that one
+  case is answered truthfully — a version with no escalation to offer cannot
+  have been asked for it. A record that exists is carried as recorded, gaps and
+  all: completing a partial one wrote `requested: false` beside
+  `escalated: true`, a pair the producing code cannot emit. Silence about a key
+  is legible to a reader; a value asserted where none was recorded is not.
+- The count beside these is `observers_reporting` — the observers that answered,
+  which is not the number of slots the configuration named. That question is
+  answered by `denominator_composition`, which lists every observer including
+  the ones that never ran and why.
 
 **What this rule cannot do.** It cannot tell a terse honest approval from a lazy
 one. `**Overall Verdict**: APPROVE / No findings` is the exact form the reviewer
@@ -1247,5 +1368,69 @@ Compression ratio: parallel agent raw → Assembly ≈ 2:1
   the human, is the primary close — a reached ratio is neither necessary nor
   sufficient on its own
 
+- Verdict determination hardened after a live failure (v3.7.1, 2026-07-27):
+  round 4 of the escalation/persona implementation review recorded the persona
+  team's REVISE as an APPROVE. The team's verdict was being re-derived by
+  searching the assembled text, and a persona had quoted
+  `{"overall_verdict": "APPROVE", ...}` inside a finding as an example of a
+  defect; the search found the quotation before the `**Overall Verdict**:
+  REVISE` on line 1. Three fixes: a structured submission now states its verdict
+  as a field and nothing in its prose overrules it; the header is recognised
+  only at the start of a line; and a JSON verdict is read by parsing a reply
+  that *is* a JSON document rather than by scanning prose for an object. The
+  residual limitation is recorded in § Substance and the denominator — the
+  last-resort word heuristic still sees verdict words inside quotations, so
+  state your verdict in the header when discussing reply shapes. Separately,
+  INV-E2 was found to have been implemented by half: it asks that a counted
+  reply *carry a verdict* and *have substance*, and only substance was checked,
+  so an opening sentence with no judgement in it entered the denominator as a
+  conservative REVISE — the exact shape that retired Fable 5, blocking
+  convergence on nobody's verdict
+
+- Three of round 4's own fixes reopened what they closed (v3.7.2, 2026-07-28):
+  round 5 of the same implementation review found that each of the three
+  verdict fixes recorded above had left a hole, and all three in the direction
+  that passes. Start-of-line header matching does not exclude a quotation,
+  because a line inside a fenced block starts a line — the header is now read
+  from what the reviewer said, with fences and blockquotes excluded, and a
+  reply that is entirely fenced is read whole because it is quoting nothing.
+  Judging a structured reply by named keys (`reasoning`, `issue`) threw away a
+  REJECT whose findings used `description` — the rule now asks whether anything
+  was said in words, under any key. And the words that count as a verdict were
+  two different lists, so `NO-GO` from a persona was a REJECT while `NO-GO`
+  from an external slot was no judgement at all; there is now one list, with
+  one precedence. Alongside these: `skip_reason` distinguishes five outcomes
+  where it previously flattened three into `transport`; `total_configured` was
+  renamed `observers_reporting` because it never counted what it claimed; the
+  eased convergence rule after an exclusion now asks whether the denominator
+  actually shrank rather than which reason fired; and a synchronous delegation
+  writes the same storage layout as the parallel one, so the lock serialising
+  two concurrent collects is no longer silently skipped
+
+- Position replaces cleverness in verdict reading (v3.7.3, 2026-07-28): round 6
+  was reviewed and rejected by five of six observers, all of them naming the
+  same shape — the fixes of round 6 had reopened what they closed, in the
+  direction that passes. Two were shipped defects: the header capture was
+  written with `\s`, which includes the newline in Ruby, so it ran past the
+  header into the following prose and recorded a stated APPROVE followed by
+  "No blocking issues" as a REJECT; and the fence regex tracked neither fence
+  length nor delimiter, so a four-backtick block quoting a three-backtick
+  sample gave the quoted verdict the reviewer's vote. Rather than a fourth
+  attempt at reading free text more carefully, the rule is now positional: the
+  reviewer's verdict is the header the reply opens with, and the fence and
+  blockquote machinery is deleted. The prompt asks for it there, and this test
+  now pins that it does. Alongside: a partial escalation record is carried as
+  recorded rather than completed with values the producing code cannot emit;
+  only a token-shaped skip reason is carried into the record; and a row with no
+  reason omits the field in the per-reviewer list as it already did in the
+  composition.
+
+  The round's other lesson was about testing, and it is recorded here because
+  it generalises past this SkillSet: **a regex pinned by one literal example is
+  free everywhere else**. Round 6's own mutation pass replaced whole functions
+  and killed 21 of 21; the review's mutation pass went after regex internals —
+  fence markers, character classes, digit ranges, word boundaries — and 18 of
+  27 survived. Mutate the inside of a pattern, not only the pattern.
+
 **Key insight**: Design reviews and implementation reviews find
 **categorically different bugs**. Both phases are necessary.

data/templates/skillsets/llm_client/lib/llm_client/claude_code_adapter.rb

@@ -183,18 +183,40 @@ module KairosMcp
           result_text = data['result'] || ''
           tool_use = extract_tool_use(result_text)
           usage = data['usage'] || {}
+          model_usage = data['modelUsage'] || {}
+
+          # The answering model is the entry that produced the output tokens,
+          # not the first hash key. `keys.first` read whichever entry the CLI
+          # inserted first, so if the envelope ever carries a second model
+          # beside the main call, the reply could be attributed to the wrong
+          # one. Every envelope probed so far carries exactly one key, so the
+          # two readings have not yet disagreed — this is defence, not a
+          # reproduced defect.
+          observed = model_usage.max_by { |_m, u| (u || {})['outputTokens'].to_i }&.first
 
           {
             'content' => tool_use ? nil : result_text,
             'tool_use' => tool_use,
             'stop_reason' => tool_use ? 'tool_use' : map_stop_reason(data['stop_reason']),
-            'model' => requested_model || data.dig('modelUsage')&.keys&.first || 'claude_code',
+            'model' => requested_model || observed || 'claude_code',
             # What the CLI reports as having answered, when it reports it.
             # Kept separate from 'model' (which echoes the request) so callers
             # can tell a request from an observation and notice when the two
             # disagree — the CLI may serve a different model than the one
-            # asked for.
-            'model_observed' => data.dig('modelUsage')&.keys&.first,
+            # asked for. That is not hypothetical: four times in the
+            # multi_llm_review loop (R6/R8/R10/R13, 2026-07) a slot requesting
+            # claude-opus-4-6 was answered by claude-haiku-4-5, and the R13
+            # reply's content confirmed the observation (it cited identifiers
+            # that exist nowhere in the reviewed code).
+            'model_observed' => observed,
+            # Diagnostic envelope, previously discarded. All four divergence
+            # incidents above were undiagnosable from the record because the
+            # fields that say what happened did not survive this method.
+            # Consumers that persist reviews should carry these through.
+            'model_usage' => model_usage.empty? ? nil : model_usage,
+            'api_error_status' => data['api_error_status'],
+            'fast_mode_state' => data['fast_mode_state'],
+            'terminal_reason' => data['terminal_reason'],
             'input_tokens' => usage['input_tokens'],
             'output_tokens' => usage['output_tokens']
           }

data/templates/skillsets/llm_client/skillset.json

@@ -1,6 +1,6 @@
 {
   "name": "llm_client",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Pure LLM provider abstraction. One API call, returns response. No loop, no retry, no fallback.",
   "author": "Masaomi Hatakeyama",
   "layer": "L1",

data/templates/skillsets/llm_client/test/test_claude_code_adapter_parse.rb

@@ -0,0 +1,104 @@
+# frozen_string_literal: true
+
+# What parse_response reads from the CLI envelope, pinned after the
+# claude_cli_opus4.6 divergence investigation (2026-07-31).
+#
+# Four times in the multi_llm_review loop (R6/R8/R10/R13) a slot requesting
+# claude-opus-4-6 was answered by claude-haiku-4-5. The record could not say
+# why, because this method discarded every diagnostic field the envelope
+# carries (modelUsage breakdown, api_error_status, fast_mode_state,
+# terminal_reason). The R13 reply's own content settled the observation side
+# — it cited identifiers that exist nowhere in the reviewed code — but the
+# cause remains open precisely because nothing was kept. These tests hold the
+# two fixes: the answering model is chosen by output tokens rather than hash
+# order, and the diagnostic fields survive into the response.
+
+require 'minitest/autorun'
+require_relative '../lib/llm_client/adapter'
+require_relative '../lib/llm_client/claude_code_adapter'
+
+module KairosMcp
+  module SkillSets
+    module LlmClient
+      class TestClaudeCodeAdapterParse < Minitest::Test
+        def setup
+          @adapter = ClaudeCodeAdapter.new({ 'timeout_seconds' => 30 })
+        end
+
+        def parse(payload, requested_model: 'claude-opus-4-6')
+          @adapter.send(:parse_response, JSON.generate(payload), requested_model: requested_model)
+        end
+
+        def envelope(model_usage:, **extra)
+          {
+            'type' => 'result', 'is_error' => false, 'result' => 'ok',
+            'stop_reason' => 'end_turn',
+            'usage' => { 'input_tokens' => 10, 'output_tokens' => 20 },
+            'modelUsage' => model_usage
+          }.merge(extra)
+        end
+
+        def test_a_single_model_envelope_is_that_model
+          out = parse(envelope(model_usage: {
+            'claude-opus-4-6' => { 'outputTokens' => 116 }
+          }))
+
+          assert_equal 'claude-opus-4-6', out['model_observed']
+        end
+
+        # The regression the old `keys.first` invited: if the envelope ever
+        # carries a second model beside the main call, insertion order must
+        # not decide which one "answered". The output tokens do.
+        def test_the_answering_model_is_the_one_that_wrote_the_output
+          out = parse(envelope(model_usage: {
+            'claude-haiku-4-5-20251001' => { 'outputTokens' => 12 },
+            'claude-opus-4-6' => { 'outputTokens' => 2048 }
+          }))
+
+          assert_equal 'claude-opus-4-6', out['model_observed']
+        end
+
+        def test_a_divergent_answer_is_observed_as_itself
+          out = parse(envelope(model_usage: {
+            'claude-haiku-4-5-20251001' => { 'outputTokens' => 900 }
+          }))
+
+          assert_equal 'claude-haiku-4-5-20251001', out['model_observed']
+          assert_equal 'claude-opus-4-6', out['model']
+          refute_equal out['model'], out['model_observed']
+        end
+
+        def test_an_envelope_without_model_usage_observes_nothing
+          out = parse(envelope(model_usage: nil))
+
+          assert_nil out['model_observed']
+          assert_nil out['model_usage']
+          assert_equal 'claude-opus-4-6', out['model']
+        end
+
+        def test_the_diagnostic_fields_survive_into_the_response
+          out = parse(envelope(
+            model_usage: { 'claude-haiku-4-5-20251001' => { 'outputTokens' => 900 } },
+            'api_error_status' => 429,
+            'fast_mode_state' => 'off',
+            'terminal_reason' => 'completed'
+          ))
+
+          assert_equal({ 'claude-haiku-4-5-20251001' => { 'outputTokens' => 900 } }, out['model_usage'])
+          assert_equal 429, out['api_error_status']
+          assert_equal 'off', out['fast_mode_state']
+          assert_equal 'completed', out['terminal_reason']
+        end
+
+        def test_a_missing_output_tokens_entry_does_not_crash_the_choice
+          out = parse(envelope(model_usage: {
+            'claude-opus-4-6' => nil,
+            'claude-haiku-4-5-20251001' => { 'outputTokens' => 5 }
+          }))
+
+          assert_equal 'claude-haiku-4-5-20251001', out['model_observed']
+        end
+      end
+    end
+  end
+end