kairos-chain 3.84.0 → 3.85.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1ebfc6fe1ff6c901c620512c56c698f6dca7bd058143e611547864a0a5460e29
4
- data.tar.gz: caa3229fcb55a9f057a88be84a7a7694e514aff6e9b31ec5a8796d5e1ca93138
3
+ metadata.gz: bb60c9f0163eaf405fdad217697364791fd3bc3d40fe15a77ef57458950c1163
4
+ data.tar.gz: f5e9dd8b9bfc9ccc36472ada9bdc0d0b00104ef06398c198c6d9963369f82d39
5
5
  SHA512:
6
- metadata.gz: 91588bb7de5f9f211409c77e90ba688432415dae81a142341481201ea412d2769d597ed1d10382a726b7ef301fbfdda77439bfa7f27e870102a296f520957656
7
- data.tar.gz: a7a9bd001776163b148e690508f6aa6f44639f01974f00da552f5264676f37f4a25a286c9acb6e502edd6c77f07f9c5de2afd963a6053a89a8d0999721bf55a4
6
+ metadata.gz: 28cb26af2e39d33a5f39e4157c5c2e74f7ff63af77384a73aadb6fe4c03e7c7a8e6c445f7deaff9fdc0b05daf47ad69fdb5a6dee3bdeecd6f40a64f146f7196d
7
+ data.tar.gz: ac81db0e9cb2ab019f2944113de79e7b4c19be5389dead92398f86dcc6b4a82f38491bfad13de2a98d68eefe0ee7ddf94d23289d131662fbff2c1ccfe6dab9cc
data/CHANGELOG.md CHANGED
@@ -4,6 +4,83 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
4
4
 
5
5
  This project follows [Semantic Versioning](https://semver.org/).
6
6
 
7
+ ## [3.85.0] - 2026-09-06
8
+
9
+ ### Changed — multi-LLM review: high effort by default, gpt-6-astra seats, gpt-5.5 retires
10
+
11
+ Templates only: `multi_llm_review/config/multi_llm_review.yml`, L1
12
+ `multi_llm_review_workflow` 3.12.0 → 3.13.0. No library code.
13
+
14
+ **Effort.** Every seat that has an effort control now runs at `high`, at every
15
+ complexity level, so `effort_map` is a constant rather than a function of complexity.
16
+ Claude CLI takes `--effort high`, codex takes `-c model_reasoning_effort=high`, Cursor
17
+ takes nothing — `cursor_adapter` builds no effort flag, so a value set for a cursor
18
+ entry would be recorded and never sent. Both the roster defaults and `effort_map` are
19
+ set, because `effort_map` overrides the roster per dispatch and an entry left at
20
+ `medium` would be silently raised. `high` rather than `xhigh`/`max` because it is the
21
+ ceiling the two providers share (Claude CLI accepts low/medium/high/xhigh/max, codex
22
+ accepts minimal/low/medium/high), and a roster split across incomparable settings is
23
+ worth less than a lower common one.
24
+
25
+ This supersedes the 2026-04-29 default-effort policy. That policy rested on one
26
+ low-vs-high measurement — 8.35 vs 8.16 — taken on the Opus 4.6 / 4.7 generation, on
27
+ none of the models now in the roster. It was not re-measured, so the supersession is a
28
+ judgement and a later measurement could reverse it. Cost is measured and rises:
29
+ gpt-6-astra spent 8,274 tokens at high against 3,150 at its default on one identical
30
+ one-line prompt.
31
+
32
+ **Roster.** `codex_gpt6-astra` replaces `codex_gpt5.6-sol`, verified before the swap
33
+ through the flags `codex_adapter` actually builds. `codex_gpt5.5` retires in the same
34
+ edit and is not replaced: it had been commented out of the instance roster since
35
+ 2026-07-30 "for round R10 only" and never restored, so the calibrated
36
+ cross-generation anchor role had already lapsed for five weeks. Codex now holds one
37
+ slot and it is uncalibrated. Roster 5 → 4, so `convergence_rule` moves 3/5 → 3/4 and
38
+ `convergence_rule_after_exclusion` 3/4 → 2/3, both on the same ceil(N × 0.6) basis and
39
+ both reference figures rather than gates. Three calibration warnings are recorded in
40
+ the L1: the 138-run seat profile is gpt-5.6-sol's and does not transfer; that corpus
41
+ was gathered at medium effort; and it was gathered on a 5-seat roster.
42
+
43
+ ### Fixed — three documents called the APPROVE ratio the closing condition
44
+
45
+ A round closes on the exhaustion of (a)+(b) findings, declared by the operator, with
46
+ new (a)+(b) P0 = 0 as the machine-side signal. The tooling has said so since v0.7
47
+ INV-R2 named the field `reference_verdict` and v0.10.1 renamed `convergence` to
48
+ `vote_tally`. Three documents had not caught up, and they are the ones actually read.
49
+
50
+ L1 `multi_llm_reviewer_evaluation` 1.5 → 1.6. § Convergence Rule (Updated) stated
51
+ `3/4 APPROVE = proceed to next step` and `4/4 APPROVE = merge-ready` with no note that
52
+ the ratio is a reference value — while L1 `multi_llm_review_workflow` names this very
53
+ section as the normative detail it defers to. The document saying "the ratio is not
54
+ the primary close" pointed at a document saying it was. The closing condition is now
55
+ stated first, the one blocking rule (any (a)/(b) REJECT) is separated from the
56
+ reference figures, and the two facts that make the ratio unusable as a criterion sit
57
+ in the section: Codex went 24 of 24 reviews without APPROVE on one design loop, and
58
+ both 2026-08 threads closed by (a)+(b) exhaustion without reaching their ratio.
59
+
60
+ L1 `design_to_implementation_workflow` 1.1 → 1.2. `merge gate: 3/3 APPROVE =
61
+ merge-ready` and `Merge criteria: 3/3 APPROVE with 0 FAIL` replaced by the closing
62
+ condition; the anti-pattern list gains reading an APPROVE count as the closing
63
+ condition, naming this document as having carried it. The reviewer lineup written here
64
+ ("3 reviewers: Opus 4.6 + Codex GPT-5.4 + Composer-2") is deleted rather than
65
+ corrected — it had been wrong since 2026-04-19, and a roster copied into a second
66
+ document goes stale silently.
67
+
68
+ `agent` SkillSet: the bundle handed to reviewers opened "aggregate per the convergence
69
+ rule below" followed by the bare ratio. The heading no longer calls it a convergence
70
+ rule, a Closing condition section states what actually closes a round, and the ratio
71
+ is labelled "Reference ratio (recorded, never sufficient on its own)".
72
+
73
+ Surviving uses of "merge gate" name the stage and its reviewer lineup, not a ratio,
74
+ and stay.
75
+
76
+ **Not reviewed by multi-LLM review**, at the operator's explicit instruction — the
77
+ change is a seat swap plus wording, and a round would have cost hours. Verification is
78
+ this agent's own and is not an independent judgement: YAML load of both configs, the
79
+ three CLI invocations run for real (gpt-6-astra and both Claude models answered at
80
+ high, with the model echoed back), `ruby -c` on `agent_step.rb`, multi_llm_review 556
81
+ runs / 1,809 assertions and agent 21 checks, all passing. No test asserts the reviewer
82
+ prompt wording, so that one line is unguarded.
83
+
7
84
  ## [3.84.0] - 2026-09-03
8
85
 
9
86
  ### Fixed — agent: second field-defect bundle (D5-b, D6, exit-record housekeeping), real-process worker tests
@@ -1,4 +1,4 @@
1
1
  module KairosMcp
2
- VERSION = "3.84.0"
2
+ VERSION = "3.85.0"
3
3
  CHANGELOG_URL = "https://github.com/masaomi/KairosChain_2026/blob/main/CHANGELOG.md"
4
4
  end
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: design_to_implementation_workflow
3
3
  description: "Full-lifecycle workflow for complex features: design review, self-review, implementation review, and final merge gate. Derived from Service Grant + Attestation Nudge experiments."
4
- version: "1.1"
4
+ version: "1.2"
5
5
  tags:
6
6
  - workflow
7
7
  - implementation
@@ -31,7 +31,8 @@ different bugs.
31
31
  │ v0.2 ──→ Multi-LLM Review R2 ──→ Fix ──→ v0.3 │
32
32
  │ (fix correctness) │
33
33
  │ │
34
- Convergence: 0 FAIL, 2/3+ APPROVE
34
+ Closes on: new (a)+(b) P0 = 0, operator declares freeze
35
+ │ (APPROVE ratio is a reference figure, not the gate) │
35
36
  ├─────────────────────────────────────────────────────────────┤
36
37
  │ IMPLEMENTATION PHASE │
37
38
  │ │
@@ -48,7 +49,7 @@ different bugs.
48
49
  │ (missing wiring, fail-open, integration gaps) │
49
50
  │ │
50
51
  │ Final Multi-LLM Review + Persona Assembly │
51
- │ (merge gate: 3/3 APPROVE = merge-ready)
52
+ │ (closes the same way; ratio stays a reference figure)
52
53
  └─────────────────────────────────────────────────────────────┘
53
54
  ```
54
55
 
@@ -76,8 +77,15 @@ different bugs.
76
77
  - Output: Complete design document with pseudocode
77
78
 
78
79
  #### Multi-LLM Review Rounds
79
- - **3 reviewers**: Claude Opus 4.6 + Codex GPT-5.4 + Composer-2
80
- - **Convergence criteria**: 0 FAIL, 2/3+ APPROVE
80
+ - **Reviewers**: the roster in `multi_llm_review/config/multi_llm_review.yml`,
81
+ which is where the count and the models live. The "3 reviewers: Opus 4.6 +
82
+ Codex GPT-5.4 + Composer-2" written here until 2026-09-06 had been wrong since
83
+ 2026-04-19 and is not restated, because a roster copied into a second document
84
+ goes stale silently
85
+ - **Closing condition**: new (a)+(b) P0 = 0, carryover P0s counted separately
86
+ with a closure verdict on each, and the freeze declared by the operator. The
87
+ APPROVE ratio the tool reports is a recorded reference value — see L1
88
+ `multi_llm_review_workflow` § Convergence Rules
81
89
  - **Typical rounds**: 2-3 for Tier 3 complexity
82
90
  - **Convergence curve**:
83
91
  - R1: Structural gaps — "this is missing" (existence)
@@ -148,7 +156,11 @@ Claude Persona Assembly (4 personas):
148
156
  - Safety-critical components
149
157
  - NOT for intermediate rounds (diminishing returns)
150
158
 
151
- **Merge criteria**: 3/3 APPROVE with 0 FAIL. Codex APPROVE is the strongest signal (see `multi_llm_reviewer_evaluation`).
159
+ **Merge criteria**: new (a)+(b) P0 = 0, with the freeze declared by the operator.
160
+ A Codex APPROVE is the strongest *reference* signal (see
161
+ `multi_llm_reviewer_evaluation`), and it is not a gate: Codex went 24 of 24
162
+ reviews without reaching APPROVE on one design loop, so a merge waiting on it
163
+ would never have merged.
152
164
 
153
165
  ## Effort Level Selection
154
166
 
@@ -192,6 +204,9 @@ Reviewer 側 (Opus 4.7) は effort-independent なので high が天井。xhigh/
192
204
 
193
205
  ## Convergence Data
194
206
 
207
+ Observed outcomes of past loops, not criteria. The APPROVE counts below record
208
+ what those particular rounds happened to reach; they are not what closed them.
209
+
195
210
  ### Service Grant (Tier 3, 2026-03-18)
196
211
  - Design: v1.0 → v1.4, 3 review rounds, 3 LLMs
197
212
  - Design review findings: R1: 8 P0/P1, R2: 2 FAIL + 28 CONCERN, R3: 0 FAIL
@@ -207,6 +222,10 @@ Reviewer 側 (Opus 4.7) は effort-independent なので high が天井。xhigh/
207
222
 
208
223
  ## Anti-Patterns
209
224
 
225
+ - **Reading an APPROVE count as the closing condition.** It is a recorded
226
+ reference value; a round closes on new (a)+(b) P0 = 0 with the freeze declared
227
+ by the operator. This document itself carried the mistake until v1.2, in the
228
+ strongest available wording ("merge gate: 3/3 APPROVE = merge-ready")
210
229
  - Implementing Phase 2+ when Phase 1 prerequisites aren't met
211
230
  - Using agent team for implementation (context fragmentation)
212
231
  - Skipping self-review (misses cheap P0 fixes)
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: multi_llm_review_workflow
3
3
  description: "Multi-LLM review methodology and execution — workflow pattern, CLI tooling, consensus analysis, Persona Assembly. Applicable to design, implementation, documentation, or any artifact."
4
- version: "3.11.0"
4
+ version: "3.13.0"
5
5
  tags:
6
6
  - workflow
7
7
  - review
@@ -90,6 +90,72 @@ Codex (c)-class value-divergent REJECTs as blocking, which causes review loops t
90
90
  fail to converge. The cross-reference exists in `related:` frontmatter; this step
91
91
  makes it an explicit pre-condition rather than an implicit hint.
92
92
 
93
+ ## Step 0.1 — Measured seat characteristics (corpus 2026-08-03 .. 2026-09-03)
94
+
95
+ Qualitative per-model profiles stay in `multi_llm_reviewer_evaluation`. What
96
+ follows is the counted complement: seat behaviour as it actually occurred, so
97
+ seat selection is a lookup rather than a recollection. A **seat** is one
98
+ reviewer slot in one run.
99
+
100
+ Corpus: 138 completed runs in the local run store, 556 seat observations, 5286
101
+ findings (P0 1473 / P1 560 / P2 2994 / P3 259). Run types: design 54,
102
+ implementation 60, document 24. Extraction script `log/mlr_extract/extract.py`
103
+ lives in the development repository and does not ship; re-run it to refresh
104
+ these numbers.
105
+
106
+ | Seat | Runs | APPROVE rate design / impl / doc | Median wall s | Median output chars | (c) share, labelled only |
107
+ |---|---|---|---|---|---|
108
+ | `claude_cli_opus4.6` | 134 | 64% (31/48) / 71% (27/38) / 86% (20/23) | 54 | 4,536 | 70% (19/27) |
109
+ | `cursor_composer2.5` | 138 | 13% (7/51) / 48% (24/50) / 25% (6/24) | 133 | 3,795 | 100% (2/2) |
110
+ | `codex_gpt5.6-sol` (retired 2026-09-05) | 138 | 1% (1/54) / 11% (7/60) / 4% (1/24) | 113 | 2,038 | 3% (5/140) |
111
+ | `claude_team_opus-5` (persona) | 131 | 0% (0/51) / 12% (7/56) / 0% (0/22) | not measured per seat | 11,512 | 27% (374/1362) |
112
+
113
+ The codex row is a profile of **gpt-5.6-sol**, which left the roster on
114
+ 2026-09-05 when gpt-6-astra replaced it. Nothing in that row transfers to the
115
+ new occupant: the seat's identity here is the model, not the slot. Until a
116
+ comparable corpus accumulates for `codex_gpt6-astra`, read the codex row as
117
+ history and treat the new seat as uncalibrated. The same caution applies to
118
+ every row from 2026-09-05 onward for a second reason — the corpus was gathered
119
+ with reviewers at medium effort, and reviewers now run at high (see § Thinking
120
+ Effort Configuration), so post-swap rounds are not directly comparable to it.
121
+
122
+ Selection consequences, each tied to the number above it:
123
+
124
+ - **A `codex` APPROVE carries the most information and an `opus4.6` APPROVE the
125
+ least.** Codex approves 1 design review in 54 and pairs that with the lowest
126
+ advisory rate in the corpus (5 of 140 labelled findings). Opus4.6 approves 20
127
+ document reviews in 23 while 19 of its 27 labelled findings are advisory. Seat
128
+ both, read them differently: the `3/4 APPROVE` threshold is met, in practice,
129
+ with opus4.6's vote already inside it, so the live question is whether two of
130
+ the remaining three agree. (That threshold is a reference figure, not a gate —
131
+ see § Convergence Rules.)
132
+ - **Volume anti-correlates with signal.** The persona seat raises 3,249 of the
133
+ 5,286 findings (61%) and 27% of its labelled ones are advisory. Seat personas
134
+ when breadth is wanted; do not seat them to obtain a verdict.
135
+ - **Design draws roughly seven times the P0 of implementation** — median 14 P0
136
+ per design-medium run against 2 per implementation-high run. Budget rounds
137
+ accordingly; a design round returning two P0 is anomalous, not clean.
138
+ - **Dropping a seat does not shorten the round.** Seats run concurrently, so
139
+ wall-clock is the slowest seat (median 179 s per run against 300–400 s summed
140
+ across seats). Removing the 54 s seat saves nothing.
141
+ - **Loops do not converge on their own.** Of 29 artifacts reviewed more than
142
+ once, the last round was REVISE in 26 and APPROVE in 3; rounds per loop median
143
+ 3, maximum 12. P0 counts are not monotone: `chain_history_erasure_fix` ran
144
+ 25, 25, 21, 14, **38**, 29, 26, 13 across rounds 1–8, and
145
+ `ruby_association_grant_2026_application_ja` returned APPROVE at round 8 with
146
+ 3 P0 and then 23 P0 at round 9. Treat a single APPROVE as an observation, not
147
+ as a close — § Convergence Rules already requires the operator's declaration.
148
+
149
+ Two limits of this corpus, both open:
150
+
151
+ - **Findings are not deduplicated across seats.** `cited_by` had exactly one
152
+ entry in 5,286 of 5,286 findings, so agreement between seats is not derivable
153
+ from the store, and the (c) shares above are per-seat rates rather than
154
+ contested classifications.
155
+ - **(a)/(b)/(c) labels are present on 1,622 of 5,286 findings (31%)**, and
156
+ `cursor` labelled 2 of its 730. Classification is applied unevenly by the
157
+ seats, so the shares are computed over labelled findings only.
158
+
93
159
  ## Step 0.25 — Unknowns Pass (pre-draft, qualifying reviews only)
94
160
 
95
161
  > **Numbering vs timing**: Step 0 and Step 0.5 execute at review time,
@@ -399,12 +465,17 @@ they disagree, the config is right and this section is stale.
399
465
  which under the default "delegate" strategy is taken by your persona team
400
466
  rather than spawned — so when you are Opus 5, Opus 4.6 is the only Claude
401
467
  CLI subprocess
402
- - [ ] Codex models: gpt-5.6-sol AND gpt-5.5 (both, not either/or), each with -m
468
+ - [ ] Codex model: gpt-6-astra, with -m. One codex slot since gpt-5.5 was
469
+ retired 2026-09-05 — do not add a second codex entry expecting the old
470
+ cross-generation pairing
403
471
  - [ ] Cursor model: composer-2.5, passed explicitly as --model composer-2.5
404
- - [ ] Total reviewer count: 5 (or 4 after orchestrator exclusion from subprocess)
472
+ - [ ] Effort: high on every seat that has an effort control — `--effort high`
473
+ for Claude CLI, `-c model_reasoning_effort=high` for codex. Cursor has no
474
+ effort control and takes no flag
475
+ - [ ] Total reviewer count: 4 (or 3 after orchestrator exclusion from subprocess)
405
476
  - [ ] Closing condition: new (a)+(b) P0 = 0, with carryover P0s counted
406
477
  separately and a closure verdict on each. The APPROVE ratio the tool
407
- reports (3/5 full roster, 3/4 after exclusion) is a reference value,
478
+ reports (3/4 full roster, 2/3 after exclusion) is a reference value,
408
479
  not the condition — see § Convergence Rules
409
480
  ```
410
481
 
@@ -423,7 +494,7 @@ Path B refuses such a slot outright; on Path A nothing refuses it but you.
423
494
  |---------|-----------------|----------------|
424
495
  | Launch a reviewer without an explicit model flag | Always pass `--model` / `-m`. A slot with no flag takes the CLI's user-editable default | "The default is the one we want" — it was, until someone changed it outside this repo |
425
496
  | Exclude orchestrator model from Agent Team Personas | Agent Team uses orchestrator model — they provide persona diversity, not epistemic diversity | LLM misreads "do not assign yourself as a reviewer" as applying to Agent Team; it applies only to subprocess CLI |
426
- | Run only Codex GPT-5.6-sol, skip 5.5 | Run both — cross-generation entries catch different things (5.5 found §5 schema contradiction in Phase 2 Case A that no other reviewer caught) | Cost-saving heuristic; roster has both for a reason |
497
+ | Run only Codex GPT-6-astra, skip 5.5 | Run both — cross-generation entries catch different things (5.5 found §5 schema contradiction in Phase 2 Case A that no other reviewer caught) | Cost-saving heuristic; roster has both for a reason |
427
498
  | Use a smaller/cheaper model as Agent Team substitute | Use the orchestrator's own model with different personas | Confusing "model diversity" with "persona diversity" — Agent Team is the latter |
428
499
  | Run 3 reviewers instead of the configured roster | Use the full roster from config | Ad-hoc "3 is enough" reasoning; the roster size is empirical |
429
500
  | Count a reply that carries only a verdict | Drop it from the denominator, and say why | A bare "APPROVE" looks like agreement and raises the bar for everyone else without contributing (see § Substance and the denominator) |
@@ -592,7 +663,7 @@ The rule applies **after** orchestrator classifies each finding as (a)/(b)/(c) p
592
663
  findings count toward the thresholds below; (c) findings are recorded as advisory
593
664
  and never block.
594
665
 
595
- - **3/5 APPROVE** full roster, or **3/4 APPROVE** after orchestrator exclusion ("exclude" strategy only — the default "delegate" strategy keeps 5 voters via collect) (no (a)/(b) REJECT) = proceed to next step
666
+ - **3/4 APPROVE** full roster, or **2/3 APPROVE** after orchestrator exclusion ("exclude" strategy only — the default "delegate" strategy keeps 4 voters via collect) (no (a)/(b) REJECT) = proceed to next step
596
667
  - **Any (a) or (b) REJECT or FAIL** = revise and re-review
597
668
  - **(c)-only REJECT** = record as advisory, non-blocking
598
669
  - **Unanimous APPROVE** (no (a)/(b)) = highest confidence, proceed
@@ -643,7 +714,7 @@ For normative detail and the underlying classification, see
643
714
  | Agreement | Meaning | Action |
644
715
  |-----------|---------|--------|
645
716
  | **N/N** (unanimous) | Architectural-level gap | Must fix |
646
- | **Majority** (e.g. 3/5, 3/4) | Implementation-level issue | Should fix |
717
+ | **Majority** (e.g. 3/4, 2/3) | Implementation-level issue | Should fix |
647
718
  | **1/N only** | Specialty-specific insight | Do NOT ignore — often the most novel finding |
648
719
 
649
720
  1/N findings are not "minority opinions to discard." They represent unique expertise.
@@ -743,7 +814,7 @@ outside this repository — see the incident recorded in § Pre-flight checklist
743
814
 
744
815
  | Tool | Command | Prompt Input | Output Collection | Model |
745
816
  |------|---------|-------------|-------------------|-------|
746
- | **Codex** | `codex exec -m <model>` | stdin pipe: `cat prompt.md \| codex exec -m <model> -` | `-o /path/output.md` | gpt-5.6-sol + gpt-5.5 (both roster entries, `-m` per entry) |
817
+ | **Codex** | `codex exec -m <model> -c model_reasoning_effort=high` | stdin pipe: `cat prompt.md \| codex exec -m <model> -` | `-o /path/output.md` | gpt-6-astra one slot since gpt-5.5 was retired 2026-09-05 |
747
818
  | **Cursor Agent** | `agent -p --model composer-2.5` | File reference (stdin NOT supported) | stdout redirect: `> output.md` | composer-2.5, passed explicitly — never relying on the CLI default |
748
819
  | **Claude Code** | Agent tool (internal) | Direct prompt string | Write to workspace file | Orchestrator model, or the declared `persona_model` when personas run elsewhere |
749
820
  | **Claude CLI (4.6)** | `claude -p --model claude-opus-4-6` | stdin pipe: `cat prompt.md \| claude -p --model claude-opus-4-6` | stdout redirect: `> output.md` | Opus 4.6 — the calibrated anchor, deliberately not a frontier model |
@@ -766,11 +837,27 @@ Based on cross-evaluation experiment (7 models × 4 tasks + Nomic, 518 CLI calls
766
837
  |------|-------|-------------|-----------|
767
838
  | **Primary (orchestrator)** | session default | (default) | Sufficient for integration, dialogue, judgment |
768
839
  | **Reviewer: Agent Team** | = orchestrator, or the declared `persona_model` | (default) | Personas inherit whichever model actually runs them |
769
- | **Reviewer: Claude CLI** | Opus 4.6, plus any frontier roster slot the orchestrator is not | (default; config `effort: medium`) | Evaluator quality is effort-independent (low≈high: 8.35 vs 8.16) — per 2026-04-29 policy reviewers stay at default |
840
+ | **Reviewer: Claude CLI** | Opus 4.6, plus any frontier roster slot the orchestrator is not | `--effort high` (config `effort: high`) | Operator instruction 2026-09-05; supersedes the 2026-04-29 default-effort policy see the note below the table |
770
841
  | **Coding sub-agent** | Opus 5 | `--effort xhigh` | Published starting point for coding/agentic work; not measured here (see note) |
771
842
  | **Design sub-agent** | Opus 5 | `--effort high` | Published starting point for intelligence-sensitive work; not measured here (see note) |
772
- | **Codex** | GPT-5.6-sol / GPT-5.5 | (no flag) | Fixed effort |
773
- | **Cursor Agent** | Composer-2.5 | (no flag) | Fixed effort |
843
+ | **Codex** | GPT-6-astra / GPT-5.5 | `-c model_reasoning_effort=high` | Same operator instruction. The earlier "(no flag) / fixed effort" entry was wrong: codex_adapter has always emitted this flag when the roster set `effort` |
844
+ | **Cursor Agent** | Composer-2.5 | (no flag) | Genuinely has no effort control — cursor_adapter builds no such flag, so an `effort:` key on a cursor roster entry is recorded and never sent |
845
+
846
+ Effort policy (2026-09-05, operator instruction). Every reviewer that HAS an
847
+ effort control runs at **high**, at every complexity level, and the `effort_map`
848
+ in `config/multi_llm_review.yml` is a constant rather than a function of
849
+ complexity. `high` is deliberate rather than maximal: it is the ceiling the two
850
+ providers share (Claude CLI accepts low/medium/high/xhigh/max, codex accepts
851
+ minimal/low/medium/high), so it is the highest setting at which the roster stays
852
+ comparable across providers.
853
+
854
+ This supersedes the 2026-04-29 policy that kept reviewers at each CLI's default.
855
+ That policy rested on one measurement — low vs high scoring 8.35 vs 8.16 in
856
+ cross-evaluation — taken on the Opus 4.6 / 4.7 generation, on none of the models
857
+ in the current roster. It was not re-measured, so it is superseded by judgement,
858
+ not by a counter-measurement, and a later measurement could reinstate it. The
859
+ cost side is measured: on one identical one-line prompt, gpt-6-astra spent 8,274
860
+ tokens at high against 3,150 at its default (2026-09-05).
774
861
 
775
862
  Note (2026-07-25): the effort experiment data is from the Opus 4.6/4.7
776
863
  generation. Opus 5 and Fable 5 effort sensitivity is not yet calibrated;
@@ -867,13 +954,13 @@ multi_llm_review(
867
954
  **Dispatcher behavior** (config: `exclude_orchestrator_model: true`, default `true`):
868
955
  - If `orchestrator_model` matches a roster entry's `model`, that entry is skipped.
869
956
  - `min_quorum` and `convergence_rule` apply to the remaining reviewers.
870
- - 5-reviewer roster → 4 reviewers; `convergence_rule_after_exclusion: "3/4 APPROVE"`
957
+ - 4-reviewer roster → 3 reviewers; `convergence_rule_after_exclusion: "2/3 APPROVE"`
871
958
  (from config) replaces the full-roster rule. This reduced count applies to the
872
959
  "exclude" strategy only. The "subprocess" strategy keeps the full roster (the
873
960
  matching entry runs as a fresh CLI process instead of being skipped). Under the
874
961
  default "delegate" strategy, the matching entry is dropped at dispatch but
875
- re-added at collect as the persona-team entry, so the voter count returns to 5
876
- and the full-roster rule (3/5 APPROVE) applies.
962
+ re-added at collect as the persona-team entry, so the voter count returns to 4
963
+ and the full-roster rule (3/4 APPROVE) applies.
877
964
  - **At most one roster entry leaves for matching the caller.** This is only
878
965
  visible on a roster carrying three or more entries on the orchestrator's own
879
966
  model: the first is taken over by the persona team, the second leaves as the
@@ -1342,16 +1429,16 @@ Step 2: Detect environment, and check the roster against config
1342
1429
  - Read the roster from config/multi_llm_review.yml — do NOT read CLI defaults
1343
1430
  and treat them as the roster. Detection only tells you whether a default has
1344
1431
  drifted; the model each slot runs is named on the command line.
1345
- - Report: "Auto mode: Codex (gpt-5.6-sol, gpt-5.5), Cursor (composer-2.5),
1432
+ - Report: "Auto mode: Codex (gpt-6-astra), Cursor (composer-2.5),
1346
1433
  Claude Team (orchestrator model), Claude CLI (opus-4.6)"
1347
1434
 
1348
- Step 3: Execute the configured roster in parallel (currently 5 slots, one of
1435
+ Step 3: Execute the configured roster in parallel (currently 4 slots, one of
1349
1436
  which is your own persona team)
1350
- - Bash(background): cat prompt.md | codex exec -m gpt-5.5 -C workspace -o log/review_codex_gpt5.5.md -
1351
- - Bash(background): cat prompt.md | codex exec -m gpt-5.6-sol -C workspace -o log/review_codex_gpt5.6-sol.md -
1437
+ - Bash(background): cat prompt.md | codex exec -m gpt-6-astra -c model_reasoning_effort=high -C workspace -o log/review_codex_gpt6-astra.md -
1352
1438
  - Bash(background): agent -p --trust --model composer-2.5 "Read prompt and review..." > log/review_cursor.md
1439
+ (no effort flag — Cursor has no effort control)
1353
1440
  - Agent(background): Claude Team (orchestrator model, e.g. Opus 5) → write to log/review_claude_team_opus5.md
1354
- - Bash(background): cat prompt.md | claude -p --model claude-opus-4-6 > log/review_claude_opus4.6.md 2>log/review_claude_opus4.6.stderr.log
1441
+ - Bash(background): cat prompt.md | claude -p --model claude-opus-4-6 --effort high > log/review_claude_opus4.6.md 2>log/review_claude_opus4.6.stderr.log
1355
1442
  (add a line per further Claude roster slot you are not; with the 2026-07-26
1356
1443
  roster an Opus 5 orchestrator has none, so opus-4.6 is the only one)
1357
1444
 
@@ -1391,7 +1478,7 @@ log/{artifact}_review{N}_consensus_{date}.md # Consensus analysis
1391
1478
  ```
1392
1479
 
1393
1480
  LLM identifiers: `claude_cli_opus5`, `claude_cli_opus4.6`,
1394
- `codex_gpt5.6-sol`, `codex_gpt5.5`, `cursor_composer2.5`, `cursor_gpt5.4`,
1481
+ `codex_gpt6-astra`, `cursor_composer2.5`, `cursor_gpt5.4`,
1395
1482
  `cursor_premium`. The delegated slot is reported as `claude_team_<model>`
1396
1483
  (e.g. `claude_team_claude-opus-5`), assembled at collect time — the roster's
1397
1484
  own labels stay CLI-neutral because either frontier entry can take either path.
@@ -1399,7 +1486,11 @@ own labels stay CLI-neutral because either frontier entry can take either path.
1399
1486
  `claude_cli_opus4.7`, `cursor_composer2`; retired 2026-07-23: `codex_gpt5.4`;
1400
1487
  retired 2026-07-25: `claude_cli_opus4.8`, `claude_team_fable5`;
1401
1488
  retired 2026-07-26: `claude_cli_fable5` — five consecutive non-substantive
1402
- returns, 85-128 characters in 5-7 seconds, no findings and no verdict text)
1489
+ returns, 85-128 characters in 5-7 seconds, no findings and no verdict text;
1490
+ retired 2026-09-05: `codex_gpt5.6-sol`, replaced by `codex_gpt6-astra`, and
1491
+ `codex_gpt5.5`, not replaced. Runs recorded under a retired identifier keep it —
1492
+ the label names the model that answered, so renaming old records would attribute
1493
+ one model's findings to another)
1403
1494
 
1404
1495
  ## Internal Agent Team Review
1405
1496
 
@@ -1695,6 +1786,44 @@ Compression ratio: parallel agent raw → Assembly ≈ 2:1
1695
1786
  four rounds was the change's account of itself, not the rules it proposed.
1696
1787
  Records: L2 `handoff_mlr_l1_norms_revision_three_rounds_and_switch_to_implementation_20260821`
1697
1788
  and L2 `mlr_v3_11_0_applied_review_r1_and_subtractive_revision_20260822`
1789
+ - Effort raised to high, and gpt-6-astra replaces gpt-5.6-sol (v3.13.0,
1790
+ 2026-09-05, operator instruction). Two changes with one config edit. **Effort**:
1791
+ every seat that has an effort control now runs at high at every complexity
1792
+ level, so `effort_map` is a constant rather than a function of complexity —
1793
+ Claude CLI gets `--effort high`, codex gets `-c model_reasoning_effort=high`,
1794
+ Cursor gets nothing because cursor_adapter builds no effort flag and a value
1795
+ set for it would be recorded and never sent. `high` rather than `xhigh`/`max`
1796
+ because it is the ceiling the two providers share, and a roster split across
1797
+ incomparable settings is worth less than a lower common one. This supersedes
1798
+ the 2026-04-29 default-effort policy, which rested on a single low-vs-high
1799
+ measurement (8.35 vs 8.16) taken on the Opus 4.6 / 4.7 generation — none of
1800
+ the models now in the roster. No counter-measurement was taken, so the
1801
+ supersession is a judgement and a later measurement could reverse it. Cost is
1802
+ measured and rises: gpt-6-astra spent 8,274 tokens at high against 3,150 at
1803
+ its default on one identical one-line prompt. **Roster**: `codex_gpt6-astra`
1804
+ replaces `codex_gpt5.6-sol`, verified before the swap through the flags
1805
+ codex_adapter actually builds. **gpt-5.5 retired in the same edit**, not
1806
+ replaced: it had been the calibrated cross-generation anchor, but it had been
1807
+ commented out of the instance roster since 2026-07-30 "for round R10 only" and
1808
+ never restored, so the anchor role had already lapsed for five weeks and the
1809
+ retirement only names the state. Codex therefore holds ONE slot, and it is
1810
+ uncalibrated — there is no longer a calibrated codex seat to read a new one
1811
+ against. Roster 5 → 4, so `convergence_rule` moves 3/5 → 3/4 and
1812
+ `convergence_rule_after_exclusion` 3/4 → 2/3, both on the same ceil(N × 0.6)
1813
+ basis. Three calibration warnings follow and none is optional: the 138-run
1814
+ seat profile in § Reviewer selection is gpt-5.6-sol's and does not transfer to
1815
+ the new occupant; the whole of that corpus was gathered at medium effort, so
1816
+ rounds from 2026-09-05 onward are not directly comparable to any of it; and
1817
+ the corpus was gathered on a 5-seat roster, so per-round finding counts shift
1818
+ for a third, independent reason. Record (a)/(b)/(c) breakdowns per round in
1819
+ `multi_llm_reviewer_evaluation` until a profile for the new seat accumulates.
1820
+ Recorded because it was got wrong in the session that made the change: the
1821
+ new ratio was first reported to the operator as "3 of 4 seats must APPROVE",
1822
+ as though it were a gate. It is not. § Convergence Rules has said since v3.5
1823
+ that the ratio is neither the only nor the primary close, `Consensus.compute`
1824
+ returns it under the name `reference_verdict`, and both 2026-08 threads closed
1825
+ by (a)+(b) exhaustion without ever reaching it. Every ratio in this document
1826
+ is a reference figure.
1698
1827
 
1699
1828
  **Key insight**: Design reviews and implementation reviews find
1700
1829
  **categorically different bugs**. Both phases are necessary.
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: multi_llm_reviewer_evaluation
3
3
  description: "Multi-LLM reviewer performance evaluation — strengths, weaknesses, value-system biases, and recommended workflows. Based on 185+ reviews (Phase 1, 2026-02 to 03) + Phase 2 Case A 4-round Codex bias study (2026-05-04)."
4
- version: "1.5"
4
+ version: "1.6"
5
5
  tags:
6
6
  - multi-llm
7
7
  - review
@@ -172,8 +172,9 @@ When a reviewer issues a P0, classify the *cause* — not just the severity:
172
172
  | (b) philosophy-aligned | Deviation from declared design principles (e.g., enumeration where invariant suffices). | **Blocking P0** |
173
173
  | (c) value-divergent | Reviewer's own style preference or generic best practice not entailed by project principles. | **Advisory only** (non-blocking) |
174
174
 
175
- When uncertain between (b) and (c), default to (c). Convergence rule applies to (a)+(b);
176
- (c) findings are recorded but do not block.
175
+ When uncertain between (b) and (c), default to (c). Only (a)+(b) findings count toward
176
+ closing a round; (c) findings are recorded but do not block. Closing is the exhaustion
177
+ of (a)+(b), not an APPROVE count — see § Convergence Rule (Updated).
177
178
 
178
179
  **Codex ↔ classes**: Codex finds genuine (a) bugs (e.g., the §5 schema contradiction).
179
180
  Codex also produces many (c) findings driven by the 3 biases above. The skill of using
@@ -201,6 +202,9 @@ Final Review: Codex APPROVE | Composer-2.5 APPROVE+ | Claude APPROVE+
201
202
  - Codex REJECT reasons are always **substantive** (not stylistic): storage model contradictions, missing call sites, fail-open security
202
203
  - When Codex finally APPROVEs, all prior FAIL/HIGH issues have been genuinely resolved
203
204
  - **Codex APPROVE = strongest merge-readiness signal** in the 3-LLM configuration
205
+ — as a *reference* signal. It is not a gate, and waiting for it is not a
206
+ closing procedure; see the caveat immediately below and § Convergence Rule
207
+ (Updated)
204
208
 
205
209
  > **Note**: The above convergence data is from the 3-reviewer configuration in
206
210
  > the Attestation Nudge session. With the 4-reviewer default (Opus 4.7 added
@@ -216,18 +220,41 @@ Final Review: Codex APPROVE | Composer-2.5 APPROVE+ | Claude APPROVE+
216
220
 
217
221
  ### Convergence Rule (Updated)
218
222
 
219
- The convergence rule applies **after** orchestrator classifies findings as (a)/(b)/(c)
220
- per § Reviewer Value-System Divergence. A REJECT whose findings are entirely (c)
221
- value-divergent is recorded but treated as non-blocking; only (a)+(b) findings count
222
- toward the rule below.
223
-
224
- - 3/4 APPROVE (no (a)/(b) REJECT) = proceed to next step (4-reviewer default)
225
- - Any (a) or (b) REJECT or FAIL = revise and re-review
226
- - **4/4 APPROVE (including Codex) = highest confidence, merge-ready**
227
- - Legacy 3-reviewer mode: 2/3 APPROVE = proceed
223
+ **The APPROVE ratio is not the convergence criterion.** A round closes on the
224
+ exhaustion of (a)+(b) findings, declared by the operator. The machine-side signal
225
+ is **new (a)+(b) P0 = 0**, counting carryover P0s separately; nothing in the
226
+ multi_llm_review SkillSet computes it and no returned field carries it, so it is
227
+ read off the findings. `Consensus.compute` returns the ratio under the name
228
+ `reference_verdict` and the vote counts under `vote_tally` both recorded
229
+ observations, neither a conclusion.
230
+
231
+ The evidence that the ratio cannot serve as the criterion is in this document:
232
+ two Codex entries went 24 of 24 reviews without reaching APPROVE on one design
233
+ loop, and both 2026-08 review threads closed by (a)+(b) exhaustion plus an
234
+ operator freeze declaration without ever reaching their ratio. A threshold a seat
235
+ is structurally unlikely to meet cannot be what closes a round.
236
+
237
+ Everything below applies **after** the orchestrator classifies findings as
238
+ (a)/(b)/(c) per § Reviewer Value-System Divergence. A REJECT whose findings are
239
+ entirely (c) value-divergent is recorded but non-blocking; only (a)+(b) findings
240
+ count.
241
+
242
+ - Any (a) or (b) REJECT or FAIL = revise and re-review. **This one blocks.**
228
243
  - Codex-only REJECT with (a)/(b) findings + others APPROVE = likely real issue, investigate before overriding
229
244
  - Codex REJECT with only (c) findings = expected per Codex value-system divergence; non-blocking
230
245
 
246
+ Reference figures, recorded and never sufficient on their own. Read what the
247
+ approving replies actually said before counting them:
248
+
249
+ - 3/4 APPROVE (no (a)/(b) REJECT) on the current 4-reviewer roster — the literal
250
+ ratio tracks the roster size and changed on 2026-09-05 when gpt-5.5 retired
251
+ - 4/4 APPROVE (including Codex) = the strongest reference signal available, still
252
+ a reference
253
+ - Legacy 3-reviewer mode: 2/3 APPROVE
254
+
255
+ Normative statement and the carryover/new split: L1 `multi_llm_review_workflow`
256
+ § Convergence Rules. Aggregation rule: project CLAUDE.md.
257
+
231
258
  ### Bug Category Differentiation Across Rounds
232
259
 
233
260
  | Review Phase | Typical Bug Category | Example |
@@ -324,6 +351,26 @@ MECHANISM of a sound invariant is (c) -> §11 / implementation review." This con
324
351
  Codex APPROVE" (not always reachable) into "freeze when only (c)/mechanism findings remain,"
325
352
  which is decidable by the orchestrator and resistant to value-divergence stalling.
326
353
 
354
+ ## Changelog
355
+
356
+ - **v1.6 (2026-09-06)**: § Convergence Rule (Updated) rewritten. It had stated
357
+ `3/4 APPROVE = proceed to next step` and `4/4 APPROVE = merge-ready` with no
358
+ note that the ratio is a reference value — while L1 `multi_llm_review_workflow`
359
+ § Convergence Rules names *this* section as the normative detail it defers to.
360
+ So the document that says "the ratio is not the primary close" pointed at a
361
+ document that said it was. Now: the closing condition (new (a)+(b) P0 = 0,
362
+ carryover counted separately, operator declares the freeze) is stated first and
363
+ the blocking rule — any (a)/(b) REJECT — is separated from the reference
364
+ figures. Two supporting facts moved into the section because they are the
365
+ reason the ratio cannot be the criterion: Codex went 24 of 24 reviews without
366
+ APPROVE on one design loop, and both 2026-08 threads closed by (a)+(b)
367
+ exhaustion without reaching their ratio. The "Codex APPROVE = strongest
368
+ merge-readiness signal" line keeps its wording and gains "as a reference
369
+ signal, not a gate". Prompted by the operator after this agent reported an
370
+ APPROVE ratio to them as a gate for the third time; the wrong belief was
371
+ traceable to the text here, not to the tooling, which had been correct since
372
+ v0.7 INV-R2 (`reference_verdict`) and v0.10.1 (`vote_tally`).
373
+
327
374
  ## Refinement Source
328
375
 
329
376
  Profiles in this knowledge are refined from accumulated L2 contexts named with prefix
@@ -2424,9 +2424,15 @@ module KairosMcp
2424
2424
  "Reviewer roster hash: #{bundle['reviewer_roster_hash']}",
2425
2425
  "Config hash: #{bundle['config_hash']}",
2426
2426
  '',
2427
- '## Run each reviewer independently and aggregate per the convergence rule below.',
2427
+ '## Run each reviewer independently, then aggregate.',
2428
2428
  '',
2429
- "Convergence rule: #{bundle['convergence_rule']}",
2429
+ '## Closing condition',
2430
+ 'A round closes on new (a)+(b) P0 = 0, counting carryover P0s',
2431
+ 'separately with a closure verdict on each, and the freeze is',
2432
+ 'declared by the operator. Read it off the findings — no field',
2433
+ 'in this bundle carries it.',
2434
+ '',
2435
+ "Reference ratio (recorded, never sufficient on its own): #{bundle['convergence_rule']}",
2430
2436
  '',
2431
2437
  '## Aggregation instructions',
2432
2438
  bundle['aggregation_instructions'].to_s,
@@ -48,6 +48,12 @@
48
48
  # ruby bench/minimum_nomic/run_gm.rb --out /tmp/smoke --turns 2 # smoke
49
49
  # ruby bench/minimum_nomic/check_gm.rb log/minimum_nomic_gm_20260810/g3 --falsify
50
50
  #
51
+ # Seating a different model, or a different provider in turn control:
52
+ # --codex-model gpt-6-astra --codex-effort high # B seat
53
+ # --gm-adapter codex --gm-model gpt-6-astra --gm-effort high
54
+ # Both go into the lineup, so a game says which generation played without the
55
+ # reader having to know when this file was edited.
56
+ #
51
57
  # One directory per game, and a fresh one every time: the run refuses to start
52
58
  # when the target already holds records. The games themselves are never
53
59
  # committed — this file is tracked, the corpus under log/ is not.
@@ -120,10 +126,47 @@ LC = KairosMcp::SkillSets::LlmClient
120
126
  # Read from ARGV here rather than in the option parser at the foot of the file,
121
127
  # because PLAYER_SPECS is a constant that the prompts and the lineup are built
122
128
  # from and it has to exist before either.
129
+ # Which model sits in the claude seat is also a run parameter, for the same
130
+ # reason. The 61-game corpus of 2026-08 was produced by editing the constant
131
+ # below in place, so "which generation played" is recoverable only from each
132
+ # game's own lineup row and never from this file's history. --claude-model and
133
+ # --claude-effort put that choice on the command line and therefore in the
134
+ # lineup. Defaults are unchanged, so an unflagged run stays comparable.
135
+ def arg_after(flag)
136
+ i = ARGV.index(flag)
137
+ i && ARGV[i + 1] && !ARGV[i + 1].start_with?('--') ? ARGV[i + 1] : nil
138
+ end
139
+
140
+ CLAUDE_SEAT_MODEL = arg_after('--claude-model') || 'claude-opus-5'
141
+ CLAUDE_SEAT_EFFORT = arg_after('--claude-effort') || 'high'
142
+ GM_MODEL = arg_after('--gm-model') || 'claude-opus-5'
143
+ GM_EFFORT = arg_after('--gm-effort') || 'high'
144
+
145
+ # The codex seat carries a model name for the same reason the claude seat does.
146
+ # gpt-5.6-sol sat there through the whole 2026-08 corpus and was written into
147
+ # the constant, so seating a later generation meant editing this file, and the
148
+ # edit left no trace anywhere except in each game's own lineup row. The effort
149
+ # default stays nil rather than becoming a level, because every stored game ran
150
+ # this seat without one and a default would silently change what "unflagged"
151
+ # means.
152
+ CODEX_SEAT_MODEL = arg_after('--codex-model') || 'gpt-5.6-sol'
153
+ CODEX_SEAT_EFFORT = arg_after('--codex-effort')
154
+
155
+ # The game master's ADAPTER, not only its model. Turn control ran on claude_code
156
+ # in every stored game, so "can a different provider hold turn control" cannot
157
+ # be asked by changing the model name alone: a codex model name handed to the
158
+ # claude_code adapter reaches a CLI that cannot serve it, and the run then
159
+ # measures the mismatch instead of the model. Moving both together is the only
160
+ # form of the question that has an answer. The adapter is recorded in the
161
+ # lineup beside the model, so a reader never has to infer it from the name.
162
+ GM_ADAPTER = arg_after('--gm-adapter') || 'claude_code'
163
+
123
164
  SEAT_POOL = {
124
- 'cursor' => { adapter: 'cursor', model: 'composer-2.5', effort: nil },
125
- 'codex' => { adapter: 'codex', model: 'gpt-5.6-sol', effort: nil },
126
- 'claude_code' => { adapter: 'claude_code', model: 'claude-opus-5', effort: 'high' }
165
+ 'cursor' => { adapter: 'cursor', model: 'composer-2.5', effort: nil },
166
+ 'codex' => { adapter: 'codex', model: CODEX_SEAT_MODEL,
167
+ effort: CODEX_SEAT_EFFORT },
168
+ 'claude_code' => { adapter: 'claude_code', model: CLAUDE_SEAT_MODEL,
169
+ effort: CLAUDE_SEAT_EFFORT }
127
170
  }.freeze
128
171
 
129
172
  DEFAULT_SEAT_ORDER = %w[cursor codex claude_code].freeze
@@ -141,7 +184,17 @@ PLAYER_SPECS = SEAT_ORDER.each_with_index.map do |key, i|
141
184
  { id: %w[A B C].fetch(i) }.merge(SEAT_POOL.fetch(key))
142
185
  end.freeze
143
186
 
144
- GM_SPEC = { id: 'GM', adapter: 'claude_code', model: 'claude-opus-5', effort: 'high' }.freeze
187
+ GM_SPEC = { id: 'GM', adapter: GM_ADAPTER, model: GM_MODEL, effort: GM_EFFORT }.freeze
188
+
189
+ # Who can reach this run's own record, over every participant rather than over
190
+ # the players alone. claude_code is the only contained adapter: it is chdired
191
+ # to an empty directory with no tools, so the record is not under its cwd and
192
+ # it has nothing to open it with. codex has a read-only sandbox rooted at the
193
+ # project root and cursor has no sandbox flag at all, so either one can read
194
+ # the record from any seat — the game master's included. Used by the lineup.
195
+ ALL_PARTICIPANTS = (PLAYER_SPECS + [GM_SPEC]).freeze
196
+ CONTAINED = ALL_PARTICIPANTS.select { |s| s[:adapter] == 'claude_code' }.freeze
197
+ UNCONTAINED = ALL_PARTICIPANTS.reject { |s| s[:adapter] == 'claude_code' }.freeze
145
198
 
146
199
  # The analyst roster equals the player roster (v0.8 §6), so self-analysis is
147
200
  # part of the output and is recorded as such.
@@ -420,13 +473,49 @@ class Run
420
473
  def call!
421
474
  load_initial_rules!
422
475
  write_lineup!
476
+ probe_identity!('before_game')
423
477
  halt = play!
478
+ probe_identity!('after_game')
424
479
  analyse! unless @utterances.empty?
425
480
  write_summary!(halt)
426
481
  @recorder.close
427
482
  halt
428
483
  end
429
484
 
485
+ # ── identity probe ──────────────────────────────────────────────────────────
486
+ #
487
+ # Asks each seat, in its own words, which model it is — once before the game
488
+ # and once after. This is SELF-REPORT and is recorded as such: a model that
489
+ # was silently re-routed has no way to know it, and a model that was not can
490
+ # still name itself wrongly. It is kept because it is the answer a reader
491
+ # would otherwise assume, and having it written down beside the transport's
492
+ # account makes the two comparable instead of leaving one of them imagined.
493
+ # The transport account (`model_observed` on every call) is the evidence.
494
+ #
495
+ # It is a separate call and touches no player or game-master prompt. Putting
496
+ # "say which model you are" into the game prompt would tell a player
497
+ # something about itself that no stored game told its players, and every
498
+ # game recorded before this would stop being comparable.
499
+ IDENTITY_PROBE = <<~P.strip
500
+ Which model are you? Reply with the exact model identifier you were invoked
501
+ as, on one line, and nothing else.
502
+ P
503
+
504
+ def probe_identity!(when_label)
505
+ (PLAYER_SPECS + [GM_SPEC]).each do |spec|
506
+ reply = call_llm(spec, [{ 'role' => 'user', 'content' => IDENTITY_PROBE }],
507
+ kind: 'probe', purpose: "identity_#{when_label}")
508
+ @recorder.write('identity_probes', {
509
+ 'at' => now_stamp, 'party' => spec[:id], 'when' => when_label,
510
+ 'requested_model' => spec[:model], 'effort' => spec[:effort],
511
+ 'self_reported' => reply&.strip,
512
+ 'evidential_status' => 'self-report; not evidence of which model answered. ' \
513
+ 'See model_observed on the matching row in calls.jsonl'
514
+ })
515
+ puts "probe #{when_label}: #{spec[:id]} (#{spec[:model]}) said #{reply&.strip.inspect}"
516
+ end
517
+ end
518
+
430
519
  private
431
520
 
432
521
  # ── adapters ────────────────────────────────────────────────────────────────
@@ -485,6 +574,51 @@ class Run
485
574
  row['output_tokens'] = res['output_tokens']
486
575
  row['token_absence_reason'] =
487
576
  "the #{spec[:adapter]} adapter returns no usage counts" if res['input_tokens'].nil?
577
+
578
+ # WHICH MODEL ACTUALLY ANSWERED. `model` above echoes the request; these
579
+ # fields are the transport's own account of what produced the output
580
+ # tokens, taken from the CLI's modelUsage envelope. They exist because a
581
+ # request can be served by a different model than the one asked for —
582
+ # a safety re-route, a fast-mode substitution, a CLI fallback — and
583
+ # nothing else in this record would show it. A seat's identity is then a
584
+ # measured fact per call rather than a claim made once in the lineup.
585
+ #
586
+ # Only the claude_code adapter reports this; the codex and cursor CLIs
587
+ # return no usage envelope, so their rows carry the reason instead of a
588
+ # silent nil.
589
+ row['model_observed'] = res['model_observed']
590
+ row['model_usage'] = res['model_usage']
591
+ row['fast_mode_state'] = res['fast_mode_state']
592
+ row['api_error_status'] = res['api_error_status']
593
+
594
+ # `model_observed` is the adapter's guess: the envelope entry with the
595
+ # most output tokens. It is wrong for short replies. The CLI places an
596
+ # internal helper call (claude-haiku, ~10-17 output tokens against a
597
+ # fixed ~918-token input) beside the main one, so a reply of a dozen
598
+ # tokens loses the comparison to the helper and the call is attributed
599
+ # to a model that answered nothing. Measured on the 2026-09-02 smoke
600
+ # run: 3 of 7 claude calls misattributed this way, all three of them
601
+ # one-line replies, while the 5,691-token analysis was attributed
602
+ # correctly.
603
+ #
604
+ # `model_served_request` does not compare sizes. It asks whether the
605
+ # model that was requested is present in the envelope having produced
606
+ # output at all, which is the question, and is unaffected by how long
607
+ # the reply was. This is the field to read for "did the seat's model
608
+ # actually answer"; `model_observed` is kept beside it unchanged so the
609
+ # two accounts stay distinguishable.
610
+ usage = res['model_usage'] || {}
611
+ row['models_in_envelope'] = usage.empty? ? nil : usage.transform_values { |u|
612
+ (u || {})['outputTokens']
613
+ }
614
+ row['model_served_request'] =
615
+ usage.empty? ? nil : (usage.dig(spec[:model], 'outputTokens').to_i.positive?)
616
+ row['model_matches_request'] =
617
+ res['model_observed'].nil? ? nil : (res['model_observed'] == spec[:model])
618
+ if usage.empty?
619
+ row['model_observation_absence_reason'] =
620
+ "the #{spec[:adapter]} adapter returns no model usage envelope"
621
+ end
488
622
  rescue StandardError => e
489
623
  row['ok'] = false
490
624
  row['error'] = "#{e.class}: #{e.message}"
@@ -589,12 +723,16 @@ class Run
589
723
  # Derived from the seat assignment, not written out: --seats moves the
590
724
  # contained adapter off C, and a hardcoded "C is contained" would then
591
725
  # be a false statement in the record about which seats could read it.
592
- 'contained_seats' => PLAYER_SPECS.select { |s| s[:adapter] == 'claude_code' }
593
- .map { |s| s[:id] } + ['GM'],
594
- 'uncontained_seats' => PLAYER_SPECS.reject { |s| s[:adapter] == 'claude_code' }
595
- .map { |s| s[:id] },
596
- 'statement' => 'containment is by seat capability, not by file location; seats ' \
597
- "#{PLAYER_SPECS.reject { |s| s[:adapter] == 'claude_code' }.map { |s| s[:id] }.join(' and ')} " \
726
+ # The game master is derived the same way and for the same reason.
727
+ # Until 2026-09-05 it was appended to contained_seats as a literal,
728
+ # which was true only while its adapter could not be changed; with
729
+ # --gm-adapter it would have become the very false statement this
730
+ # derivation exists to prevent, and a codex or cursor game master can
731
+ # read this directory exactly as a player on that adapter can.
732
+ 'contained_seats' => CONTAINED.map { |s| s[:id] },
733
+ 'uncontained_seats' => UNCONTAINED.map { |s| s[:id] },
734
+ 'statement' => 'containment is by seat capability, not by file location; ' \
735
+ "#{UNCONTAINED.empty? ? 'no participant' : "seats #{UNCONTAINED.map { |s| s[:id] }.join(' and ')}"} " \
598
736
  'could read this directory if they looked. Not fixed, recorded.',
599
737
  'write_mode' => 'append-only, one game per directory; the run refuses to start when the ' \
600
738
  'directory already holds records, so a game is never destroyed and two ' \
@@ -933,6 +1071,27 @@ class Run
933
1071
  'calls_failed_by_participant' => calls.reject { |c| c['ok'] }
934
1072
  .group_by { |c| c['participant'] }
935
1073
  .transform_values(&:length),
1074
+ # Which model answered, counted rather than assumed. A run where every
1075
+ # claude call was served by the model that was asked for reads
1076
+ # observed_model_mismatches: 0 with a denominator; a re-routed run reads
1077
+ # the substitute's name in observed_models_by_participant.
1078
+ 'observed_models_by_participant' => calls.select { |c| c['models_in_envelope'] }
1079
+ .group_by { |c| c['participant'] }
1080
+ .transform_values { |rows|
1081
+ rows.flat_map { |c|
1082
+ c['models_in_envelope'].keys
1083
+ }.tally
1084
+ },
1085
+ 'calls_with_model_observation' => calls.count { |c| c['models_in_envelope'] },
1086
+ # The one to read. Counts calls where the requested model produced no
1087
+ # output in the envelope — a genuine substitution.
1088
+ 'calls_requested_model_did_not_answer' =>
1089
+ calls.count { |c| c['model_served_request'] == false },
1090
+ # The size-comparison heuristic's disagreement rate, kept for calibration
1091
+ # of that heuristic only. A nonzero count here with zero above means the
1092
+ # short-reply misattribution, not a substitution.
1093
+ 'observed_model_mismatches' => calls.count { |c| c['model_matches_request'] == false },
1094
+ 'fast_mode_states' => calls.map { |c| c['fast_mode_state'] }.compact.tally,
936
1095
  'gm_reasks' => calls.count { |c| c['purpose'] == 'turn_control_reask' },
937
1096
  'gm_turns_unreadable' => @gm_turns.count { |g| !g['readable'] },
938
1097
  'deliveries_recorded' => File.readlines(File.join(@out, 'records', 'deliveries.jsonl')).length,
@@ -970,6 +1129,15 @@ if __FILE__ == $PROGRAM_NAME
970
1129
  # Consumed at the head of this file, where PLAYER_SPECS is built. Declared
971
1130
  # here only so that parse! accepts it instead of dying on an unknown flag.
972
1131
  o.on('--seats LIST') { |_| }
1132
+ # Likewise consumed at the head of the file, where SEAT_POOL and GM_SPEC
1133
+ # are built.
1134
+ o.on('--claude-model MODEL') { |_| }
1135
+ o.on('--claude-effort LEVEL') { |_| }
1136
+ o.on('--codex-model MODEL') { |_| }
1137
+ o.on('--codex-effort LEVEL') { |_| }
1138
+ o.on('--gm-model MODEL') { |_| }
1139
+ o.on('--gm-effort LEVEL') { |_| }
1140
+ o.on('--gm-adapter NAME') { |_| }
973
1141
  end.parse!(ARGV)
974
1142
 
975
1143
  started = Time.now
@@ -4,12 +4,21 @@
4
4
  # to avoid duplication.
5
5
 
6
6
  # Convergence rules
7
- # Roster has 5 reviewers (claude_cli_opus5, claude_cli_opus4.6,
8
- # codex_gpt5.6-sol, codex_gpt5.5, cursor_composer2.5).
7
+ #
8
+ # READ THIS FIRST: the ratio below is NOT the condition on which a round
9
+ # closes. Consensus.compute returns it as `reference_verdict` (v0.7 INV-R2) —
10
+ # a recorded observation. A round closes on the exhaustion of (a)+(b) findings,
11
+ # declared by the operator, and the machine-side signal is "new (a)+(b) P0 = 0",
12
+ # which nothing in this SkillSet computes and no returned field carries. Both
13
+ # 2026-08 review threads closed that way without ever reaching their ratio. See
14
+ # L1 multi_llm_review_workflow § Convergence Rules.
15
+ #
16
+ # Roster has 4 reviewers (claude_cli_opus5, claude_cli_opus4.6,
17
+ # codex_gpt6-astra, cursor_composer2.5).
9
18
  # Rules are ratio-based (parser interprets "N/M" as N/M fraction applied
10
19
  # to successful count), so the literal numerator/denominator is
11
20
  # informational; what matters is the ratio.
12
- convergence_rule: "3/5 APPROVE" # ceil(5 * 0.6) = 3 of the 5-reviewer roster
21
+ convergence_rule: "3/4 APPROVE" # ceil(4 * 0.6) = 3 of the 4-reviewer roster
13
22
  min_quorum: 2 # minimum successful reviews for any verdict
14
23
 
15
24
  # Self-referential orchestrator exclusion.
@@ -32,15 +41,16 @@ exclude_orchestrator_model: true
32
41
  # "exclude": legacy behavior — drop the matching reviewer entirely.
33
42
  # "subprocess": spawn fresh claude -p for the matching reviewer.
34
43
  default_orchestrator_strategy: "delegate"
35
- # After excluding 1 orchestrator from 54 reviewers. Same ceil(N * 0.6)
36
- # majority basis (ceil(4 * 0.6) = 33 of 4 must APPROVE). The two rules are
37
- # not the same literal ratio (3/5 = 0.60 vs 3/4 = 0.75); since the parser
38
- # applies the ratio to the successful count, the post-exclusion rule is
39
- # stricter when some reviewers fail. Accepted as-is.
44
+ # After excluding 1 orchestrator from 43 reviewers. Same ceil(N * 0.6)
45
+ # majority basis (ceil(3 * 0.6) = 22 of 3 must APPROVE). The two rules are
46
+ # not the same literal ratio (3/4 = 0.75 vs 2/3 = 0.67); since the parser
47
+ # applies the ratio to the successful count, they diverge when some reviewers
48
+ # fail. Accepted as-is, and cheaply so now: both values are reference figures,
49
+ # not gates.
40
50
  # Applies to the "exclude" strategy only — "subprocess" keeps the full
41
51
  # roster, and "delegate" re-adds the orchestrator slot at collect, so the
42
- # full-roster rule (3/5) governs both (verified live 2026-06-10).
43
- convergence_rule_after_exclusion: "3/4 APPROVE"
52
+ # full-roster rule (3/4) governs both (verified live 2026-06-10).
53
+ convergence_rule_after_exclusion: "2/3 APPROVE"
44
54
 
45
55
  # Two-phase delegation (orchestrator_strategy: "delegate").
46
56
  # Phase 1 dispatches subprocess reviewers synchronously, persists their
@@ -96,9 +106,15 @@ max_concurrent: 2 # semaphore limit (2 for laptop, 4 for CI)
96
106
  # about, so an omission stops the run instead of quietly changing who reviewed.
97
107
  # effort is optional. role_label is used in consensus output.
98
108
  #
99
- # Default effort per reviewer (medium baseline). When the multi_llm_review
100
- # tool receives a `complexity` argument (or auto-detects it), these defaults
101
- # are overridden per-dispatch by the effort_map below.
109
+ # Default effort per reviewer (high baseline since 2026-09-05). When the
110
+ # multi_llm_review tool receives a `complexity` argument (or auto-detects it),
111
+ # these defaults are overridden per-dispatch by the effort_map below — so the
112
+ # effort_map is the value that actually reaches the CLI, and both are set to
113
+ # high. A roster entry left at medium here would be silently raised by the map.
114
+ #
115
+ # Cursor entries carry no `effort` key on purpose: cursor_adapter has no effort
116
+ # handling at all, so the key would be accepted here and dropped at dispatch,
117
+ # recording an effort the run never used.
102
118
  reviewers:
103
119
  # Rotating orchestrator slots (Opus 5 / Fable 5, 2026-07-25).
104
120
  # Both frontier Claude models sit in the roster. Whichever one is the
@@ -112,7 +128,7 @@ reviewers:
112
128
  # PersonaAssembly at collect time.
113
129
  - provider: claude_code
114
130
  model: claude-opus-5
115
- effort: medium
131
+ effort: high
116
132
  role_label: claude_cli_opus5
117
133
 
118
134
  # Fable 5 retired from the roster 2026-07-26 after five consecutive
@@ -130,7 +146,7 @@ reviewers:
130
146
  # anchor on the Claude side, deliberately not a frontier model.
131
147
  - provider: claude_code
132
148
  model: claude-opus-4-6
133
- effort: medium
149
+ effort: high
134
150
  role_label: claude_cli_opus4.6
135
151
 
136
152
  # Opus 4.8 retired 2026-07-25 when Opus 5 entered the roster (same
@@ -142,28 +158,43 @@ reviewers:
142
158
  # (a)/(b)/(c) breakdowns per round (see multi_llm_reviewer_evaluation)
143
159
  # until a profile accumulates.
144
160
 
145
- # GPT-5.6-sol added 2026-07-23, replacing gpt-5.4 (retired same day; to
146
- # restore, set model back to gpt-5.4). 5.6-sol's reviewer bias profile is
147
- # not yet calibrated record (a)/(b)/(c) breakdowns per round (see
148
- # multi_llm_reviewer_evaluation) until a profile accumulates. gpt-5.5
149
- # stays as the calibrated anchor for cross-generation comparison.
161
+ # GPT-6-astra added 2026-09-05 at the operator's instruction, replacing
162
+ # gpt-5.6-sol (which had held this slot since 2026-07-23, itself replacing
163
+ # gpt-5.4). To restore either, set model and role_label back. Verified before
164
+ # the swap through the flags this adapter actually builds:
165
+ # codex exec --sandbox read-only -m gpt-6-astra -c model_reasoning_effort=high
166
+ # answered with `model: gpt-6-astra` / `reasoning effort: high` in its header.
167
+ #
168
+ # gpt-6-astra's reviewer bias profile is not calibrated — record (a)/(b)/(c)
169
+ # breakdowns per round (see multi_llm_reviewer_evaluation) until one
170
+ # accumulates. The 138-run profile recorded for codex_gpt5.6-sol in L1
171
+ # multi_llm_review_workflow belongs to the retired model and does NOT carry
172
+ # over to this slot. Since gpt-5.5 was retired the same day, codex holds ONE
173
+ # slot and it is uncalibrated — there is no longer a calibrated codex anchor
174
+ # to read a new seat against.
150
175
  - provider: codex
151
- model: gpt-5.6-sol
152
- effort: medium
153
- role_label: codex_gpt5.6-sol
176
+ model: gpt-6-astra
177
+ effort: high
178
+ role_label: codex_gpt6-astra
154
179
 
155
- # GPT-5.5 added 2026-04-24. Both codex entries dispatch via the same
156
- # `codex exec` CLI; the adapter passes
157
- # `-m <model>` so the response can be attributed per-model.
180
+ # GPT-5.5 (added 2026-04-24) RETIRED 2026-09-05 at the operator's
181
+ # instruction. It had held the second codex slot as the calibrated
182
+ # cross-generation anchor. In practice it had not run since 2026-07-30, when
183
+ # it was commented out of the instance roster "for round R10 only" and never
184
+ # restored — so the anchor role it was kept for had already lapsed for five
185
+ # weeks before this made the lapse explicit. The roster drops from 5 to 4 and
186
+ # codex now holds one slot, gpt-6-astra.
158
187
  #
159
- # Transport backend (2026-07-03): codex entries default to the `codex exec` CLI.
160
- # Add `backend: mcp` to route a codex reviewer through `codex mcp-server` instead
161
- # (CodexMcpAdapter, read-only sandbox). Opt-in for A/B against the CLI path;
162
- # keep the default CLI until MCP parity is confirmed (see log 20260702 Appendix F).
163
- - provider: codex
164
- model: gpt-5.5
165
- effort: medium
166
- role_label: codex_gpt5.5
188
+ # To restore, re-add an entry with model gpt-5.5, effort high, role_label
189
+ # codex_gpt5.5, and widen convergence_rule back to a 5-seat basis
190
+ # (ceil(5 * 0.6) = 3, "3/5 APPROVE"; after exclusion "3/4 APPROVE").
191
+ #
192
+ # Transport backend (2026-07-03), retained for whichever codex entries exist:
193
+ # they default to the `codex exec` CLI, and the adapter passes `-m <model>` so
194
+ # the response can be attributed per-model. Add `backend: mcp` to route a codex
195
+ # reviewer through `codex mcp-server` instead (CodexMcpAdapter, read-only
196
+ # sandbox). Opt-in for A/B against the CLI path; keep the default CLI until MCP
197
+ # parity is confirmed (see log 20260702 Appendix F).
167
198
 
168
199
  # Model pinned explicitly 2026-07-27. This entry previously carried no
169
200
  # `model` key and relied on the cursor CLI default, which is user-editable
@@ -222,7 +253,7 @@ reviewers:
222
253
  escalation_reviewers:
223
254
  - provider: claude_code
224
255
  model: claude-fable-5
225
- effort: medium
256
+ effort: high
226
257
  role_label: claude_cli_fable5
227
258
 
228
259
  # INV-E2 (substance) has no setting here on purpose. A reply counts as a review
@@ -237,31 +268,45 @@ escalation_reviewers:
237
268
  # or accepts an explicit `complexity` input. The mapped effort overrides the
238
269
  # reviewer roster's default.
239
270
  #
240
- # Policy (2026-04-29): Reviewers stay at each LLM's DEFAULT effort regardless of
241
- # review_type or artifact size. Cross-evaluation experiments showed reviewer
242
- # verdict quality is effort-independent. Pushing reviewers above default wastes
243
- # cost without quality gain. High effort is reserved for the post-aggregation
244
- # REVISION phase (separate orchestrator action, not this SkillSet).
271
+ # Policy (2026-09-05, operator instruction): every reviewer that HAS an effort
272
+ # control runs at high, at every complexity level. Effort is therefore a
273
+ # constant here, not a function of complexity; the map is kept rather than
274
+ # deleted because deleting it would silently hand control back to the roster
275
+ # entries, which is a second place to forget.
276
+ #
277
+ # This SUPERSEDES the 2026-04-29 policy ("reviewers stay at each LLM's default
278
+ # effort; verdict quality is effort-independent"). That policy rested on a
279
+ # cross-evaluation measurement of low vs high scoring 8.35 vs 8.16 — taken on
280
+ # the Opus 4.6 / 4.7 generation, on none of the models now in the roster. It was
281
+ # not re-measured against Opus 5 or gpt-6-astra, so it is superseded on the
282
+ # operator's judgement rather than on a counter-measurement. Cost rises:
283
+ # gpt-6-astra spent 8,274 tokens at high vs 3,150 at its default on the same
284
+ # one-line prompt (measured 2026-09-05).
285
+ #
286
+ # `cursor` deliberately has no key at any level. cursor_adapter builds no effort
287
+ # flag, so a value here would be recorded and never sent.
245
288
  effort_map:
246
289
  low:
247
- claude_code: medium # was: low — collapsed to default per new policy
248
- codex: medium
290
+ claude_code: high
291
+ codex: high
249
292
  medium:
250
- claude_code: medium
251
- codex: medium
293
+ claude_code: high
294
+ codex: high
252
295
  high:
253
- claude_code: medium # was: high — reviewers stay at default
254
- codex: medium
296
+ claude_code: high
297
+ codex: high
255
298
  critical:
256
- claude_code: medium # was: high reviewers stay at default. critical work
257
- # belongs in revision phase (high+directive+anti-enum),
258
- # not in reviewer effort.
259
- codex: medium
299
+ claude_code: high # `high` is the ceiling the two providers share: claude
300
+ # accepts low/medium/high/xhigh/max, codex accepts
301
+ # minimal/low/medium/high. Raising this level alone
302
+ # would split the roster across incomparable settings.
303
+ codex: high
260
304
 
261
305
  # Auto-detection thresholds (used when complexity arg not given)
262
- # Note: per the policy above, all complexity levels currently map to medium for
263
- # reviewers. The thresholds remain so that explicit `complexity` inputs and
264
- # future per-pool overrides (e.g. revision-phase callers) can still differentiate.
306
+ # Note: per the policy above, all complexity levels currently map to high for
307
+ # reviewers, so detected complexity changes nothing about effort today. The
308
+ # thresholds remain so that explicit `complexity` inputs and future per-pool
309
+ # overrides (e.g. revision-phase callers) can still differentiate.
265
310
  auto_complexity:
266
311
  small_artifact_chars: 500
267
312
  large_artifact_chars: 5000
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: kairos-chain
3
3
  version: !ruby/object:Gem::Version
4
- version: 3.84.0
4
+ version: 3.85.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Masaomi Hatakeyama