kairos-chain 3.62.0 → 3.63.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d07ef9eddf8433aed348e6ff25d910547da041e89abdf94dc6e5008f2148f417
4
- data.tar.gz: bddb2825ffb56e57681be7a1c368e683103133dad90888dff052a8fc5c430ae8
3
+ metadata.gz: 96a6f46091b47614f44d089e6459dde7e8435cf7caa893c9b335b7359ad25394
4
+ data.tar.gz: 965451d952fb8fa61d3bfcc581241242de0f25eb492dddc56ebe913e4d07683b
5
5
  SHA512:
6
- metadata.gz: f432b43e89098c0d794b8e96d8241009802a6ca41e71804ab77f0ad73abaac07aa0b413889f5d84f0425e22bfddcc6f9fa9a3cbc968196bd6ddbda843e89d1f1
7
- data.tar.gz: ff82bd049b30fd06ba14d529ff3910ca9d540c071f441519c23f0189619b90a1b1bfa072449713ba8b53bd575cd5dde763eed6022d141465849f8b6769e5e862
6
+ metadata.gz: 1c3812249637a923526a17b29cbbf34549f3f3039f51ae37941fa990ceec99611d68e88de280b39ff673aaf5f64e55be3133b079578b9f2887df8a6a6c81f102
7
+ data.tar.gz: a3688710826e2de6bd2d05ee24b0f9c6194e32e6d9dd8587c776b5e3bfa1b614549af903eee8a9bac295b5e0efc36c5e50537d5a54b8352acd40e05f2ef291c1
data/CHANGELOG.md CHANGED
@@ -4,6 +4,46 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
4
4
 
5
5
  This project follows [Semantic Versioning](https://semver.org/).
6
6
 
7
+ ## [3.63.0] - 2026-08-06
8
+
9
+ ### Changed
10
+
11
+ - **`multi_llm_review` findings gain a weight axis, and a worker death no
12
+ longer discards completed seats** (SkillSet 0.9.1 → 0.10.0, frozen after one
13
+ review round; every deployment-grounded finding fixed in-round).
14
+
15
+ The severity of a finding said what kind of defect it was, never what it
16
+ costs: measured on a three-persona panel, 3 of one round's 7 P0s were
17
+ factually correct findings that cost nobody anything, landing at the same
18
+ severity as a defect that silently corrupts published output. The reviewer
19
+ prompt contract now requires a `[consequence: who is harmed, and how]`
20
+ clause on every P0, and aggregation records an unclaused P0 at P2 with the
21
+ stated severity and the demotion reason kept beside it. Only presence is
22
+ checked, mechanically; whether a stated consequence is real or trivial
23
+ stays the orchestrator's call. The clause is read before the byte bound
24
+ cuts the tail, the first non-empty clause counts, deduplication compares
25
+ issues without their clauses, and the demotion mark survives merging in
26
+ any member order. Reviewer prompts also stop naming the round number, and
27
+ the L1 workflow knowledge (v3.10.0) adds the matching orchestrator-side
28
+ rule: never tell a reviewer its finding count, the round number, or prior
29
+ verdicts — counts a reviewer performs select for finding-production over
30
+ finding-weight.
31
+
32
+ Separately, one stuck seat used to cost a whole round: a stale heartbeat
33
+ killed the detached worker and every completed seat's reply died with it,
34
+ because nothing left the worker's memory until all seats were done. The
35
+ worker now persists each seat's outcome the moment it is decided
36
+ (`partial_results.json`, atomic, single writer — including the
37
+ dispatcher's own deadline skips, so recovery cannot relabel a reached
38
+ seat as lost). The collect tool's crash and timeout branches recover the
39
+ completed seats, but only after the reaper CONFIRMS the worker's death: a
40
+ stale heartbeat is not a death certificate, and a record sealed over a
41
+ live worker would be permanently false once idempotent replay pins it.
42
+ Unconfirmed death falls back to the previous retryable total-loss report.
43
+ Seats the worker never reached enter the denominator as skip rows named
44
+ `worker_crashed_seat_lost`, and the payload carries `worker_failure`
45
+ naming the death and the recovered and lost seat labels.
46
+
7
47
  ## [3.62.0] - 2026-08-06
8
48
 
9
49
  ### Changed
@@ -1,4 +1,4 @@
1
1
  module KairosMcp
2
- VERSION = "3.62.0"
2
+ VERSION = "3.63.0"
3
3
  CHANGELOG_URL = "https://github.com/masaomi/KairosChain_2026/blob/main/CHANGELOG.md"
4
4
  end
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: multi_llm_review_workflow
3
3
  description: "Multi-LLM review methodology and execution — workflow pattern, CLI tooling, consensus analysis, Persona Assembly. Applicable to design, implementation, documentation, or any artifact."
4
- version: "3.8.0"
4
+ version: "3.10.0"
5
5
  tags:
6
6
  - workflow
7
7
  - review
@@ -498,9 +498,58 @@ The user always has the final say.
498
498
  `open-question` or `defect` in the round's L2 record
499
499
  If no (a)/(b) blocking findings → proceed to next phase
500
500
  If any (a)/(b) finding → repeat from [2] with revised artifact
501
+ (the revision obeys § Revision Discipline below)
501
502
  (c) findings are recorded as advisory; non-blocking
502
503
  ```
503
504
 
505
+ ## Revision Discipline (between rounds)
506
+
507
+ > Evidence base: 35 recorded runs across 8 threads, 2026-08-03 → 08-06
508
+ > (tokens in `.kairos/multi_llm_review/pending/`), two of which ran to
509
+ > convergence. Same validation caveat as Step -1: a strong regularity in one
510
+ > instance's logs, not yet reproduced elsewhere. Analysis record: L2
511
+ > `mlr_p0_inflation_analysis_and_opus46_no_verdict_diagnosis_20260806`.
512
+
513
+ The strongest predictor of round N+1's raw P0 count in those logs is whether
514
+ the round-N revision **added mechanism** to the artifact — not the artifact's
515
+ size, not reviewer strictness:
516
+
517
+ - chain_history_erasure v0.5 added two invariants and a recount section
518
+ (draft 9.3k → 19.1k chars): raw P0 went 13 → 38, and 17 of the 38 targeted
519
+ the added or rewritten sections. v0.8, similar in size (18.6k) but authored
520
+ under an explicit "no new mechanism" rule, closed at 13 with one external
521
+ slot finding zero P0s.
522
+ - mlr_evidence_fix R1's fix added four unrequested defensive mechanisms; R2
523
+ returned 41 findings, nearly all of them defects inside the additions. All
524
+ four mechanisms were later removed, each for a measured reason.
525
+ - Deletions never generated findings: chain erasure v0.7 deleted three
526
+ mechanisms — zero new P0s against the deletions, confirmed in writing by
527
+ three slots.
528
+ - Each thread converged within 1–2 rounds of switching to subtractive
529
+ revisions; neither converged while revisions were additive.
530
+
531
+ Rules:
532
+
533
+ 1. **A revision closes findings by deletion, correction, or naming — never by
534
+ default-adding.** New mechanisms, invariants, sections, or defensive
535
+ layers do not enter a revision unprompted. If a finding appears to require
536
+ new mechanism, put the question to the operator ("finding X seems to need
537
+ mechanism Y — add, defer to backlog, or drop?") before drafting it in.
538
+ Explanatory prose is a lighter form of the same risk: a sentence added
539
+ only to justify a retreat became the sole blocking finding of the round
540
+ that followed it (mlr_evidence R6 — "adding an explanation creates a new
541
+ claim").
542
+ 2. **Prefer a revision author that is not the model whose additions are under
543
+ review.** This extends the existing separation principle — the deciding
544
+ context never authors what judges it — from verification to revision.
545
+ Opus 5 has a measured additive propensity (scope-broadening; the v0.5
546
+ explosion above), but the model is the pressure, not the cause: Fable 5
547
+ also added-and-broke (v0.6's fsync/realpath additions, v0.7's predicate 4
548
+ — a fatal genesis-rejecting defect) until the subtractive rule was
549
+ imposed, and Opus 5 converged mlr_evidence once its revisions became
550
+ subtractive (retreat + removal). Combine both levers: the subtractive
551
+ rule always, a different-model reviser when available.
552
+
504
553
  ## Review Types
505
554
 
506
555
  | Type | Focus | Reviewers See | Typical Use |
@@ -548,6 +597,19 @@ and philosophy-aligned finding has been answered is converged whether or not the
548
597
  numerator moved. Do not treat a reached ratio as sufficient on its own either:
549
598
  check what the approving replies actually said before counting them.
550
599
 
600
+ **Count carryover and new (a)/(b) P0s separately; the machine-side signal of
601
+ convergence is "new P0 = 0", not the APPROVE ratio.** Require each persona to
602
+ state a closure verdict on its own prior-round P0s — closed / open /
603
+ half-closed, with grounds. This format is validated live (chain erasure
604
+ R6–R8) and is what makes the carryover/new split computable. A round whose
605
+ (a)+(b) findings are all carryover with closure verdicts, and whose revision
606
+ drew zero new P0s (observed without exception when the revision was
607
+ subtractive — see § Revision Discipline), is a freeze candidate for the
608
+ operator regardless of the numerator. Neither of the two 2026-08 threads
609
+ ever reached its APPROVE ratio; both closed by (a)+(b) exhaustion + operator
610
+ freeze declaration — the intended close described above, now with a
611
+ measurable trigger.
612
+
551
613
  **Escalating raises the bar.** The rule is a ratio over the observers that
552
614
  counted, so adding reserve observers with `escalate: true` raises the number of
553
615
  agreements required. That is the intended cost of a wider panel, not a defect —
@@ -1027,6 +1089,26 @@ exclude it would be to exclude every honest terse approval with it. When a round
1027
1089
  reaches its ratio, read what the approving replies actually said before treating
1028
1090
  the ratio as convergence. That judgement is the human's and no rule replaces it.
1029
1091
 
1092
+ #### A no_verdict streak is a seat-environment signal, not a dead reviewer
1093
+
1094
+ Before treating a slot as dead, read its `raw_text_excerpt` / `stated_text`
1095
+ in the pending record. Diagnosed live (2026-08-06, `claude_cli_opus4.6`,
1096
+ four consecutive no_verdict exclusions across one implementation-review
1097
+ thread): the CLI itself was healthy throughout. The subprocess seat runs
1098
+ sandboxed — no tools, empty working directory — so on **implementation**
1099
+ artifacts that cite file paths, the model attempted to read code before
1100
+ judging: two rounds opened with pseudo-tool-call markup, one opened with
1101
+ "the repository is not accessible" and stated its verdict header only
1102
+ further down, where the positional rule correctly refuses it. The same
1103
+ seat, in the same period, complied on **design** artifacts (verdict on
1104
+ line 1, counted every round). Remedies, in order: state in the subprocess
1105
+ prompt that the seat has no file access and must review the artifact text
1106
+ alone, marking unverifiable claims `[INFERRED]` (the grounding_rules block
1107
+ already licenses this); keep prompt rule #6 (full artifact inline) honest
1108
+ for implementation reviews; only then consider `--add-dir` with read-only
1109
+ tools, accepting CLAUDE.md contamination. A streak that survives those
1110
+ remedies is a real outage.
1111
+
1030
1112
  #### Async/Parallel Collect Timing — Iron Rule
1031
1113
 
1032
1114
  When `delegation.parallel.default: true` (the v3.x default), Call 1 returns
@@ -1107,13 +1189,35 @@ Every review prompt MUST include these 7 items:
1107
1189
  1. **Output filename table** — so each reviewer knows where to save
1108
1190
  2. **Auto-execution commands** — ready-to-run CLI per reviewer
1109
1191
  3. **Review instructions** — what to focus on, what NOT to re-review
1110
- 4. **Review history** (R2+) — table of previous rounds and findings
1192
+ 4. **Prior findings to verify** (R2+) — the findings the revision addresses,
1193
+ so the reviewer can judge each closed / open / half-closed. Findings only:
1194
+ no per-reviewer verdict history, no finding counts, no round tallies
1195
+ (see § Reviewer incentive rule)
1111
1196
  5. **Context** — architecture summary for reviewers unfamiliar with codebase
1112
1197
  6. **Full artifact content inline** — reviewers may not have file access
1113
1198
  7. **Severity ratings + output format** — structured template for review output
1114
1199
 
1115
1200
  All prompt content MUST be in **English** for consistent parsing across LLM tools.
1116
1201
 
1202
+ ### Reviewer incentive rule
1203
+
1204
+ **Never tell a reviewer — subprocess or persona — that its finding count is
1205
+ compared across rounds, which round this is, or what verdicts were given
1206
+ before.** A reviewer told its count is watched treats the count as the
1207
+ deliverable, and that selects for finding-*production* over finding-*weight*
1208
+ (observed 2026-08-06: an orchestrator wrote "your finding count is compared
1209
+ across rounds" into persona prompts during the project_orientation_report
1210
+ loop; of the round's 7 P0s, 3 were factually correct findings that cost
1211
+ nobody anything). Convergence — carryover vs new, (a)+(b) exhaustion, the
1212
+ ratio — is measured by the orchestrator from the record, after the replies
1213
+ are in. The reviewer receives the artifact, the review criteria, and the
1214
+ prior findings it must verify. Nothing else about the loop's state.
1215
+
1216
+ What this rule does NOT forbid: passing prior findings for closure
1217
+ verification (rule #4 — that is content, not score-keeping), and the
1218
+ carryover/new split in § Convergence Rules (that is orchestrator-side
1219
+ bookkeeping the reviewer never sees).
1220
+
1117
1221
  ### XML Block Structure for Review Prompts
1118
1222
 
1119
1223
  Review prompts SHOULD use XML blocks to give LLMs explicit structural contracts.
@@ -1432,5 +1536,43 @@ Compression ratio: parallel agent raw → Assembly ≈ 2:1
1432
1536
  fence markers, character classes, digit ranges, word boundaries — and 18 of
1433
1537
  27 survived. Mutate the inside of a pattern, not only the pattern.
1434
1538
 
1539
+ - Revision discipline, new-P0 convergence signal, and no_verdict seat
1540
+ diagnosis (v3.9.0, 2026-08-06): cross-thread analysis of 35 recorded runs
1541
+ (8 threads, 2026-08-03 → 08-06) established that raw P0 growth tracks
1542
+ additive revisions, not reviewer severity — every mechanism a revision
1543
+ added became the next round's battleground, deletions drew zero new P0s
1544
+ in every measured case, and both threads that converged did so within
1545
+ 1–2 rounds of switching to subtractive revisions (one under a Fable 5
1546
+ reviser, one under the same Opus 5 orchestrator that had produced the
1547
+ additive explosion). New § Revision Discipline encodes the subtractive
1548
+ rule and the reviser-separation preference. § Convergence Rules gains the
1549
+ carryover/new P0 split, with "new (a)+(b) P0 = 0" as the machine-side
1550
+ freeze-candidate signal — neither thread ever reached its APPROVE ratio;
1551
+ both closed by (a)+(b) exhaustion + operator freeze. § Substance and the
1552
+ denominator gains the seat-environment diagnosis: `claude_cli_opus4.6`'s
1553
+ four-round no_verdict streak was the sandboxed seat colliding with
1554
+ implementation artifacts (pseudo-tool-calls, "repository not accessible"
1555
+ preamble), not a dead reviewer — the same seat counted every round on
1556
+ design artifacts in the same period. Analysis record: L2
1557
+ `mlr_p0_inflation_analysis_and_opus46_no_verdict_diagnosis_20260806`
1558
+
1559
+ - Reviewer incentive rule and the finding weight axis (v3.10.0, 2026-08-06):
1560
+ § Prompt Generation Rules gains the Reviewer incentive rule — reviewer
1561
+ prompts never mention finding counts, round numbers, or prior verdicts;
1562
+ convergence is measured orchestrator-side from the record, and prior
1563
+ findings are passed for closure verification only (rule #4 reworded
1564
+ accordingly, from "review history table" to "prior findings to verify").
1565
+ Motivating observation: an orchestrator told personas their counts were
1566
+ compared across rounds, and 3 of the round's 7 P0s were factually correct
1567
+ findings that cost nobody anything. In the same change the SkillSet
1568
+ (0.10.0) adds the weight axis mechanically: the prompt contract requires a
1569
+ `[consequence: who is harmed, and how]` clause on every P0, and
1570
+ aggregation records a P0 without one at P2, keeping the stated severity
1571
+ and the demotion reason beside it (`severity_stated` /
1572
+ `severity_demoted: consequence_missing`). Presence is checked
1573
+ mechanically; whether a stated consequence is real or trivial stays the
1574
+ orchestrator's call, per the (a)/(b)/(c) discipline. Handoff record: L2
1575
+ `handoff_mlr_finding_weight_axis_and_reviewer_incentive_20260806`
1576
+
1435
1577
  **Key insight**: Design reviews and implementation reviews find
1436
1578
  **categorically different bugs**. Both phases are necessary.
@@ -259,12 +259,29 @@ begin
259
259
  # and on between-reviewer progress (counter advances when result arrives).
260
260
  # v0.3.0 PR3 pushes MainState ticks via a per-result hook below.
261
261
 
262
+ # Per-seat persistence: each reply is written the moment it arrives, so a
263
+ # worker death with one seat stuck leaves the finished seats recoverable
264
+ # (collect's crash/timeout recovery reads partial_results.json). Runs on
265
+ # the dispatch collecting thread — the worker stays this file's single
266
+ # writer (§6.3). subprocess_results.json still supersedes it on clean exit.
267
+ partial_by_index = {}
268
+ on_result = lambda do |idx, result|
269
+ partial_by_index[idx.to_s] = MLR::ReviewSerializer.serialize(result)
270
+ PS.write_partial_results(token, {
271
+ 'schema_version' => 1,
272
+ 'token' => token,
273
+ 'updated_at' => Time.now.iso8601,
274
+ 'results_by_index' => partial_by_index
275
+ })
276
+ end
277
+
262
278
  results = dispatcher.dispatch(
263
279
  (request['reviewers'] || []).map { |r| r.transform_keys(&:to_sym) },
264
280
  request['messages'] || [],
265
281
  request['system_prompt'] || '',
266
282
  context: nil,
267
- review_context: request['review_context'] || 'independent'
283
+ review_context: request['review_context'] || 'independent',
284
+ on_result: on_result
268
285
  )
269
286
 
270
287
  # v3.24.3: counter-only signal (no enter_call!/exit_call! pair). bump_counter!
@@ -287,6 +304,15 @@ begin
287
304
  }
288
305
  }
289
306
  PS.write_subprocess_results(token, payload)
307
+ # The partial file is superseded the moment the full results land; left
308
+ # behind, it is a stale copy of reviewer replies sitting beside the record
309
+ # forever (completed token dirs are never reaped). Removed on the clean
310
+ # path only — on a crash it IS the record collect recovers from.
311
+ begin
312
+ File.unlink(PS.partial_results_path(token))
313
+ rescue Errno::ENOENT
314
+ nil
315
+ end
290
316
  PS.transition_to_terminal!(token, 'done')
291
317
  exit 0
292
318
  rescue StandardError => e
@@ -601,6 +601,15 @@ module KairosMcp
601
601
  end
602
602
  end
603
603
 
604
+ # The consequence clause a finding carries, when it carries one. The
605
+ # severity axis says what KIND of defect this is; the consequence
606
+ # clause is the WEIGHT axis — who is harmed, and how, if it is never
607
+ # fixed. Measured 2026-08-06 (project_orientation_report checker,
608
+ # R5): of 7 P0s, 3 were factually correct findings that cost nobody
609
+ # anything, and the two kinds landed at the same severity because the
610
+ # record had nowhere to say the difference.
611
+ CONSEQUENCE_RE = /\[\s*consequence:\s*([^\]]*)\]/i
612
+
604
613
  # Collect severity-tagged findings from all successful reviews.
605
614
  # Deduplicates by first 80 chars (case-insensitive).
606
615
  def self.aggregate_findings(parsed_verdicts)
@@ -611,23 +620,7 @@ module KairosMcp
611
620
 
612
621
  # Extract "P0: ...", "P1-1: ...", "**P0**:", etc.
613
622
  text.scan(/\*{0,2}(P[0-3])\*{0,2}[-\s]*\d*[.:]\s*(.+?)(?=\n\s*\n|\n\s*\*{0,2}P[0-3]|\z)/mi) do |sev, issue|
614
- all_findings << {
615
- severity: sev.upcase,
616
- # This used to be `[0..200]`, an inclusive Range, so every
617
- # finding longer than 201 characters lost its tail — the quoted
618
- # line, the file:line, the failure condition — before anything
619
- # downstream could bound it on purpose.
620
- #
621
- # It is bounded here rather than nowhere, and here rather than
622
- # further upstream, because this is the point where both costs
623
- # are decided at once: what gets stored, and what the sanitizer
624
- # is about to normalise character by character. A bound placed
625
- # on the reply instead was measured not to work, since NFKC runs
626
- # between the two and expands by up to 11x. The bound is in
627
- # bytes for the same reason — bytes are what is spent.
628
- issue: Sanitizer.clamp_finding_bytes(issue.strip),
629
- cited_by: [r[:role_label]]
630
- }
623
+ all_findings << build_finding(sev, issue, r[:role_label])
631
624
  end
632
625
  end
633
626
 
@@ -659,7 +652,7 @@ module KairosMcp
659
652
  # (`issue_variants_omitted`), because a silently shortened list
660
653
  # reads as "this is all there was" — the failure this whole change
661
654
  # exists to remove.
662
- grouped = all_findings.group_by { |f| f[:issue][0..79].downcase }
655
+ grouped = all_findings.group_by { |f| dedup_key(f[:issue]) }
663
656
  grouped.map do |_key, findings|
664
657
  merged_severity = findings.map { |f| f[:severity] }.min # P0 < P1 < P2
665
658
  # `.min` over a non-empty group returns one of its own members, so
@@ -672,6 +665,25 @@ module KairosMcp
672
665
  issue: representative[:issue],
673
666
  cited_by: findings.flat_map { |f| f[:cited_by] }.uniq
674
667
  }
668
+ # The representative's wording and its weight travel together, like
669
+ # its severity does. When the representative states no consequence,
670
+ # another member's is carried rather than none: a group where ONE
671
+ # reviewer said who is harmed is a finding whose harm is known.
672
+ consequence = representative[:consequence] ||
673
+ findings.map { |f| f[:consequence] }.compact.first
674
+ row[:consequence] = consequence if consequence
675
+ # The demotion mark is carried from ANY member, not only the
676
+ # representative — copied from the representative alone it
677
+ # vanished order-dependently whenever a non-demoted member of the
678
+ # same severity happened to sort first (R1 finding, three seats
679
+ # independently). A row that merged to P0 carries no mark: some
680
+ # member stated the harm, the row holds its full severity, and
681
+ # nothing was demoted AWAY from what the record shows.
682
+ if row[:severity] != 'P0' &&
683
+ (demoted = findings.find { |f| f[:severity_stated] })
684
+ row[:severity_stated] = demoted[:severity_stated]
685
+ row[:severity_demoted] = demoted[:severity_demoted]
686
+ end
675
687
  if variants.size > 1
676
688
  row[:issue_variants] = variants.first(MAX_ISSUE_VARIANTS)
677
689
  omitted = variants.size - MAX_ISSUE_VARIANTS
@@ -680,6 +692,68 @@ module KairosMcp
680
692
  row
681
693
  end.sort_by { |f| f[:severity] }
682
694
  end
695
+
696
+ # The dedup key is the issue WITHOUT its consequence clause. The clause
697
+ # stays in the issue text (display, replay), but two reviewers naming
698
+ # one defect — one saying who is harmed, one not — are still one
699
+ # finding, and the merge is what lets the stated consequence carry the
700
+ # row's severity for both. Keyed on the raw text instead, the clause
701
+ # lands inside the first 80 characters of any short issue and splits
702
+ # the group, moving the finding count and the convergence denominator.
703
+ # Whitespace is normalised for the same reason: removing the clause
704
+ # must not leave a gap that fails the comparison it was removed for.
705
+ def self.dedup_key(issue)
706
+ issue.gsub(CONSEQUENCE_RE, ' ').gsub(/\s+/, ' ').strip[0..79].downcase
707
+ end
708
+
709
+ # One extracted finding, weight axis applied at the point of entry.
710
+ #
711
+ # A P0 that does not say who is harmed is recorded at P2, with the
712
+ # stated severity and the demotion reason kept beside it — the record
713
+ # says what the reviewer wrote AND what the rule did with it. Only
714
+ # PRESENCE is checked, mechanically, by construction: whether a stated
715
+ # consequence is real or trivial is a judgment call, and it belongs to
716
+ # the orchestrator reading the record, not to a heuristic here — the
717
+ # same division of labour as the substance rule above, which asks
718
+ # "said anything?" and never "said anything good?".
719
+ #
720
+ # The clause is copied into its own field but NOT stripped from the
721
+ # issue text: the display shows the issue line, a later round's prompt
722
+ # replays it, and the consequence should survive in both. (The dedup
723
+ # key, above, strips the clause before comparing — the one place the
724
+ # issue is read WITHOUT it.)
725
+ def self.build_finding(sev, issue, role_label)
726
+ raw = issue.strip
727
+ # The clause is read from the UNCLAMPED text, before the byte bound
728
+ # cuts the tail — a finding long enough to lose its closing bracket
729
+ # to the clamp would otherwise read as having stated nothing, and
730
+ # the demotion would bury exactly the P0 whose author complied (R1
731
+ # finding, three seats). Scan rather than match, first NON-empty
732
+ # clause: "[consequence: ] ... [consequence: real harm]" states a
733
+ # harm, and reading only the first bracket pair would demote past it.
734
+ consequence = raw.scan(CONSEQUENCE_RE)
735
+ .map { |c| c[0].strip }
736
+ .find { |c| !c.empty? }
737
+ # Bounded in BYTES at the point of extraction — this is the clamp
738
+ # that bounds the sanitizer's input, and the reasoning for bytes
739
+ # (NFKC expands up to 11x downstream) is at clamp_finding_bytes.
740
+ # This used to be `[0..200]`, an inclusive Range, so every finding
741
+ # longer than 201 characters lost its tail before anything
742
+ # downstream could bound it on purpose.
743
+ finding = {
744
+ severity: sev.upcase,
745
+ issue: Sanitizer.clamp_finding_bytes(raw),
746
+ cited_by: [role_label]
747
+ }
748
+ if consequence
749
+ finding[:consequence] = Sanitizer.clamp_finding_bytes(consequence)
750
+ elsif finding[:severity] == 'P0'
751
+ finding[:severity] = 'P2'
752
+ finding[:severity_stated] = 'P0'
753
+ finding[:severity_demoted] = 'consequence_missing'
754
+ end
755
+ finding
756
+ end
683
757
  end
684
758
  end
685
759
  end
@@ -33,9 +33,17 @@ module KairosMcp
33
33
  # @param system_prompt [String] system prompt for llm_call
34
34
  # @param context [InvocationContext] for invoke_tool
35
35
  # @param review_context [String] 'independent' or 'project_aware'
36
+ # @param on_result [#call, nil] called with (index, result) for every
37
+ # seat outcome this dispatch decides — arrived replies as they
38
+ # arrive, and the dispatch_timeout skips synthesized at the
39
+ # deadline. The caller uses this to persist each seat's TRUE
40
+ # outcome, so a later crash recovery reads "this seat timed out"
41
+ # rather than relabelling a reached seat as lost (R1 finding). Runs
42
+ # on the collecting thread; a hook failure is logged and never
43
+ # fails the dispatch.
36
44
  # @return [Array<Hash>] results indexed by reviewer position
37
45
  def dispatch(reviewers, messages, system_prompt, context:,
38
- review_context: 'independent')
46
+ review_context: 'independent', on_result: nil)
39
47
  dispatch_id = SecureRandom.hex(8)
40
48
  deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + @timeout
41
49
  results = Array.new(reviewers.size)
@@ -94,11 +102,17 @@ module KairosMcp
94
102
  i, result = entry
95
103
  results[i] = result
96
104
  collected += 1
105
+ notify_result(on_result, i, result)
97
106
  end
98
- # Mark uncollected as timed out
107
+ # Mark uncollected as timed out. Notified like an arrived
108
+ # reply: the skip IS this dispatch's decision about the seat,
109
+ # and a persisted record that omits it lets a crash recovery
110
+ # relabel a reached-and-timed-out seat as one the worker never
111
+ # got to.
99
112
  reviewers.each_with_index do |r, i|
100
113
  next if results[i]
101
114
  results[i] = build_skip(r, 'dispatch_timeout')
115
+ notify_result(on_result, i, results[i])
102
116
  end
103
117
  break
104
118
  end
@@ -111,6 +125,7 @@ module KairosMcp
111
125
  i, result = entry
112
126
  results[i] = result
113
127
  collected += 1
128
+ notify_result(on_result, i, result)
114
129
  end
115
130
 
116
131
  # Kill in-flight subprocesses from this dispatch
@@ -136,6 +151,15 @@ module KairosMcp
136
151
 
137
152
  private
138
153
 
154
+ # A hook failure must not fail the dispatch: the hook exists to save
155
+ # replies from a dying worker, and a hook that could kill the dispatch
156
+ # would create the loss it guards against.
157
+ def notify_result(on_result, idx, result)
158
+ on_result&.call(idx, result)
159
+ rescue StandardError => e
160
+ warn "[multi_llm_review::Dispatcher] on_result hook failed: #{e.class}: #{e.message}"
161
+ end
162
+
139
163
  def bump_main_state_counter
140
164
  return unless defined?(KairosMcp::SkillSets::MultiLlmReview::MainState)
141
165
  # v3.24.3: counter-only bump. exit_call! is private; bump_counter!
@@ -72,6 +72,12 @@ module KairosMcp
72
72
  def gc_eligible_path(token); File.join(token_dir(token), 'gc.eligible'); end
73
73
  def request_path(token); File.join(token_dir(token), 'request.json'); end
74
74
  def subprocess_results_path(token); File.join(token_dir(token), 'subprocess_results.json'); end
75
+ # Per-seat results written as each seat completes, so a worker that
76
+ # dies with one seat stuck does not take the finished seats' replies
77
+ # with it (R3 2026-08-06 lost three completed external seats to one
78
+ # stale heartbeat). Superseded by subprocess_results.json on a clean
79
+ # exit; read by collect's crash/timeout recovery path only.
80
+ def partial_results_path(token); File.join(token_dir(token), 'partial_results.json'); end
75
81
  def worker_pid_path(token); File.join(token_dir(token), 'worker.pid'); end
76
82
  def worker_heartbeat_path(token); File.join(token_dir(token), 'worker.heartbeat'); end
77
83
  def worker_tick_path(token); File.join(token_dir(token), 'worker.tick'); end
@@ -105,6 +111,7 @@ module KairosMcp
105
111
  def write_collected(token, data); atomic_write_json(collected_path(token), data); end
106
112
  def write_request(token, data); atomic_write_json(request_path(token), data); end
107
113
  def write_subprocess_results(token, data); atomic_write_json(subprocess_results_path(token), data); end
114
+ def write_partial_results(token, data); atomic_write_json(partial_results_path(token), data); end
108
115
  def write_worker_pid(token, data); atomic_write_json(worker_pid_path(token), data); end
109
116
  def write_marker(token, data); atomic_write_json(marker_path(token), data); end
110
117
  def write_completed(token, data); atomic_write_json(completed_path(token), data); end
@@ -172,6 +179,7 @@ module KairosMcp
172
179
  def load_collected(token); load_json_transient(collected_path(token)); end
173
180
  def load_request(token); load_json_transient(request_path(token)); end
174
181
  def load_subprocess_results(token); load_json_transient(subprocess_results_path(token)); end
182
+ def load_partial_results(token); load_json_transient(partial_results_path(token)); end
175
183
  def load_worker_pid(token); load_json_transient(worker_pid_path(token)); end
176
184
 
177
185
  # Mutate state.json under a read-modify-write block, serialized
@@ -77,7 +77,13 @@ module KairosMcp
77
77
  parts << "<task>"
78
78
  parts << "Review the provided artifact for #{review_type} correctness."
79
79
  parts << "Target: #{artifact_name}"
80
- parts << "Round: R#{review_round}"
80
+ # The round number is deliberately NOT given to the reviewer
81
+ # (2026-08-06). Telling a reviewer which round it is in — like
82
+ # telling it its finding count is compared across rounds — turns the
83
+ # count into something the reviewer performs, and selects for
84
+ # finding-production over finding-weight. Convergence is measured by
85
+ # the orchestrator from the record; the reviewer needs the artifact,
86
+ # the criteria, and the prior findings to verify, nothing else.
81
87
  if review_round > 1 && prior_findings && !prior_findings.empty?
82
88
  parts << "Scope: Review the revisions addressing prior findings."
83
89
  parts << ""
@@ -161,14 +167,22 @@ module KairosMcp
161
167
  line one is read.
162
168
 
163
169
  For each finding, use this single-line format (one finding per line):
164
- P0: <issue description> [location: file:line]
165
- P1: <issue description> [location: file:line]
170
+ P0: <issue description> [consequence: <who is harmed, and how, if this is never fixed>] [location: file:line]
171
+ P1: <issue description> [consequence: ...] [location: file:line]
166
172
  P2: <issue description> [location: file:line]
167
173
  P3: <issue description> [location: file:line]
168
174
 
175
+ The consequence clause is REQUIRED for P0 and recommended for
176
+ P1; P1 findings are never demoted for lacking one. A finding can be
177
+ factually correct and still cost nobody anything; the consequence
178
+ clause is where you say who hits the defect and what happens to
179
+ them. A P0 with no consequence clause, or an empty one, is
180
+ recorded at P2. Do not restate the issue as its own consequence —
181
+ name the person or process that is harmed.
182
+
169
183
  Example:
170
- P0: Missing input validation in dispatcher timeout path [location: dispatcher.rb:120]
171
- P1: Thread safety issue with shared counter [location: consensus.rb:45]
184
+ P0: Missing input validation in dispatcher timeout path [consequence: a caller passing a negative timeout crashes the worker and the whole round's reviews are lost] [location: dispatcher.rb:120]
185
+ P1: Thread safety issue with shared counter [consequence: concurrent collects double-count usage] [location: consensus.rb:45]
172
186
 
173
187
  If no issues found, state "No findings" and verdict APPROVE.
174
188
  </structured_output_contract>
@@ -157,7 +157,8 @@ module KairosMcp
157
157
  # reaches the payload by the same path.
158
158
  #
159
159
  # @param findings [Array<Hash>] finding rows with String keys
160
- # @return [Array<Hash>] the same rows, 'issue' and 'issue_variants' bound
160
+ # @return [Array<Hash>] the same rows, 'issue', 'consequence' and
161
+ # 'issue_variants' bound
161
162
  def self.bound_findings_for_record(findings)
162
163
  findings.map do |f|
163
164
  row = f.merge(
@@ -165,6 +166,15 @@ module KairosMcp
165
166
  sanitize_finding_text(f['issue'], max_len: FINDING_RECORD_MAX_LEN)
166
167
  )
167
168
  )
169
+ if row['consequence']
170
+ # Reviewer text like the issue it was extracted from; it reaches
171
+ # the record by the same path.
172
+ row = row.merge(
173
+ 'consequence' => clamp_finding_bytes(
174
+ sanitize_finding_text(f['consequence'], max_len: FINDING_RECORD_MAX_LEN)
175
+ )
176
+ )
177
+ end
168
178
  if row['issue_variants']
169
179
  row = row.merge(
170
180
  'issue_variants' => Array(row['issue_variants']).map do |v|
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "multi_llm_review",
3
- "version": "0.9.1",
4
- "description": "v0.9.1 (seat access, frozen 2026-08-06 after a one-round review): reviewer prompts carry a <seat_access> block beside the inline artifact — a seat that cannot read the repository must not attempt tool calls and must not open by saying it will read files; it reviews the artifact text alone, marks unverifiable claims [INFERRED], and still opens with its verdict line. The wording is conditional because seats differ (codex runs --sandbox read-only and can read the repository), and the block is not emitted for by_reference delivery, whose existing cannot-read instruction it would contradict. Root cause fixed: the claude subprocess seat runs with tools disabled in an empty working directory, and on implementation artifacts citing file paths it opened with pseudo-tool-call markup or a cannot-access preamble instead of its verdict line, leaving four consecutive rounds as no_verdict while counting every round on design artifacts. v0.9.0 (evidence fidelity, frozen 2026-08-06 after six review rounds): a finding reaches the record whole. It used to be cut at 201 bytes by an inclusive Range in aggregation and bounded again by the 500-character display limit, so a downstream instance measured 18 of 21 findings arriving at exactly 201 bytes with reviews[].raw_text empty on every row. Findings are now bounded in BYTES at FINDING_RECORD_MAX_LEN = 8000 for the record, with DEFAULT_MAX_LEN = 500 still applied by every path that takes a finding into a prompt. Deduplication still keys on the first 80 characters — widening it would stop two reviewers describing one defect from merging, moving the finding count and the convergence denominator — but the surviving text is no longer arbitrary: issue comes from a member whose severity equals the merged severity, distinct texts survive in issue_variants (capped at MAX_ISSUE_VARIANTS = 8, with issue_variants_omitted naming what the cap dropped). Every row now carries raw_text_excerpt unconditionally (4096 bytes) and the reviewer's reply in raw_text on request (include_raw_text, 65536 bytes). Both are SANITISED TRANSCRIPTIONS, not verbatim records, and both tool schemas say so: the text is byte-clamped, NFKC-normalised, stripped of invisible characters, tag-escaped, then byte-clamped again. No field states whether they hold the whole reply, because nothing in this SkillSet can know — a completeness flag was implemented, measured wrong in both directions, and removed. On delegated runs the pending-state record keeps each subprocess reply as it arrived; on single-phase runs the returned payload is the only form there is. Findings carried into a later round's prompt are sanitised and folded to one line, a path that previously took reviewer text into a prompt with no sanitisation at all. v0.8.0 (v0.7 record schema, design frozen 2026-08-01): the verdict vocabulary is the three canonical words plus tense forms only (INV-R1); the ratio and threshold are recorded reference values, not the run's conclusion — the top-level verdict field became reference_verdict and the run is closed by the operator's declaration outside the record (INV-R2); the persona team occupies one seat, its derivation rule is recorded, and a submission smaller than convened — including empty — is accepted with the shortfall on the record (INV-R3/R4); every run writes an existence marker at dispatch, completed records are never garbage-collected, and expired runs are reduced to a minimal trace instead of erased (INV-R4); a divergence-excluded tally is carried beside the main one (INV-R5); the record names its pre-declared spec and carries transport diagnostics as state tags (INV-R6); artifact delivery is a per-seat attribute (inline | by_reference) and an unreachable delivery is refused rather than dispatched (INV-R7). Parallel multi-LLM review orchestration. Dispatches review prompts to N LLM backends via llm_client, collects verdicts, and computes consensus. v0.6.0: reserve observers (escalate) and a declarable persona execution model; the observer set is built in one pass with explicit precedence (ObserverSet); every slot must name its model and role_label, and duplicate names — including the persona team's own — are refused. A reply's verdict is no longer inferred from its prose: it is read from a declared field, from the header the reply opens with when that header carries a verdict name and nothing else, or not at all, in which case the reply leaves the denominator with no_verdict recorded beside its name. The record says why every observer did or did not count (denominator_composition, five skip_reason values, observers_reporting), and the per-reviewer row is written by one mapping rather than two. v0.5.2: the cursor reviewer pins model composer-2.5 instead of inheriting the cursor CLI default, which is operator-editable and had silently become an Anthropic model. v0.5.1: Fable 5 retired from the roster (five consecutive silent returns), convergence 3/5; orchestrator_model description now states the bare-ID rule so a caller does not review its own output. v0.5.0: adds multi_llm_review_wait (Phase 1.5) for explicit subprocess completion gating with next_action recovery hints, and Path A/B doc disambiguation. v0.4.0 (Phase 12): feedback_text + schema_version, sanitization contract for prompt-injection defense, and multi_llm_review_bundle tool for human-handoff paths without dispatch.",
3
+ "version": "0.10.0",
4
+ "description": "v0.10.0 (finding weight axis + worker-death recovery, 2026-08-06): findings gain a consequence axis beside the severity axis — the prompt contract requires a [consequence: who is harmed, and how, if this is never fixed] clause on every P0, and aggregation records a P0 without one at P2, keeping the stated severity and the demotion reason on the row (severity_stated / severity_demoted: consequence_missing). Only PRESENCE is checked, mechanically; whether a stated consequence is real or trivial stays the orchestrator's call. The dedup key strips the clause so two reviewers naming one defect still merge, and a group where one member states the harm carries it for the row. Motivating measurement (project_orientation_report checker, R5): 3 of 7 P0s were factually correct findings that cost nobody anything. Reviewer prompts also stop naming the round number — telling a reviewer which round it is in (like telling it its counts are compared) selects for finding-production over finding-weight; the L1 workflow v3.10.0 Reviewer incentive rule states the orchestrator-side half. And a worker death no longer discards completed seats: the worker persists each reply as it arrives (partial_results.json), and collect's crash/timeout branches recover the finished seats, entering every unreached seat in the denominator as a skip row (worker_crashed_seat_lost) with the death named in payload.worker_failure — R3 2026-08-06 lost three completed external seats to one stale heartbeat because the only exit was total loss. v0.9.1 (seat access, frozen 2026-08-06 after a one-round review): reviewer prompts carry a <seat_access> block beside the inline artifact — a seat that cannot read the repository must not attempt tool calls and must not open by saying it will read files; it reviews the artifact text alone, marks unverifiable claims [INFERRED], and still opens with its verdict line. The wording is conditional because seats differ (codex runs --sandbox read-only and can read the repository), and the block is not emitted for by_reference delivery, whose existing cannot-read instruction it would contradict. Root cause fixed: the claude subprocess seat runs with tools disabled in an empty working directory, and on implementation artifacts citing file paths it opened with pseudo-tool-call markup or a cannot-access preamble instead of its verdict line, leaving four consecutive rounds as no_verdict while counting every round on design artifacts. v0.9.0 (evidence fidelity, frozen 2026-08-06 after six review rounds): a finding reaches the record whole. It used to be cut at 201 bytes by an inclusive Range in aggregation and bounded again by the 500-character display limit, so a downstream instance measured 18 of 21 findings arriving at exactly 201 bytes with reviews[].raw_text empty on every row. Findings are now bounded in BYTES at FINDING_RECORD_MAX_LEN = 8000 for the record, with DEFAULT_MAX_LEN = 500 still applied by every path that takes a finding into a prompt. Deduplication still keys on the first 80 characters — widening it would stop two reviewers describing one defect from merging, moving the finding count and the convergence denominator — but the surviving text is no longer arbitrary: issue comes from a member whose severity equals the merged severity, distinct texts survive in issue_variants (capped at MAX_ISSUE_VARIANTS = 8, with issue_variants_omitted naming what the cap dropped). Every row now carries raw_text_excerpt unconditionally (4096 bytes) and the reviewer's reply in raw_text on request (include_raw_text, 65536 bytes). Both are SANITISED TRANSCRIPTIONS, not verbatim records, and both tool schemas say so: the text is byte-clamped, NFKC-normalised, stripped of invisible characters, tag-escaped, then byte-clamped again. No field states whether they hold the whole reply, because nothing in this SkillSet can know — a completeness flag was implemented, measured wrong in both directions, and removed. On delegated runs the pending-state record keeps each subprocess reply as it arrived; on single-phase runs the returned payload is the only form there is. Findings carried into a later round's prompt are sanitised and folded to one line, a path that previously took reviewer text into a prompt with no sanitisation at all. v0.8.0 (v0.7 record schema, design frozen 2026-08-01): the verdict vocabulary is the three canonical words plus tense forms only (INV-R1); the ratio and threshold are recorded reference values, not the run's conclusion — the top-level verdict field became reference_verdict and the run is closed by the operator's declaration outside the record (INV-R2); the persona team occupies one seat, its derivation rule is recorded, and a submission smaller than convened — including empty — is accepted with the shortfall on the record (INV-R3/R4); every run writes an existence marker at dispatch, completed records are never garbage-collected, and expired runs are reduced to a minimal trace instead of erased (INV-R4); a divergence-excluded tally is carried beside the main one (INV-R5); the record names its pre-declared spec and carries transport diagnostics as state tags (INV-R6); artifact delivery is a per-seat attribute (inline | by_reference) and an unreachable delivery is refused rather than dispatched (INV-R7). Parallel multi-LLM review orchestration. Dispatches review prompts to N LLM backends via llm_client, collects verdicts, and computes consensus. v0.6.0: reserve observers (escalate) and a declarable persona execution model; the observer set is built in one pass with explicit precedence (ObserverSet); every slot must name its model and role_label, and duplicate names — including the persona team's own — are refused. A reply's verdict is no longer inferred from its prose: it is read from a declared field, from the header the reply opens with when that header carries a verdict name and nothing else, or not at all, in which case the reply leaves the denominator with no_verdict recorded beside its name. The record says why every observer did or did not count (denominator_composition, five skip_reason values, observers_reporting), and the per-reviewer row is written by one mapping rather than two. v0.5.2: the cursor reviewer pins model composer-2.5 instead of inheriting the cursor CLI default, which is operator-editable and had silently become an Anthropic model. v0.5.1: Fable 5 retired from the roster (five consecutive silent returns), convergence 3/5; orchestrator_model description now states the bare-ID rule so a caller does not review its own output. v0.5.0: adds multi_llm_review_wait (Phase 1.5) for explicit subprocess completion gating with next_action recovery hints, and Path A/B doc disambiguation. v0.4.0 (Phase 12): feedback_text + schema_version, sanitization contract for prompt-injection defense, and multi_llm_review_bundle tool for human-handoff paths without dispatch.",
5
5
  "author": "Masaomi Hatakeyama",
6
6
  "layer": "L1",
7
7
  "depends_on": [