kairos-chain 3.68.0 → 3.71.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +137 -0
- data/bin/kairos-chain +10 -1
- data/lib/kairos_mcp/version.rb +1 -1
- data/templates/knowledge/multi_llm_review_workflow/assets/review_dashboard.html +332 -0
- data/templates/knowledge/multi_llm_review_workflow/multi_llm_review_workflow.md +39 -2
- data/templates/knowledge/multi_llm_review_workflow/scripts/render_dashboard.rb +55 -0
- data/templates/skillsets/agent/tools/agent_step.rb +7 -1
- data/templates/skillsets/kairos_hook_projector/lib/readable_gate_setup.rb +19 -3
- data/templates/skillsets/kairos_hook_projector/test/test_readable_gate_setup.rb +61 -0
- data/templates/skillsets/minimum_nomic/README.md +51 -0
- data/templates/skillsets/minimum_nomic/bin/mutate.rb +218 -0
- data/templates/skillsets/minimum_nomic/bin/score_detections.rb +134 -0
- data/templates/skillsets/minimum_nomic/report/report_template.html +247 -0
- data/templates/skillsets/minimum_nomic/skillset.json +1 -1
- data/templates/skillsets/multi_llm_review/lib/multi_llm_review/consensus.rb +2 -2
- data/templates/skillsets/multi_llm_review/skillset.json +2 -2
- data/templates/skillsets/multi_llm_review/test/test_multi_llm_review.rb +10 -10
- data/templates/skillsets/multi_llm_review/test/test_mutation_survivors.rb +2 -2
- data/templates/skillsets/multi_llm_review/test/test_observer_set.rb +10 -10
- data/templates/skillsets/multi_llm_review/test/test_observer_set_seams.rb +13 -13
- data/templates/skillsets/multi_llm_review/test/test_tool_wiring.rb +24 -24
- data/templates/skillsets/multi_llm_review/tools/multi_llm_review.rb +2 -2
- data/templates/skillsets/multi_llm_review/tools/multi_llm_review_collect.rb +2 -2
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b8cf3acf9bb95eb62a4fc78d85359540a9089973819fce152278a962007e6faa
|
|
4
|
+
data.tar.gz: bcffe011916e72faecc52aeecc68743a7ef25a96a108b509d9cd80531d04b542
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 759b4cdfe1cc4d6031708d669820994f090d6bf946191d7b3823b9b1f8a3969f9e11193222e8103e065ae20eea290e218638913262ff83225c6250f5f12640b9
|
|
7
|
+
data.tar.gz: 8823dd97245c068d475951c908bcdd77807c43b7b0ce6705680d9413ad52bee608bd246c2a97214df6e0253869f87187cce9aaa92f647289a7a1068156c9a01b
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,143 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
This project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [3.71.0] - 2026-08-17
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **The round dashboard ships with the L1 `multi_llm_review_workflow` entry**
|
|
12
|
+
(v3.10.2). `scripts/render_dashboard.rb` reads a round summary as JSON on
|
|
13
|
+
stdin, fills `assets/review_dashboard.html`, and writes a self-contained page.
|
|
14
|
+
These are the worked example `resource_render` names in its own description
|
|
15
|
+
and in its default output derivation (`render_dashboard.rb` →
|
|
16
|
+
`dashboard.html`), and neither was in the distribution — they existed on one
|
|
17
|
+
instance only. A fresh install therefore had a core tool whose documented
|
|
18
|
+
example pointed at files that were not on disk. `dashboard.html` itself is
|
|
19
|
+
deliberately not shipped: it is one June 2026 render of
|
|
20
|
+
`kairos_hook_projector_stage1_design_v0.1`, and the renderer recreates it.
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- **An L1 entry's `assets/` and `scripts/` are deleted by an upgrade without
|
|
25
|
+
appearing in the report.** The 3.70.0 upgrade removed three files from this
|
|
26
|
+
instance's `multi_llm_review_workflow` entry (two dashboard pages and the
|
|
27
|
+
renderer, 30.9 KB, last touched 2026-06-02) while the L1 section of the report
|
|
28
|
+
said only `[UPDATED] multi_llm_review_workflow` and `Conflicts: 0`. Mechanism:
|
|
29
|
+
`UpgradeAnalyzer#analyze_knowledge` hashes `<name>.md` alone to decide
|
|
30
|
+
new / unchanged / updated / user_modified / conflict, and the apply step
|
|
31
|
+
replaces the whole entry directory, so any subdirectory content the
|
|
32
|
+
distribution does not carry is removed silently. The L0 section, by contrast,
|
|
33
|
+
reports `[KEPT] … (user-modified)`. Shipping the two files removes the
|
|
34
|
+
deletion for this entry; the general reporting gap is unfixed and recorded.
|
|
35
|
+
Note for anyone adding assets to a shipped entry: the `.md` must change in the
|
|
36
|
+
same release, or the entry is classified `:unchanged` and the new files never
|
|
37
|
+
install.
|
|
38
|
+
- **The dashboard's gate stated a closing condition the project does not use.**
|
|
39
|
+
It required every blocking-pool seat to APPROVE *and* the displayed round's
|
|
40
|
+
entire (a)+(b) count to be zero. Findings may now carry `carryover: true`
|
|
41
|
+
(raised in an earlier round, still open; an absent flag means new), and the
|
|
42
|
+
panel reports a **freeze candidate** when new (a)+(b) is zero, showing the vote
|
|
43
|
+
tally beside it as a reference value. Driven through the real renderer and the
|
|
44
|
+
real gate function: a round with zero new and one carryover (a) at 1 of 2 seats
|
|
45
|
+
approving reports `GATE NOT PASSED` under the old rule and `FREEZE CANDIDATE`
|
|
46
|
+
under this one — and that is the state both 2026-08 review threads actually
|
|
47
|
+
closed in. A round with one new (a) and one carryover (a) at 2 of 2 approving
|
|
48
|
+
reports `NOT CLOSED — 1 new blocking P0 (a+b), 1 carryover`.
|
|
49
|
+
- `Consensus.aggregate`'s `@return` line documented a `:convergence` key the
|
|
50
|
+
method no longer returns; the 3.70.0 rename matched bracket and definition
|
|
51
|
+
forms only. Comment only.
|
|
52
|
+
|
|
53
|
+
## [3.70.0] - 2026-08-17
|
|
54
|
+
|
|
55
|
+
### Changed
|
|
56
|
+
|
|
57
|
+
- **The block of vote counts `multi_llm_review` returns is named `vote_tally`,
|
|
58
|
+
not `convergence`** (SkillSet v0.10.1). It holds `approve_count`,
|
|
59
|
+
`reject_count`, `skip_count`, `successful_count`, `threshold` and `rule` —
|
|
60
|
+
vote arithmetic. The closing condition the L1 workflow states is
|
|
61
|
+
"new (a)+(b) P0 = 0, carryover counted separately", which this SkillSet does
|
|
62
|
+
not compute and no returned field carries. INV-R2 had already demoted the
|
|
63
|
+
ratio to a recorded reference value in a comment at `consensus.rb:156` while
|
|
64
|
+
leaving the field's name intact, so every round handed the orchestrator a
|
|
65
|
+
block whose name claimed what its contents could not answer. Renamed rather
|
|
66
|
+
than given a new sibling field: with nothing in the payload called
|
|
67
|
+
convergence, the criterion has to be fetched from the findings, whereas a new
|
|
68
|
+
field would have left the misleading name in place. 62 sites across 8 files —
|
|
69
|
+
one definition, four production reads, 57 test assertions. One SkillSet
|
|
70
|
+
boundary is crossed: `agent`'s `agent_step.rb` reads this column and now reads
|
|
71
|
+
`vote_tally` with a fallback to `convergence`, following the pattern that file
|
|
72
|
+
already uses for the v0.7 `verdict` → `reference_verdict` rename, because
|
|
73
|
+
records written earlier still say `convergence`. Consumers outside this
|
|
74
|
+
repository that read the old key get nil. Falsified: with the three
|
|
75
|
+
production files reverted to the old name and the tests left renamed, the five
|
|
76
|
+
affected test files produce 44 errors; restored, the suite is 556 runs / 1809
|
|
77
|
+
assertions / 0 failures. Merged at the operator's instruction without
|
|
78
|
+
multi-LLM review.
|
|
79
|
+
- **The L1 `multi_llm_review_workflow` pre-flight checklist states the closing
|
|
80
|
+
condition, not the ratio** (v3.10.1). The checklist line read "Convergence
|
|
81
|
+
rule: 3/5 APPROVE (full) or 3/4 APPROVE (after exclusion)", while
|
|
82
|
+
§ Convergence Rules — 200 lines further down a 1578-line file — states that
|
|
83
|
+
the machine-side signal is "new (a)+(b) P0 = 0" and the ratio is auxiliary.
|
|
84
|
+
The checklist is what gets read before dispatch, so the ratio was the
|
|
85
|
+
operative rule in practice regardless of the prose. The line now leads with
|
|
86
|
+
the closing condition and keeps both ratios beside it as reference values. No
|
|
87
|
+
rule changed; the order in which a reader meets them did.
|
|
88
|
+
|
|
89
|
+
### Known issue, recorded rather than fixed
|
|
90
|
+
|
|
91
|
+
- `consensus[:vote_tally][:reason]` is read in two places
|
|
92
|
+
(`multi_llm_review.rb:548`, `multi_llm_review_collect.rb:431`) and is never
|
|
93
|
+
written, so the `|| 'quorum not met'` fallback is the only reachable value.
|
|
94
|
+
Changing it would alter an operator-facing message, which is a new claim.
|
|
95
|
+
|
|
96
|
+
## [3.69.0] - 2026-08-15
|
|
97
|
+
|
|
98
|
+
### Added
|
|
99
|
+
|
|
100
|
+
- **The mutation experiment and the report shape ship with `minimum_nomic`**
|
|
101
|
+
(SkillSet v0.2.0). Until now the bench could play a game and score it, and the
|
|
102
|
+
score could not rank models: over five games scored twice, the spread between
|
|
103
|
+
judges was 1.35–1.50 points while the spread between the judged was 0.67–0.71,
|
|
104
|
+
and re-scoring the same cell moved it 0.73. Asking a model for a number
|
|
105
|
+
measures the asker.
|
|
106
|
+
|
|
107
|
+
`bin/mutate.rb` replaces that with a question whose answer we hold: copy a
|
|
108
|
+
finished game, reverse exactly one vote, and see whether the analyst notices.
|
|
109
|
+
It writes one arm per seat plus a `clean/` control — not optional, because
|
|
110
|
+
without it a reported contradiction cannot be told from a confabulated one —
|
|
111
|
+
and a manifest naming what was planted where. It refuses to overwrite an
|
|
112
|
+
output directory, to make anything but exactly one substitution, and to leave
|
|
113
|
+
a grammatical scar.
|
|
114
|
+
|
|
115
|
+
`bin/score_detections.rb` prints the evidence and **returns no number**. A
|
|
116
|
+
first pass over 81 verdicts by keyword matching undercounted one analyst by
|
|
117
|
+
four, because its findings were written purely as a contrast — "its reasoning
|
|
118
|
+
says it will vote for its proposal, but it votes against" — which contains no
|
|
119
|
+
keyword. A script returning a count here would have shipped that error
|
|
120
|
+
silently.
|
|
121
|
+
|
|
122
|
+
`report/report_template.html` carries the chapter shape the 2026-08 report
|
|
123
|
+
settled on after being rewritten once: TL;DR before chapter 1, the run diagram
|
|
124
|
+
immediately after, then method, results, discussion, appendices. Placeholders
|
|
125
|
+
are in capitals and each structural choice carries its reason in a comment.
|
|
126
|
+
|
|
127
|
+
Three defects were found by running it rather than by reading it, and are
|
|
128
|
+
fixed here. Vote phrasing is not stable between games, so the flat pattern
|
|
129
|
+
list matched nothing in a fresh game whose ballots read `**Vote:** IN FAVOR`;
|
|
130
|
+
detection is now a marker search followed by a polarity flip within a short
|
|
131
|
+
window. Substituting by first occurrence rewrote a *quoted rule* — "adopted if
|
|
132
|
+
more than half of the players vote in favor of it" — instead of the speaker's
|
|
133
|
+
ballot; substitution is now by offset with the expected text asserted there.
|
|
134
|
+
And a copied arm inherits the original game's analyses, written before
|
|
135
|
+
anything was planted, which the worksheet was presenting as readings of the
|
|
136
|
+
mutated record; every analysis kind is now offset by what the arm inherited.
|
|
137
|
+
|
|
138
|
+
Demonstrated end to end from an installed gem: `skillset install
|
|
139
|
+
minimum_nomic`, a 6-turn game (11.3 minutes, analysts included), `mutate.rb`
|
|
140
|
+
planting three arms and a control, `reanalyse.rb` on two of them, and
|
|
141
|
+
`score_detections.rb`. All three analysts caught the planted reversal; the
|
|
142
|
+
control produced no false alarm.
|
|
143
|
+
|
|
7
144
|
## [3.67.0] - 2026-08-15
|
|
8
145
|
|
|
9
146
|
### Added
|
data/bin/kairos-chain
CHANGED
|
@@ -511,7 +511,16 @@ when 'gate'
|
|
|
511
511
|
puts "installed : #{installed['status']} — #{installed['detail']}"
|
|
512
512
|
puts "declaration: #{result.data['declaration']}"
|
|
513
513
|
puts
|
|
514
|
-
|
|
514
|
+
if result.data['already_declared']
|
|
515
|
+
# Says what happened, not what did not. An earlier line here read
|
|
516
|
+
# "Nothing was written" — false of a run that had just re-applied an
|
|
517
|
+
# edited threshold into the harness config.
|
|
518
|
+
puts "#{gate_mode} already declared the readable gate. Its declaration was left"
|
|
519
|
+
puts 'as it is and re-applied, so any thresholds edited in it are now live.'
|
|
520
|
+
puts 'The lines above are a fresh read of the result.'
|
|
521
|
+
else
|
|
522
|
+
puts 'Installed, and reporting only — blocking is false, as the catalogue ships it.'
|
|
523
|
+
end
|
|
515
524
|
# Reported, not treated as failure: the gate is on either way.
|
|
516
525
|
if result.data['verdict'] == 'OPEN_QUESTIONS'
|
|
517
526
|
puts
|
data/lib/kairos_mcp/version.rb
CHANGED
|
@@ -0,0 +1,332 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="UTF-8">
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
+
<title>Multi-LLM Review Dashboard</title>
|
|
7
|
+
<style>
|
|
8
|
+
:root {
|
|
9
|
+
--bg: #0d1117; --surface: #161b22; --border: #30363d;
|
|
10
|
+
--text: #e6edf3; --text-muted: #8b949e;
|
|
11
|
+
--approve: #3fb950; --reject: #f85149; --revise: #d29922;
|
|
12
|
+
--advisory: #58a6ff; --font: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
|
|
13
|
+
}
|
|
14
|
+
* { margin: 0; padding: 0; box-sizing: border-box; }
|
|
15
|
+
body { font-family: var(--font); background: var(--bg); color: var(--text); padding: 24px; }
|
|
16
|
+
h1 { font-size: 1.4rem; margin-bottom: 4px; }
|
|
17
|
+
.subtitle { color: var(--text-muted); font-size: 0.85rem; margin-bottom: 24px; }
|
|
18
|
+
.grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); gap: 16px; margin-bottom: 24px; }
|
|
19
|
+
.card { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 16px; }
|
|
20
|
+
.card h2 { font-size: 0.95rem; color: var(--text-muted); margin-bottom: 12px; text-transform: uppercase; letter-spacing: 0.05em; }
|
|
21
|
+
.verdict { display: inline-block; padding: 2px 10px; border-radius: 12px; font-size: 0.8rem; font-weight: 600; }
|
|
22
|
+
.verdict.approve { background: rgba(63,185,80,0.15); color: var(--approve); }
|
|
23
|
+
.verdict.reject { background: rgba(248,81,73,0.15); color: var(--reject); }
|
|
24
|
+
.verdict.revise { background: rgba(210,153,34,0.15); color: var(--revise); }
|
|
25
|
+
table { width: 100%; border-collapse: collapse; font-size: 0.85rem; }
|
|
26
|
+
th, td { text-align: left; padding: 8px 12px; border-bottom: 1px solid var(--border); }
|
|
27
|
+
th { color: var(--text-muted); font-weight: 500; }
|
|
28
|
+
.finding-tag { display: inline-block; padding: 1px 6px; border-radius: 4px; font-size: 0.75rem; font-weight: 600; margin-right: 4px; }
|
|
29
|
+
.finding-a { background: rgba(248,81,73,0.15); color: var(--reject); }
|
|
30
|
+
.finding-b { background: rgba(210,153,34,0.15); color: var(--revise); }
|
|
31
|
+
.finding-c { background: rgba(88,166,255,0.15); color: var(--advisory); }
|
|
32
|
+
.stat-number { font-size: 2rem; font-weight: 700; }
|
|
33
|
+
.stat-label { color: var(--text-muted); font-size: 0.8rem; }
|
|
34
|
+
.bar-row { display: flex; align-items: center; gap: 8px; margin-bottom: 6px; }
|
|
35
|
+
.bar-label { width: 120px; font-size: 0.8rem; color: var(--text-muted); text-align: right; }
|
|
36
|
+
.bar-track { flex: 1; height: 20px; background: var(--border); border-radius: 4px; overflow: hidden; display: flex; }
|
|
37
|
+
.bar-seg { height: 100%; transition: width 0.3s; }
|
|
38
|
+
.bar-seg.a { background: var(--reject); }
|
|
39
|
+
.bar-seg.b { background: var(--revise); }
|
|
40
|
+
.bar-seg.c { background: var(--advisory); }
|
|
41
|
+
.round-tabs { display: flex; gap: 8px; margin-bottom: 16px; }
|
|
42
|
+
.round-tab { padding: 6px 16px; border-radius: 6px; border: 1px solid var(--border); background: transparent;
|
|
43
|
+
color: var(--text-muted); cursor: pointer; font-size: 0.85rem; }
|
|
44
|
+
.round-tab.active { background: var(--surface); color: var(--text); border-color: var(--advisory); }
|
|
45
|
+
.legend { display: flex; gap: 16px; margin-bottom: 16px; font-size: 0.8rem; color: var(--text-muted); }
|
|
46
|
+
.legend-item { display: flex; align-items: center; gap: 4px; }
|
|
47
|
+
.legend-dot { width: 10px; height: 10px; border-radius: 2px; }
|
|
48
|
+
.instructions { background: var(--surface); border: 1px solid var(--border); border-radius: 8px;
|
|
49
|
+
padding: 16px; margin-bottom: 24px; font-size: 0.85rem; color: var(--text-muted); }
|
|
50
|
+
.instructions code { background: var(--border); padding: 2px 6px; border-radius: 4px; font-size: 0.8rem; color: var(--text); }
|
|
51
|
+
textarea { width: 100%; min-height: 120px; background: var(--bg); border: 1px solid var(--border);
|
|
52
|
+
border-radius: 6px; padding: 12px; color: var(--text); font-family: monospace; font-size: 0.8rem; resize: vertical; }
|
|
53
|
+
button { padding: 8px 16px; border-radius: 6px; border: 1px solid var(--border); background: var(--surface);
|
|
54
|
+
color: var(--text); cursor: pointer; font-size: 0.85rem; }
|
|
55
|
+
button:hover { border-color: var(--advisory); }
|
|
56
|
+
button.primary { background: rgba(88,166,255,0.15); border-color: var(--advisory); color: var(--advisory); }
|
|
57
|
+
.actions { display: flex; gap: 8px; margin-top: 12px; }
|
|
58
|
+
#json-error { color: var(--reject); font-size: 0.8rem; margin-top: 4px; min-height: 1.2em; }
|
|
59
|
+
</style>
|
|
60
|
+
</head>
|
|
61
|
+
<body>
|
|
62
|
+
|
|
63
|
+
<h1>Multi-LLM Review Dashboard</h1>
|
|
64
|
+
<p class="subtitle">KairosChain L1: multi_llm_review_workflow — HTML resource (assets/)</p>
|
|
65
|
+
|
|
66
|
+
<div class="instructions">
|
|
67
|
+
<strong>How to use:</strong> Paste a review result JSON into the editor below, or use the sample data to explore.
|
|
68
|
+
The JSON format follows the <code>(a)/(b)/(c)</code> finding classification from the multi-LLM review workflow.
|
|
69
|
+
After visualizing, use <strong>Copy as Prompt</strong> to feed the analysis back into Claude Code.
|
|
70
|
+
</div>
|
|
71
|
+
|
|
72
|
+
<div class="card" style="margin-bottom: 24px;">
|
|
73
|
+
<h2>Review Data Input</h2>
|
|
74
|
+
<textarea id="json-input" placeholder='Paste review JSON here...'></textarea>
|
|
75
|
+
<div id="json-error"></div>
|
|
76
|
+
<div class="actions">
|
|
77
|
+
<button class="primary" onclick="loadData()">Visualize</button>
|
|
78
|
+
<button onclick="loadSample()">Load Sample</button>
|
|
79
|
+
<button onclick="copyAsPrompt()">Copy as Prompt</button>
|
|
80
|
+
</div>
|
|
81
|
+
</div>
|
|
82
|
+
|
|
83
|
+
<div id="dashboard" style="display:none;">
|
|
84
|
+
|
|
85
|
+
<div class="round-tabs" id="round-tabs"></div>
|
|
86
|
+
|
|
87
|
+
<div class="legend">
|
|
88
|
+
<div class="legend-item"><div class="legend-dot" style="background:var(--reject)"></div> (a) deployment-grounded</div>
|
|
89
|
+
<div class="legend-item"><div class="legend-dot" style="background:var(--revise)"></div> (b) philosophy-aligned</div>
|
|
90
|
+
<div class="legend-item"><div class="legend-dot" style="background:var(--advisory)"></div> (c) value-divergent</div>
|
|
91
|
+
</div>
|
|
92
|
+
|
|
93
|
+
<div class="grid">
|
|
94
|
+
<div class="card">
|
|
95
|
+
<h2>Consensus</h2>
|
|
96
|
+
<div id="consensus-verdicts"></div>
|
|
97
|
+
<div style="margin-top:12px;">
|
|
98
|
+
<span class="stat-label">New blocking P0 (a+b): </span>
|
|
99
|
+
<span id="blocking-count" class="stat-number" style="font-size:1.4rem;"></span>
|
|
100
|
+
<span class="stat-label" style="margin-left:16px;">Carryover (a+b): </span>
|
|
101
|
+
<span id="carryover-count" class="stat-number" style="font-size:1.4rem;"></span>
|
|
102
|
+
<span class="stat-label" style="margin-left:16px;">Advisory (c): </span>
|
|
103
|
+
<span id="advisory-count" class="stat-number" style="font-size:1.4rem; color:var(--advisory);"></span>
|
|
104
|
+
</div>
|
|
105
|
+
</div>
|
|
106
|
+
<div class="card">
|
|
107
|
+
<h2>Gate Status</h2>
|
|
108
|
+
<div id="gate-status" style="font-size:1.1rem; margin-top:8px;"></div>
|
|
109
|
+
</div>
|
|
110
|
+
</div>
|
|
111
|
+
|
|
112
|
+
<div class="card" style="margin-bottom:16px;">
|
|
113
|
+
<h2>Finding Distribution by Reviewer</h2>
|
|
114
|
+
<div id="finding-bars"></div>
|
|
115
|
+
</div>
|
|
116
|
+
|
|
117
|
+
<div class="card">
|
|
118
|
+
<h2>All Findings</h2>
|
|
119
|
+
<table>
|
|
120
|
+
<thead><tr><th>Reviewer</th><th>Class</th><th>Finding</th><th>Severity</th></tr></thead>
|
|
121
|
+
<tbody id="findings-table"></tbody>
|
|
122
|
+
</table>
|
|
123
|
+
</div>
|
|
124
|
+
</div>
|
|
125
|
+
|
|
126
|
+
<script>
|
|
127
|
+
const SAMPLE = {
|
|
128
|
+
artifact: "kairos_hook_projector_stage1_design_v0.1",
|
|
129
|
+
rounds: [
|
|
130
|
+
{
|
|
131
|
+
round: 1,
|
|
132
|
+
reviewers: [
|
|
133
|
+
{
|
|
134
|
+
id: "philosophy_persona",
|
|
135
|
+
label: "Philosophy Persona (4.7)",
|
|
136
|
+
pool: "blocking",
|
|
137
|
+
verdict: "APPROVE",
|
|
138
|
+
findings: [
|
|
139
|
+
{ class: "b", text: "Inv-C2: substrate reference (plugin_projector) should be invariant, not named dependency", severity: "P0" },
|
|
140
|
+
{ class: "c", text: "Consider adding explicit Prop 5 recording annotation", severity: "P2" }
|
|
141
|
+
]
|
|
142
|
+
},
|
|
143
|
+
{
|
|
144
|
+
id: "engineering_persona",
|
|
145
|
+
label: "Engineering Persona (4.7)",
|
|
146
|
+
pool: "blocking",
|
|
147
|
+
verdict: "REVISE",
|
|
148
|
+
findings: [
|
|
149
|
+
{ class: "a", text: "Inv-2 vs Inv-5 timestamp semantics contradict each other", severity: "P0" },
|
|
150
|
+
{ class: "a", text: "Orphaned compile record lifecycle not specified", severity: "P0" },
|
|
151
|
+
{ class: "b", text: "Inv-O1 declaration-order is mechanism dressed as invariant", severity: "P0" },
|
|
152
|
+
{ class: "c", text: "Prefer explicit error types over string matching", severity: "P2" }
|
|
153
|
+
]
|
|
154
|
+
},
|
|
155
|
+
{
|
|
156
|
+
id: "claude_cli_4.6",
|
|
157
|
+
label: "Claude CLI (Opus 4.6)",
|
|
158
|
+
pool: "blocking",
|
|
159
|
+
verdict: "REVISE",
|
|
160
|
+
findings: [
|
|
161
|
+
{ class: "b", text: "Substrate-as-invariant: compiler must not name plugin_projector", severity: "P0" },
|
|
162
|
+
{ class: "a", text: "mode_name binding integrity across recompile not guaranteed", severity: "P0" },
|
|
163
|
+
{ class: "c", text: "Section ordering could improve readability", severity: "P2" }
|
|
164
|
+
]
|
|
165
|
+
},
|
|
166
|
+
{
|
|
167
|
+
id: "codex_gpt5.4",
|
|
168
|
+
label: "Codex (GPT-5.4)",
|
|
169
|
+
pool: "advisory",
|
|
170
|
+
verdict: "REJECT",
|
|
171
|
+
findings: [
|
|
172
|
+
{ class: "c", text: "Exhaustiveness of Inv-C3 not formally verifiable", severity: "P0" },
|
|
173
|
+
{ class: "c", text: "Missing rollback procedure specification", severity: "P0" },
|
|
174
|
+
{ class: "a", text: "Race condition possible if two compiles run concurrently", severity: "P1" }
|
|
175
|
+
]
|
|
176
|
+
},
|
|
177
|
+
{
|
|
178
|
+
id: "cursor_composer2.5",
|
|
179
|
+
label: "Cursor (Composer 2.5)",
|
|
180
|
+
pool: "advisory",
|
|
181
|
+
verdict: "REJECT",
|
|
182
|
+
findings: [
|
|
183
|
+
{ class: "c", text: "Design lacks complete error taxonomy", severity: "P0" },
|
|
184
|
+
{ class: "c", text: "No performance benchmarks specified", severity: "P1" },
|
|
185
|
+
{ class: "a", text: "File path validation insufficient for symlink traversal", severity: "P1" }
|
|
186
|
+
]
|
|
187
|
+
}
|
|
188
|
+
]
|
|
189
|
+
}
|
|
190
|
+
]
|
|
191
|
+
};
|
|
192
|
+
|
|
193
|
+
let currentData = null;
|
|
194
|
+
let currentRound = 0;
|
|
195
|
+
|
|
196
|
+
function loadSample() {
|
|
197
|
+
document.getElementById('json-input').value = JSON.stringify(SAMPLE, null, 2);
|
|
198
|
+
loadData();
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
function loadData() {
|
|
202
|
+
const errEl = document.getElementById('json-error');
|
|
203
|
+
errEl.textContent = '';
|
|
204
|
+
try {
|
|
205
|
+
currentData = JSON.parse(document.getElementById('json-input').value);
|
|
206
|
+
currentRound = 0;
|
|
207
|
+
renderRoundTabs();
|
|
208
|
+
renderRound(0);
|
|
209
|
+
document.getElementById('dashboard').style.display = 'block';
|
|
210
|
+
} catch (e) {
|
|
211
|
+
errEl.textContent = 'JSON parse error: ' + e.message;
|
|
212
|
+
}
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
function renderRoundTabs() {
|
|
216
|
+
const tabs = document.getElementById('round-tabs');
|
|
217
|
+
tabs.innerHTML = '';
|
|
218
|
+
currentData.rounds.forEach((r, i) => {
|
|
219
|
+
const btn = document.createElement('button');
|
|
220
|
+
btn.className = 'round-tab' + (i === currentRound ? ' active' : '');
|
|
221
|
+
btn.textContent = 'Round ' + r.round;
|
|
222
|
+
btn.onclick = () => { currentRound = i; renderRoundTabs(); renderRound(i); };
|
|
223
|
+
tabs.appendChild(btn);
|
|
224
|
+
});
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
function renderRound(idx) {
|
|
228
|
+
const round = currentData.rounds[idx];
|
|
229
|
+
const reviewers = round.reviewers;
|
|
230
|
+
|
|
231
|
+
// Verdicts
|
|
232
|
+
const vEl = document.getElementById('consensus-verdicts');
|
|
233
|
+
vEl.innerHTML = reviewers.map(r =>
|
|
234
|
+
`<span style="margin-right:12px;">${r.label}: <span class="verdict ${r.verdict.toLowerCase()}">${r.verdict}</span></span>`
|
|
235
|
+
).join('<br>');
|
|
236
|
+
|
|
237
|
+
// Counts. A finding carrying carryover: true was raised in an earlier round
|
|
238
|
+
// and is still open; it does not enter the closing condition. An absent flag
|
|
239
|
+
// means new, which is what the round-1 shape and every earlier input produce.
|
|
240
|
+
let blocking = 0, carryover = 0, advisory = 0;
|
|
241
|
+
reviewers.forEach(r => r.findings.forEach(f => {
|
|
242
|
+
if (f.class === 'a' || f.class === 'b') {
|
|
243
|
+
if (f.carryover) carryover++; else blocking++;
|
|
244
|
+
} else advisory++;
|
|
245
|
+
}));
|
|
246
|
+
document.getElementById('blocking-count').textContent = blocking;
|
|
247
|
+
document.getElementById('carryover-count').textContent = carryover;
|
|
248
|
+
document.getElementById('advisory-count').textContent = advisory;
|
|
249
|
+
|
|
250
|
+
// Gate. The closing condition is new (a)+(b) = 0 — CLAUDE.md § Aggregation
|
|
251
|
+
// rule, L1 multi_llm_review_workflow § Convergence Rules. The vote tally is a
|
|
252
|
+
// reference value, shown and not required: the roster ratio can be
|
|
253
|
+
// arithmetically unreachable, and the intended close is (a)+(b) exhaustion
|
|
254
|
+
// declared by the operator. This panel therefore reports a freeze candidate,
|
|
255
|
+
// never a passed gate.
|
|
256
|
+
const blockingReviewers = reviewers.filter(r => r.pool === 'blocking');
|
|
257
|
+
const approveCount = blockingReviewers.filter(r => r.verdict === 'APPROVE').length;
|
|
258
|
+
const gateEl = document.getElementById('gate-status');
|
|
259
|
+
const votes = `votes ${approveCount}/${blockingReviewers.length} APPROVE (reference)`;
|
|
260
|
+
gateEl.innerHTML = blocking === 0
|
|
261
|
+
? `<span style="color:var(--approve);">✓ FREEZE CANDIDATE</span> — 0 new blocking P0, ${carryover} carryover still open — ${votes}`
|
|
262
|
+
: `<span style="color:var(--reject);">✗ NOT CLOSED</span> — ${blocking} new blocking P0 (a+b), ${carryover} carryover — ${votes}`;
|
|
263
|
+
|
|
264
|
+
// Bars
|
|
265
|
+
const barsEl = document.getElementById('finding-bars');
|
|
266
|
+
barsEl.innerHTML = '';
|
|
267
|
+
reviewers.forEach(r => {
|
|
268
|
+
const a = r.findings.filter(f => f.class === 'a').length;
|
|
269
|
+
const b = r.findings.filter(f => f.class === 'b').length;
|
|
270
|
+
const c = r.findings.filter(f => f.class === 'c').length;
|
|
271
|
+
const total = Math.max(a + b + c, 1);
|
|
272
|
+
const maxFindings = Math.max(...reviewers.map(rv => rv.findings.length), 1);
|
|
273
|
+
const scale = 100 / maxFindings;
|
|
274
|
+
barsEl.innerHTML += `<div class="bar-row">
|
|
275
|
+
<div class="bar-label">${r.id}</div>
|
|
276
|
+
<div class="bar-track">
|
|
277
|
+
<div class="bar-seg a" style="width:${a * scale}%" title="(a) ${a}"></div>
|
|
278
|
+
<div class="bar-seg b" style="width:${b * scale}%" title="(b) ${b}"></div>
|
|
279
|
+
<div class="bar-seg c" style="width:${c * scale}%" title="(c) ${c}"></div>
|
|
280
|
+
</div>
|
|
281
|
+
<span style="font-size:0.8rem;color:var(--text-muted);width:40px;">${a+b+c}</span>
|
|
282
|
+
</div>`;
|
|
283
|
+
});
|
|
284
|
+
|
|
285
|
+
// Table
|
|
286
|
+
const tbody = document.getElementById('findings-table');
|
|
287
|
+
tbody.innerHTML = '';
|
|
288
|
+
reviewers.forEach(r => {
|
|
289
|
+
r.findings.forEach(f => {
|
|
290
|
+
tbody.innerHTML += `<tr>
|
|
291
|
+
<td>${r.id}</td>
|
|
292
|
+
<td><span class="finding-tag finding-${f.class}">(${f.class})</span></td>
|
|
293
|
+
<td>${f.text}</td>
|
|
294
|
+
<td>${f.severity}</td>
|
|
295
|
+
</tr>`;
|
|
296
|
+
});
|
|
297
|
+
});
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
function copyAsPrompt() {
|
|
301
|
+
if (!currentData) return;
|
|
302
|
+
const round = currentData.rounds[currentRound];
|
|
303
|
+
let prompt = `## Multi-LLM Review Round ${round.round} Summary\n\n`;
|
|
304
|
+
prompt += `**Artifact:** ${currentData.artifact}\n\n`;
|
|
305
|
+
|
|
306
|
+
const blockingReviewers = round.reviewers.filter(r => r.pool === 'blocking');
|
|
307
|
+
const approves = blockingReviewers.filter(r => r.verdict === 'APPROVE').length;
|
|
308
|
+
prompt += `**Gate:** ${approves}/${blockingReviewers.length} blocking APPROVE\n\n`;
|
|
309
|
+
|
|
310
|
+
prompt += `### Blocking findings (a+b) — must address:\n\n`;
|
|
311
|
+
round.reviewers.forEach(r => {
|
|
312
|
+
r.findings.filter(f => f.class !== 'c').forEach(f => {
|
|
313
|
+
prompt += `- [${r.id}] (${f.class}) ${f.severity}: ${f.text}\n`;
|
|
314
|
+
});
|
|
315
|
+
});
|
|
316
|
+
|
|
317
|
+
prompt += `\n### Advisory findings (c) — reviewer preference:\n\n`;
|
|
318
|
+
round.reviewers.forEach(r => {
|
|
319
|
+
r.findings.filter(f => f.class === 'c').forEach(f => {
|
|
320
|
+
prompt += `- [${r.id}] (${f.class}) ${f.severity}: ${f.text}\n`;
|
|
321
|
+
});
|
|
322
|
+
});
|
|
323
|
+
|
|
324
|
+
navigator.clipboard.writeText(prompt).then(() => {
|
|
325
|
+
const btn = event.target;
|
|
326
|
+
btn.textContent = 'Copied!';
|
|
327
|
+
setTimeout(() => btn.textContent = 'Copy as Prompt', 1500);
|
|
328
|
+
});
|
|
329
|
+
}
|
|
330
|
+
</script>
|
|
331
|
+
</body>
|
|
332
|
+
</html>
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: multi_llm_review_workflow
|
|
3
3
|
description: "Multi-LLM review methodology and execution — workflow pattern, CLI tooling, consensus analysis, Persona Assembly. Applicable to design, implementation, documentation, or any artifact."
|
|
4
|
-
version: "3.10.
|
|
4
|
+
version: "3.10.2"
|
|
5
5
|
tags:
|
|
6
6
|
- workflow
|
|
7
7
|
- review
|
|
@@ -400,7 +400,10 @@ they disagree, the config is right and this section is stale.
|
|
|
400
400
|
- [ ] Codex models: gpt-5.6-sol AND gpt-5.5 (both, not either/or), each with -m
|
|
401
401
|
- [ ] Cursor model: composer-2.5, passed explicitly as --model composer-2.5
|
|
402
402
|
- [ ] Total reviewer count: 5 (or 4 after orchestrator exclusion from subprocess)
|
|
403
|
-
- [ ]
|
|
403
|
+
- [ ] Closing condition: new (a)+(b) P0 = 0, with carryover P0s counted
|
|
404
|
+
separately and a closure verdict on each. The APPROVE ratio the tool
|
|
405
|
+
reports (3/5 full roster, 3/4 after exclusion) is a reference value,
|
|
406
|
+
not the condition — see § Convergence Rules
|
|
404
407
|
```
|
|
405
408
|
|
|
406
409
|
**Every slot names its model on the command line (INV-E5).** A reviewer launched
|
|
@@ -1574,5 +1577,39 @@ Compression ratio: parallel agent raw → Assembly ≈ 2:1
|
|
|
1574
1577
|
orchestrator's call, per the (a)/(b)/(c) discipline. Handoff record: L2
|
|
1575
1578
|
`handoff_mlr_finding_weight_axis_and_reviewer_incentive_20260806`
|
|
1576
1579
|
|
|
1580
|
+
- The Path A pre-flight checklist states the closing condition, not the ratio
|
|
1581
|
+
(v3.10.1, 2026-08-17): the checklist line read "Convergence rule: 3/5 APPROVE
|
|
1582
|
+
(full) or 3/4 APPROVE (after exclusion)" while § Convergence Rules, 200 lines
|
|
1583
|
+
further down in a 1578-line file, states that the machine-side signal is
|
|
1584
|
+
"new (a)+(b) P0 = 0" and the ratio is auxiliary. The checklist is what is read
|
|
1585
|
+
before dispatch, so the ratio was the operative rule in practice regardless of
|
|
1586
|
+
what the prose said. The line now leads with the closing condition and keeps
|
|
1587
|
+
the two ratios beside it as reference values. No rule changed; the order in
|
|
1588
|
+
which a reader meets them did. Operator observation, 2026-08-17: attention
|
|
1589
|
+
failed to land on the P0 criterion round after round.
|
|
1590
|
+
|
|
1591
|
+
- The round dashboard ships with this entry, and its gate states the closing
|
|
1592
|
+
condition (v3.10.2, 2026-08-17): `scripts/render_dashboard.rb` reads a round
|
|
1593
|
+
summary as JSON on stdin, fills `assets/review_dashboard.html`, and writes a
|
|
1594
|
+
self-contained page — the worked example `resource_render` names in its own
|
|
1595
|
+
description and default output derivation (`render_dashboard.rb` →
|
|
1596
|
+
`dashboard.html`). Both files had existed only on one instance, so a fresh
|
|
1597
|
+
install had a tool whose documented example pointed at absent files, and an
|
|
1598
|
+
upgrade of this entry deleted them: the update decision hashes
|
|
1599
|
+
`multi_llm_review_workflow.md` alone, and the apply step replaces the whole
|
|
1600
|
+
entry directory, so anything under `assets/` or `scripts/` that the
|
|
1601
|
+
distribution does not carry is removed without appearing in the report. The
|
|
1602
|
+
input shape is assembled by hand and is not the review tool's payload:
|
|
1603
|
+
`{artifact, rounds:[{round, reviewers:[{id, label, pool: blocking|advisory,
|
|
1604
|
+
verdict, findings:[{class: a|b|c, text, severity}]}]}]}`. Findings may carry
|
|
1605
|
+
`carryover: true`, meaning raised in an earlier round and still open; an
|
|
1606
|
+
absent flag means new. The gate reports a **freeze candidate** when new
|
|
1607
|
+
(a)+(b) is zero, and shows the vote tally as a reference value beside it. It
|
|
1608
|
+
previously required every blocking-pool seat to APPROVE *and* the round's
|
|
1609
|
+
whole (a)+(b) count to be zero, which is unreachable in the state both
|
|
1610
|
+
2026-08 threads actually closed in — driven through the real renderer, a round
|
|
1611
|
+
with zero new and one carryover (a) at 1 of 2 seats approving reports
|
|
1612
|
+
"GATE NOT PASSED" under the old rule and "FREEZE CANDIDATE" under this one.
|
|
1613
|
+
|
|
1577
1614
|
**Key insight**: Design reviews and implementation reviews find
|
|
1578
1615
|
**categorically different bugs**. Both phases are necessary.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# render_dashboard.rb — Multi-LLM Review Dashboard renderer
|
|
5
|
+
#
|
|
6
|
+
# Protocol:
|
|
7
|
+
# stdin: JSON review data
|
|
8
|
+
# stdout: self-contained HTML with embedded data
|
|
9
|
+
# stderr: error messages
|
|
10
|
+
# exit 0: success, non-0: failure
|
|
11
|
+
|
|
12
|
+
require 'json'
|
|
13
|
+
|
|
14
|
+
data = $stdin.read
|
|
15
|
+
begin
|
|
16
|
+
parsed = JSON.parse(data)
|
|
17
|
+
rescue JSON::ParserError => e
|
|
18
|
+
$stderr.puts "Invalid JSON: #{e.message}"
|
|
19
|
+
exit 1
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
template_path = File.join(__dir__, '..', 'assets', 'review_dashboard.html')
|
|
23
|
+
unless File.exist?(template_path)
|
|
24
|
+
$stderr.puts "Template not found: #{template_path}"
|
|
25
|
+
exit 1
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
template = File.read(template_path)
|
|
29
|
+
|
|
30
|
+
# Replace the SAMPLE constant with actual data and auto-load on page open
|
|
31
|
+
json_literal = JSON.generate(parsed)
|
|
32
|
+
|
|
33
|
+
# Replace the SAMPLE object with actual data
|
|
34
|
+
html = template.sub(
|
|
35
|
+
/const SAMPLE = \{.+?\n\};/m,
|
|
36
|
+
"const SAMPLE = #{JSON.pretty_generate(parsed)};"
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
# Add auto-load: call loadSample() on DOMContentLoaded
|
|
40
|
+
unless html.include?('auto-loaded')
|
|
41
|
+
html = html.sub(
|
|
42
|
+
'</script>',
|
|
43
|
+
"\n// auto-loaded by render_dashboard.rb\n" \
|
|
44
|
+
"document.addEventListener('DOMContentLoaded', loadSample);\n" \
|
|
45
|
+
'</script>'
|
|
46
|
+
)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Update subtitle to show generation timestamp
|
|
50
|
+
html = html.sub(
|
|
51
|
+
'HTML resource (assets/)',
|
|
52
|
+
"Generated #{Time.now.strftime('%Y-%m-%d %H:%M')}"
|
|
53
|
+
)
|
|
54
|
+
|
|
55
|
+
$stdout.write(html)
|
|
@@ -1748,7 +1748,13 @@ module KairosMcp
|
|
|
1748
1748
|
# records still carry 'verdict'; read whichever the record
|
|
1749
1749
|
# speaks.
|
|
1750
1750
|
verdict: parsed['reference_verdict'] || parsed['verdict'],
|
|
1751
|
-
|
|
1751
|
+
# multi_llm_review 0.10.1 renamed this column to vote_tally for
|
|
1752
|
+
# the same reason INV-R2 renamed the one above: the block holds
|
|
1753
|
+
# vote counts and a threshold, and calling it convergence made
|
|
1754
|
+
# the ratio read as the run's closing condition. Records written
|
|
1755
|
+
# before that rename still say 'convergence'; read whichever the
|
|
1756
|
+
# record speaks.
|
|
1757
|
+
vote_tally: parsed['vote_tally'] || parsed['convergence'],
|
|
1752
1758
|
aggregated_findings: (parsed['aggregated_findings'] || []).map { |f|
|
|
1753
1759
|
f.transform_keys(&:to_sym)
|
|
1754
1760
|
},
|