kairos-chain 3.69.0 → 3.71.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +89 -0
- data/lib/kairos_mcp/version.rb +1 -1
- data/templates/knowledge/multi_llm_review_workflow/assets/review_dashboard.html +332 -0
- data/templates/knowledge/multi_llm_review_workflow/multi_llm_review_workflow.md +39 -2
- data/templates/knowledge/multi_llm_review_workflow/scripts/render_dashboard.rb +55 -0
- data/templates/skillsets/agent/tools/agent_step.rb +7 -1
- data/templates/skillsets/multi_llm_review/lib/multi_llm_review/consensus.rb +2 -2
- data/templates/skillsets/multi_llm_review/skillset.json +2 -2
- data/templates/skillsets/multi_llm_review/test/test_multi_llm_review.rb +10 -10
- data/templates/skillsets/multi_llm_review/test/test_mutation_survivors.rb +2 -2
- data/templates/skillsets/multi_llm_review/test/test_observer_set.rb +10 -10
- data/templates/skillsets/multi_llm_review/test/test_observer_set_seams.rb +13 -13
- data/templates/skillsets/multi_llm_review/test/test_tool_wiring.rb +24 -24
- data/templates/skillsets/multi_llm_review/tools/multi_llm_review.rb +2 -2
- data/templates/skillsets/multi_llm_review/tools/multi_llm_review_collect.rb +2 -2
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b8cf3acf9bb95eb62a4fc78d85359540a9089973819fce152278a962007e6faa
|
|
4
|
+
data.tar.gz: bcffe011916e72faecc52aeecc68743a7ef25a96a108b509d9cd80531d04b542
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 759b4cdfe1cc4d6031708d669820994f090d6bf946191d7b3823b9b1f8a3969f9e11193222e8103e065ae20eea290e218638913262ff83225c6250f5f12640b9
|
|
7
|
+
data.tar.gz: 8823dd97245c068d475951c908bcdd77807c43b7b0ce6705680d9413ad52bee608bd246c2a97214df6e0253869f87187cce9aaa92f647289a7a1068156c9a01b
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,95 @@ All notable changes to the `kairos-chain` gem will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
This project follows [Semantic Versioning](https://semver.org/).
|
|
6
6
|
|
|
7
|
+
## [3.71.0] - 2026-08-17
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **The round dashboard ships with the L1 `multi_llm_review_workflow` entry**
|
|
12
|
+
(v3.10.2). `scripts/render_dashboard.rb` reads a round summary as JSON on
|
|
13
|
+
stdin, fills `assets/review_dashboard.html`, and writes a self-contained page.
|
|
14
|
+
These are the worked example `resource_render` names in its own description
|
|
15
|
+
and in its default output derivation (`render_dashboard.rb` →
|
|
16
|
+
`dashboard.html`), and neither was in the distribution — they existed on one
|
|
17
|
+
instance only. A fresh install therefore had a core tool whose documented
|
|
18
|
+
example pointed at files that were not on disk. `dashboard.html` itself is
|
|
19
|
+
deliberately not shipped: it is one June 2026 render of
|
|
20
|
+
`kairos_hook_projector_stage1_design_v0.1`, and the renderer recreates it.
|
|
21
|
+
|
|
22
|
+
### Fixed
|
|
23
|
+
|
|
24
|
+
- **An L1 entry's `assets/` and `scripts/` are deleted by an upgrade without
|
|
25
|
+
appearing in the report.** The 3.70.0 upgrade removed three files from this
|
|
26
|
+
instance's `multi_llm_review_workflow` entry (two dashboard pages and the
|
|
27
|
+
renderer, 30.9 KB, last touched 2026-06-02) while the L1 section of the report
|
|
28
|
+
said only `[UPDATED] multi_llm_review_workflow` and `Conflicts: 0`. Mechanism:
|
|
29
|
+
`UpgradeAnalyzer#analyze_knowledge` hashes `<name>.md` alone to decide
|
|
30
|
+
new / unchanged / updated / user_modified / conflict, and the apply step
|
|
31
|
+
replaces the whole entry directory, so any subdirectory content the
|
|
32
|
+
distribution does not carry is removed silently. The L0 section, by contrast,
|
|
33
|
+
reports `[KEPT] … (user-modified)`. Shipping the two files removes the
|
|
34
|
+
deletion for this entry; the general reporting gap is unfixed and recorded.
|
|
35
|
+
Note for anyone adding assets to a shipped entry: the `.md` must change in the
|
|
36
|
+
same release, or the entry is classified `:unchanged` and the new files never
|
|
37
|
+
install.
|
|
38
|
+
- **The dashboard's gate stated a closing condition the project does not use.**
|
|
39
|
+
It required every blocking-pool seat to APPROVE *and* the displayed round's
|
|
40
|
+
entire (a)+(b) count to be zero. Findings may now carry `carryover: true`
|
|
41
|
+
(raised in an earlier round, still open; an absent flag means new), and the
|
|
42
|
+
panel reports a **freeze candidate** when new (a)+(b) is zero, showing the vote
|
|
43
|
+
tally beside it as a reference value. Driven through the real renderer and the
|
|
44
|
+
real gate function: a round with zero new and one carryover (a) at 1 of 2 seats
|
|
45
|
+
approving reports `GATE NOT PASSED` under the old rule and `FREEZE CANDIDATE`
|
|
46
|
+
under this one — and that is the state both 2026-08 review threads actually
|
|
47
|
+
closed in. A round with one new (a) and one carryover (a) at 2 of 2 approving
|
|
48
|
+
reports `NOT CLOSED — 1 new blocking P0 (a+b), 1 carryover`.
|
|
49
|
+
- `Consensus.aggregate`'s `@return` line documented a `:convergence` key the
|
|
50
|
+
method no longer returns; the 3.70.0 rename matched bracket and definition
|
|
51
|
+
forms only. Comment only.
|
|
52
|
+
|
|
53
|
+
## [3.70.0] - 2026-08-17
|
|
54
|
+
|
|
55
|
+
### Changed
|
|
56
|
+
|
|
57
|
+
- **The block of vote counts `multi_llm_review` returns is named `vote_tally`,
|
|
58
|
+
not `convergence`** (SkillSet v0.10.1). It holds `approve_count`,
|
|
59
|
+
`reject_count`, `skip_count`, `successful_count`, `threshold` and `rule` —
|
|
60
|
+
vote arithmetic. The closing condition the L1 workflow states is
|
|
61
|
+
"new (a)+(b) P0 = 0, carryover counted separately", which this SkillSet does
|
|
62
|
+
not compute and no returned field carries. INV-R2 had already demoted the
|
|
63
|
+
ratio to a recorded reference value in a comment at `consensus.rb:156` while
|
|
64
|
+
leaving the field's name intact, so every round handed the orchestrator a
|
|
65
|
+
block whose name claimed what its contents could not answer. Renamed rather
|
|
66
|
+
than given a new sibling field: with nothing in the payload called
|
|
67
|
+
convergence, the criterion has to be fetched from the findings, whereas a new
|
|
68
|
+
field would have left the misleading name in place. 62 sites across 8 files —
|
|
69
|
+
one definition, four production reads, 57 test assertions. One SkillSet
|
|
70
|
+
boundary is crossed: `agent`'s `agent_step.rb` reads this column and now reads
|
|
71
|
+
`vote_tally` with a fallback to `convergence`, following the pattern that file
|
|
72
|
+
already uses for the v0.7 `verdict` → `reference_verdict` rename, because
|
|
73
|
+
records written earlier still say `convergence`. Consumers outside this
|
|
74
|
+
repository that read the old key get nil. Falsified: with the three
|
|
75
|
+
production files reverted to the old name and the tests left renamed, the five
|
|
76
|
+
affected test files produce 44 errors; restored, the suite is 556 runs / 1809
|
|
77
|
+
assertions / 0 failures. Merged at the operator's instruction without
|
|
78
|
+
multi-LLM review.
|
|
79
|
+
- **The L1 `multi_llm_review_workflow` pre-flight checklist states the closing
|
|
80
|
+
condition, not the ratio** (v3.10.1). The checklist line read "Convergence
|
|
81
|
+
rule: 3/5 APPROVE (full) or 3/4 APPROVE (after exclusion)", while
|
|
82
|
+
§ Convergence Rules — 200 lines further down a 1578-line file — states that
|
|
83
|
+
the machine-side signal is "new (a)+(b) P0 = 0" and the ratio is auxiliary.
|
|
84
|
+
The checklist is what gets read before dispatch, so the ratio was the
|
|
85
|
+
operative rule in practice regardless of the prose. The line now leads with
|
|
86
|
+
the closing condition and keeps both ratios beside it as reference values. No
|
|
87
|
+
rule changed; the order in which a reader meets them did.
|
|
88
|
+
|
|
89
|
+
### Known issue, recorded rather than fixed
|
|
90
|
+
|
|
91
|
+
- `consensus[:vote_tally][:reason]` is read in two places
|
|
92
|
+
(`multi_llm_review.rb:548`, `multi_llm_review_collect.rb:431`) and is never
|
|
93
|
+
written, so the `|| 'quorum not met'` fallback is the only reachable value.
|
|
94
|
+
Changing it would alter an operator-facing message, which is a new claim.
|
|
95
|
+
|
|
7
96
|
## [3.69.0] - 2026-08-15
|
|
8
97
|
|
|
9
98
|
### Added
|
data/lib/kairos_mcp/version.rb
CHANGED
|
@@ -0,0 +1,332 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="UTF-8">
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
+
<title>Multi-LLM Review Dashboard</title>
|
|
7
|
+
<style>
|
|
8
|
+
:root {
|
|
9
|
+
--bg: #0d1117; --surface: #161b22; --border: #30363d;
|
|
10
|
+
--text: #e6edf3; --text-muted: #8b949e;
|
|
11
|
+
--approve: #3fb950; --reject: #f85149; --revise: #d29922;
|
|
12
|
+
--advisory: #58a6ff; --font: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
|
|
13
|
+
}
|
|
14
|
+
* { margin: 0; padding: 0; box-sizing: border-box; }
|
|
15
|
+
body { font-family: var(--font); background: var(--bg); color: var(--text); padding: 24px; }
|
|
16
|
+
h1 { font-size: 1.4rem; margin-bottom: 4px; }
|
|
17
|
+
.subtitle { color: var(--text-muted); font-size: 0.85rem; margin-bottom: 24px; }
|
|
18
|
+
.grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(280px, 1fr)); gap: 16px; margin-bottom: 24px; }
|
|
19
|
+
.card { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 16px; }
|
|
20
|
+
.card h2 { font-size: 0.95rem; color: var(--text-muted); margin-bottom: 12px; text-transform: uppercase; letter-spacing: 0.05em; }
|
|
21
|
+
.verdict { display: inline-block; padding: 2px 10px; border-radius: 12px; font-size: 0.8rem; font-weight: 600; }
|
|
22
|
+
.verdict.approve { background: rgba(63,185,80,0.15); color: var(--approve); }
|
|
23
|
+
.verdict.reject { background: rgba(248,81,73,0.15); color: var(--reject); }
|
|
24
|
+
.verdict.revise { background: rgba(210,153,34,0.15); color: var(--revise); }
|
|
25
|
+
table { width: 100%; border-collapse: collapse; font-size: 0.85rem; }
|
|
26
|
+
th, td { text-align: left; padding: 8px 12px; border-bottom: 1px solid var(--border); }
|
|
27
|
+
th { color: var(--text-muted); font-weight: 500; }
|
|
28
|
+
.finding-tag { display: inline-block; padding: 1px 6px; border-radius: 4px; font-size: 0.75rem; font-weight: 600; margin-right: 4px; }
|
|
29
|
+
.finding-a { background: rgba(248,81,73,0.15); color: var(--reject); }
|
|
30
|
+
.finding-b { background: rgba(210,153,34,0.15); color: var(--revise); }
|
|
31
|
+
.finding-c { background: rgba(88,166,255,0.15); color: var(--advisory); }
|
|
32
|
+
.stat-number { font-size: 2rem; font-weight: 700; }
|
|
33
|
+
.stat-label { color: var(--text-muted); font-size: 0.8rem; }
|
|
34
|
+
.bar-row { display: flex; align-items: center; gap: 8px; margin-bottom: 6px; }
|
|
35
|
+
.bar-label { width: 120px; font-size: 0.8rem; color: var(--text-muted); text-align: right; }
|
|
36
|
+
.bar-track { flex: 1; height: 20px; background: var(--border); border-radius: 4px; overflow: hidden; display: flex; }
|
|
37
|
+
.bar-seg { height: 100%; transition: width 0.3s; }
|
|
38
|
+
.bar-seg.a { background: var(--reject); }
|
|
39
|
+
.bar-seg.b { background: var(--revise); }
|
|
40
|
+
.bar-seg.c { background: var(--advisory); }
|
|
41
|
+
.round-tabs { display: flex; gap: 8px; margin-bottom: 16px; }
|
|
42
|
+
.round-tab { padding: 6px 16px; border-radius: 6px; border: 1px solid var(--border); background: transparent;
|
|
43
|
+
color: var(--text-muted); cursor: pointer; font-size: 0.85rem; }
|
|
44
|
+
.round-tab.active { background: var(--surface); color: var(--text); border-color: var(--advisory); }
|
|
45
|
+
.legend { display: flex; gap: 16px; margin-bottom: 16px; font-size: 0.8rem; color: var(--text-muted); }
|
|
46
|
+
.legend-item { display: flex; align-items: center; gap: 4px; }
|
|
47
|
+
.legend-dot { width: 10px; height: 10px; border-radius: 2px; }
|
|
48
|
+
.instructions { background: var(--surface); border: 1px solid var(--border); border-radius: 8px;
|
|
49
|
+
padding: 16px; margin-bottom: 24px; font-size: 0.85rem; color: var(--text-muted); }
|
|
50
|
+
.instructions code { background: var(--border); padding: 2px 6px; border-radius: 4px; font-size: 0.8rem; color: var(--text); }
|
|
51
|
+
textarea { width: 100%; min-height: 120px; background: var(--bg); border: 1px solid var(--border);
|
|
52
|
+
border-radius: 6px; padding: 12px; color: var(--text); font-family: monospace; font-size: 0.8rem; resize: vertical; }
|
|
53
|
+
button { padding: 8px 16px; border-radius: 6px; border: 1px solid var(--border); background: var(--surface);
|
|
54
|
+
color: var(--text); cursor: pointer; font-size: 0.85rem; }
|
|
55
|
+
button:hover { border-color: var(--advisory); }
|
|
56
|
+
button.primary { background: rgba(88,166,255,0.15); border-color: var(--advisory); color: var(--advisory); }
|
|
57
|
+
.actions { display: flex; gap: 8px; margin-top: 12px; }
|
|
58
|
+
#json-error { color: var(--reject); font-size: 0.8rem; margin-top: 4px; min-height: 1.2em; }
|
|
59
|
+
</style>
|
|
60
|
+
</head>
|
|
61
|
+
<body>
|
|
62
|
+
|
|
63
|
+
<h1>Multi-LLM Review Dashboard</h1>
|
|
64
|
+
<p class="subtitle">KairosChain L1: multi_llm_review_workflow — HTML resource (assets/)</p>
|
|
65
|
+
|
|
66
|
+
<div class="instructions">
|
|
67
|
+
<strong>How to use:</strong> Paste a review result JSON into the editor below, or use the sample data to explore.
|
|
68
|
+
The JSON format follows the <code>(a)/(b)/(c)</code> finding classification from the multi-LLM review workflow.
|
|
69
|
+
After visualizing, use <strong>Copy as Prompt</strong> to feed the analysis back into Claude Code.
|
|
70
|
+
</div>
|
|
71
|
+
|
|
72
|
+
<div class="card" style="margin-bottom: 24px;">
|
|
73
|
+
<h2>Review Data Input</h2>
|
|
74
|
+
<textarea id="json-input" placeholder='Paste review JSON here...'></textarea>
|
|
75
|
+
<div id="json-error"></div>
|
|
76
|
+
<div class="actions">
|
|
77
|
+
<button class="primary" onclick="loadData()">Visualize</button>
|
|
78
|
+
<button onclick="loadSample()">Load Sample</button>
|
|
79
|
+
<button onclick="copyAsPrompt()">Copy as Prompt</button>
|
|
80
|
+
</div>
|
|
81
|
+
</div>
|
|
82
|
+
|
|
83
|
+
<div id="dashboard" style="display:none;">
|
|
84
|
+
|
|
85
|
+
<div class="round-tabs" id="round-tabs"></div>
|
|
86
|
+
|
|
87
|
+
<div class="legend">
|
|
88
|
+
<div class="legend-item"><div class="legend-dot" style="background:var(--reject)"></div> (a) deployment-grounded</div>
|
|
89
|
+
<div class="legend-item"><div class="legend-dot" style="background:var(--revise)"></div> (b) philosophy-aligned</div>
|
|
90
|
+
<div class="legend-item"><div class="legend-dot" style="background:var(--advisory)"></div> (c) value-divergent</div>
|
|
91
|
+
</div>
|
|
92
|
+
|
|
93
|
+
<div class="grid">
|
|
94
|
+
<div class="card">
|
|
95
|
+
<h2>Consensus</h2>
|
|
96
|
+
<div id="consensus-verdicts"></div>
|
|
97
|
+
<div style="margin-top:12px;">
|
|
98
|
+
<span class="stat-label">New blocking P0 (a+b): </span>
|
|
99
|
+
<span id="blocking-count" class="stat-number" style="font-size:1.4rem;"></span>
|
|
100
|
+
<span class="stat-label" style="margin-left:16px;">Carryover (a+b): </span>
|
|
101
|
+
<span id="carryover-count" class="stat-number" style="font-size:1.4rem;"></span>
|
|
102
|
+
<span class="stat-label" style="margin-left:16px;">Advisory (c): </span>
|
|
103
|
+
<span id="advisory-count" class="stat-number" style="font-size:1.4rem; color:var(--advisory);"></span>
|
|
104
|
+
</div>
|
|
105
|
+
</div>
|
|
106
|
+
<div class="card">
|
|
107
|
+
<h2>Gate Status</h2>
|
|
108
|
+
<div id="gate-status" style="font-size:1.1rem; margin-top:8px;"></div>
|
|
109
|
+
</div>
|
|
110
|
+
</div>
|
|
111
|
+
|
|
112
|
+
<div class="card" style="margin-bottom:16px;">
|
|
113
|
+
<h2>Finding Distribution by Reviewer</h2>
|
|
114
|
+
<div id="finding-bars"></div>
|
|
115
|
+
</div>
|
|
116
|
+
|
|
117
|
+
<div class="card">
|
|
118
|
+
<h2>All Findings</h2>
|
|
119
|
+
<table>
|
|
120
|
+
<thead><tr><th>Reviewer</th><th>Class</th><th>Finding</th><th>Severity</th></tr></thead>
|
|
121
|
+
<tbody id="findings-table"></tbody>
|
|
122
|
+
</table>
|
|
123
|
+
</div>
|
|
124
|
+
</div>
|
|
125
|
+
|
|
126
|
+
<script>
|
|
127
|
+
const SAMPLE = {
|
|
128
|
+
artifact: "kairos_hook_projector_stage1_design_v0.1",
|
|
129
|
+
rounds: [
|
|
130
|
+
{
|
|
131
|
+
round: 1,
|
|
132
|
+
reviewers: [
|
|
133
|
+
{
|
|
134
|
+
id: "philosophy_persona",
|
|
135
|
+
label: "Philosophy Persona (4.7)",
|
|
136
|
+
pool: "blocking",
|
|
137
|
+
verdict: "APPROVE",
|
|
138
|
+
findings: [
|
|
139
|
+
{ class: "b", text: "Inv-C2: substrate reference (plugin_projector) should be invariant, not named dependency", severity: "P0" },
|
|
140
|
+
{ class: "c", text: "Consider adding explicit Prop 5 recording annotation", severity: "P2" }
|
|
141
|
+
]
|
|
142
|
+
},
|
|
143
|
+
{
|
|
144
|
+
id: "engineering_persona",
|
|
145
|
+
label: "Engineering Persona (4.7)",
|
|
146
|
+
pool: "blocking",
|
|
147
|
+
verdict: "REVISE",
|
|
148
|
+
findings: [
|
|
149
|
+
{ class: "a", text: "Inv-2 vs Inv-5 timestamp semantics contradict each other", severity: "P0" },
|
|
150
|
+
{ class: "a", text: "Orphaned compile record lifecycle not specified", severity: "P0" },
|
|
151
|
+
{ class: "b", text: "Inv-O1 declaration-order is mechanism dressed as invariant", severity: "P0" },
|
|
152
|
+
{ class: "c", text: "Prefer explicit error types over string matching", severity: "P2" }
|
|
153
|
+
]
|
|
154
|
+
},
|
|
155
|
+
{
|
|
156
|
+
id: "claude_cli_4.6",
|
|
157
|
+
label: "Claude CLI (Opus 4.6)",
|
|
158
|
+
pool: "blocking",
|
|
159
|
+
verdict: "REVISE",
|
|
160
|
+
findings: [
|
|
161
|
+
{ class: "b", text: "Substrate-as-invariant: compiler must not name plugin_projector", severity: "P0" },
|
|
162
|
+
{ class: "a", text: "mode_name binding integrity across recompile not guaranteed", severity: "P0" },
|
|
163
|
+
{ class: "c", text: "Section ordering could improve readability", severity: "P2" }
|
|
164
|
+
]
|
|
165
|
+
},
|
|
166
|
+
{
|
|
167
|
+
id: "codex_gpt5.4",
|
|
168
|
+
label: "Codex (GPT-5.4)",
|
|
169
|
+
pool: "advisory",
|
|
170
|
+
verdict: "REJECT",
|
|
171
|
+
findings: [
|
|
172
|
+
{ class: "c", text: "Exhaustiveness of Inv-C3 not formally verifiable", severity: "P0" },
|
|
173
|
+
{ class: "c", text: "Missing rollback procedure specification", severity: "P0" },
|
|
174
|
+
{ class: "a", text: "Race condition possible if two compiles run concurrently", severity: "P1" }
|
|
175
|
+
]
|
|
176
|
+
},
|
|
177
|
+
{
|
|
178
|
+
id: "cursor_composer2.5",
|
|
179
|
+
label: "Cursor (Composer 2.5)",
|
|
180
|
+
pool: "advisory",
|
|
181
|
+
verdict: "REJECT",
|
|
182
|
+
findings: [
|
|
183
|
+
{ class: "c", text: "Design lacks complete error taxonomy", severity: "P0" },
|
|
184
|
+
{ class: "c", text: "No performance benchmarks specified", severity: "P1" },
|
|
185
|
+
{ class: "a", text: "File path validation insufficient for symlink traversal", severity: "P1" }
|
|
186
|
+
]
|
|
187
|
+
}
|
|
188
|
+
]
|
|
189
|
+
}
|
|
190
|
+
]
|
|
191
|
+
};
|
|
192
|
+
|
|
193
|
+
let currentData = null;
|
|
194
|
+
let currentRound = 0;
|
|
195
|
+
|
|
196
|
+
function loadSample() {
|
|
197
|
+
document.getElementById('json-input').value = JSON.stringify(SAMPLE, null, 2);
|
|
198
|
+
loadData();
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
function loadData() {
|
|
202
|
+
const errEl = document.getElementById('json-error');
|
|
203
|
+
errEl.textContent = '';
|
|
204
|
+
try {
|
|
205
|
+
currentData = JSON.parse(document.getElementById('json-input').value);
|
|
206
|
+
currentRound = 0;
|
|
207
|
+
renderRoundTabs();
|
|
208
|
+
renderRound(0);
|
|
209
|
+
document.getElementById('dashboard').style.display = 'block';
|
|
210
|
+
} catch (e) {
|
|
211
|
+
errEl.textContent = 'JSON parse error: ' + e.message;
|
|
212
|
+
}
|
|
213
|
+
}
|
|
214
|
+
|
|
215
|
+
function renderRoundTabs() {
|
|
216
|
+
const tabs = document.getElementById('round-tabs');
|
|
217
|
+
tabs.innerHTML = '';
|
|
218
|
+
currentData.rounds.forEach((r, i) => {
|
|
219
|
+
const btn = document.createElement('button');
|
|
220
|
+
btn.className = 'round-tab' + (i === currentRound ? ' active' : '');
|
|
221
|
+
btn.textContent = 'Round ' + r.round;
|
|
222
|
+
btn.onclick = () => { currentRound = i; renderRoundTabs(); renderRound(i); };
|
|
223
|
+
tabs.appendChild(btn);
|
|
224
|
+
});
|
|
225
|
+
}
|
|
226
|
+
|
|
227
|
+
function renderRound(idx) {
|
|
228
|
+
const round = currentData.rounds[idx];
|
|
229
|
+
const reviewers = round.reviewers;
|
|
230
|
+
|
|
231
|
+
// Verdicts
|
|
232
|
+
const vEl = document.getElementById('consensus-verdicts');
|
|
233
|
+
vEl.innerHTML = reviewers.map(r =>
|
|
234
|
+
`<span style="margin-right:12px;">${r.label}: <span class="verdict ${r.verdict.toLowerCase()}">${r.verdict}</span></span>`
|
|
235
|
+
).join('<br>');
|
|
236
|
+
|
|
237
|
+
// Counts. A finding carrying carryover: true was raised in an earlier round
|
|
238
|
+
// and is still open; it does not enter the closing condition. An absent flag
|
|
239
|
+
// means new, which is what the round-1 shape and every earlier input produce.
|
|
240
|
+
let blocking = 0, carryover = 0, advisory = 0;
|
|
241
|
+
reviewers.forEach(r => r.findings.forEach(f => {
|
|
242
|
+
if (f.class === 'a' || f.class === 'b') {
|
|
243
|
+
if (f.carryover) carryover++; else blocking++;
|
|
244
|
+
} else advisory++;
|
|
245
|
+
}));
|
|
246
|
+
document.getElementById('blocking-count').textContent = blocking;
|
|
247
|
+
document.getElementById('carryover-count').textContent = carryover;
|
|
248
|
+
document.getElementById('advisory-count').textContent = advisory;
|
|
249
|
+
|
|
250
|
+
// Gate. The closing condition is new (a)+(b) = 0 — CLAUDE.md § Aggregation
|
|
251
|
+
// rule, L1 multi_llm_review_workflow § Convergence Rules. The vote tally is a
|
|
252
|
+
// reference value, shown and not required: the roster ratio can be
|
|
253
|
+
// arithmetically unreachable, and the intended close is (a)+(b) exhaustion
|
|
254
|
+
// declared by the operator. This panel therefore reports a freeze candidate,
|
|
255
|
+
// never a passed gate.
|
|
256
|
+
const blockingReviewers = reviewers.filter(r => r.pool === 'blocking');
|
|
257
|
+
const approveCount = blockingReviewers.filter(r => r.verdict === 'APPROVE').length;
|
|
258
|
+
const gateEl = document.getElementById('gate-status');
|
|
259
|
+
const votes = `votes ${approveCount}/${blockingReviewers.length} APPROVE (reference)`;
|
|
260
|
+
gateEl.innerHTML = blocking === 0
|
|
261
|
+
? `<span style="color:var(--approve);">✓ FREEZE CANDIDATE</span> — 0 new blocking P0, ${carryover} carryover still open — ${votes}`
|
|
262
|
+
: `<span style="color:var(--reject);">✗ NOT CLOSED</span> — ${blocking} new blocking P0 (a+b), ${carryover} carryover — ${votes}`;
|
|
263
|
+
|
|
264
|
+
// Bars
|
|
265
|
+
const barsEl = document.getElementById('finding-bars');
|
|
266
|
+
barsEl.innerHTML = '';
|
|
267
|
+
reviewers.forEach(r => {
|
|
268
|
+
const a = r.findings.filter(f => f.class === 'a').length;
|
|
269
|
+
const b = r.findings.filter(f => f.class === 'b').length;
|
|
270
|
+
const c = r.findings.filter(f => f.class === 'c').length;
|
|
271
|
+
const total = Math.max(a + b + c, 1);
|
|
272
|
+
const maxFindings = Math.max(...reviewers.map(rv => rv.findings.length), 1);
|
|
273
|
+
const scale = 100 / maxFindings;
|
|
274
|
+
barsEl.innerHTML += `<div class="bar-row">
|
|
275
|
+
<div class="bar-label">${r.id}</div>
|
|
276
|
+
<div class="bar-track">
|
|
277
|
+
<div class="bar-seg a" style="width:${a * scale}%" title="(a) ${a}"></div>
|
|
278
|
+
<div class="bar-seg b" style="width:${b * scale}%" title="(b) ${b}"></div>
|
|
279
|
+
<div class="bar-seg c" style="width:${c * scale}%" title="(c) ${c}"></div>
|
|
280
|
+
</div>
|
|
281
|
+
<span style="font-size:0.8rem;color:var(--text-muted);width:40px;">${a+b+c}</span>
|
|
282
|
+
</div>`;
|
|
283
|
+
});
|
|
284
|
+
|
|
285
|
+
// Table
|
|
286
|
+
const tbody = document.getElementById('findings-table');
|
|
287
|
+
tbody.innerHTML = '';
|
|
288
|
+
reviewers.forEach(r => {
|
|
289
|
+
r.findings.forEach(f => {
|
|
290
|
+
tbody.innerHTML += `<tr>
|
|
291
|
+
<td>${r.id}</td>
|
|
292
|
+
<td><span class="finding-tag finding-${f.class}">(${f.class})</span></td>
|
|
293
|
+
<td>${f.text}</td>
|
|
294
|
+
<td>${f.severity}</td>
|
|
295
|
+
</tr>`;
|
|
296
|
+
});
|
|
297
|
+
});
|
|
298
|
+
}
|
|
299
|
+
|
|
300
|
+
function copyAsPrompt() {
|
|
301
|
+
if (!currentData) return;
|
|
302
|
+
const round = currentData.rounds[currentRound];
|
|
303
|
+
let prompt = `## Multi-LLM Review Round ${round.round} Summary\n\n`;
|
|
304
|
+
prompt += `**Artifact:** ${currentData.artifact}\n\n`;
|
|
305
|
+
|
|
306
|
+
const blockingReviewers = round.reviewers.filter(r => r.pool === 'blocking');
|
|
307
|
+
const approves = blockingReviewers.filter(r => r.verdict === 'APPROVE').length;
|
|
308
|
+
prompt += `**Gate:** ${approves}/${blockingReviewers.length} blocking APPROVE\n\n`;
|
|
309
|
+
|
|
310
|
+
prompt += `### Blocking findings (a+b) — must address:\n\n`;
|
|
311
|
+
round.reviewers.forEach(r => {
|
|
312
|
+
r.findings.filter(f => f.class !== 'c').forEach(f => {
|
|
313
|
+
prompt += `- [${r.id}] (${f.class}) ${f.severity}: ${f.text}\n`;
|
|
314
|
+
});
|
|
315
|
+
});
|
|
316
|
+
|
|
317
|
+
prompt += `\n### Advisory findings (c) — reviewer preference:\n\n`;
|
|
318
|
+
round.reviewers.forEach(r => {
|
|
319
|
+
r.findings.filter(f => f.class === 'c').forEach(f => {
|
|
320
|
+
prompt += `- [${r.id}] (${f.class}) ${f.severity}: ${f.text}\n`;
|
|
321
|
+
});
|
|
322
|
+
});
|
|
323
|
+
|
|
324
|
+
navigator.clipboard.writeText(prompt).then(() => {
|
|
325
|
+
const btn = event.target;
|
|
326
|
+
btn.textContent = 'Copied!';
|
|
327
|
+
setTimeout(() => btn.textContent = 'Copy as Prompt', 1500);
|
|
328
|
+
});
|
|
329
|
+
}
|
|
330
|
+
</script>
|
|
331
|
+
</body>
|
|
332
|
+
</html>
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: multi_llm_review_workflow
|
|
3
3
|
description: "Multi-LLM review methodology and execution — workflow pattern, CLI tooling, consensus analysis, Persona Assembly. Applicable to design, implementation, documentation, or any artifact."
|
|
4
|
-
version: "3.10.
|
|
4
|
+
version: "3.10.2"
|
|
5
5
|
tags:
|
|
6
6
|
- workflow
|
|
7
7
|
- review
|
|
@@ -400,7 +400,10 @@ they disagree, the config is right and this section is stale.
|
|
|
400
400
|
- [ ] Codex models: gpt-5.6-sol AND gpt-5.5 (both, not either/or), each with -m
|
|
401
401
|
- [ ] Cursor model: composer-2.5, passed explicitly as --model composer-2.5
|
|
402
402
|
- [ ] Total reviewer count: 5 (or 4 after orchestrator exclusion from subprocess)
|
|
403
|
-
- [ ]
|
|
403
|
+
- [ ] Closing condition: new (a)+(b) P0 = 0, with carryover P0s counted
|
|
404
|
+
separately and a closure verdict on each. The APPROVE ratio the tool
|
|
405
|
+
reports (3/5 full roster, 3/4 after exclusion) is a reference value,
|
|
406
|
+
not the condition — see § Convergence Rules
|
|
404
407
|
```
|
|
405
408
|
|
|
406
409
|
**Every slot names its model on the command line (INV-E5).** A reviewer launched
|
|
@@ -1574,5 +1577,39 @@ Compression ratio: parallel agent raw → Assembly ≈ 2:1
|
|
|
1574
1577
|
orchestrator's call, per the (a)/(b)/(c) discipline. Handoff record: L2
|
|
1575
1578
|
`handoff_mlr_finding_weight_axis_and_reviewer_incentive_20260806`
|
|
1576
1579
|
|
|
1580
|
+
- The Path A pre-flight checklist states the closing condition, not the ratio
|
|
1581
|
+
(v3.10.1, 2026-08-17): the checklist line read "Convergence rule: 3/5 APPROVE
|
|
1582
|
+
(full) or 3/4 APPROVE (after exclusion)" while § Convergence Rules, 200 lines
|
|
1583
|
+
further down in a 1578-line file, states that the machine-side signal is
|
|
1584
|
+
"new (a)+(b) P0 = 0" and the ratio is auxiliary. The checklist is what is read
|
|
1585
|
+
before dispatch, so the ratio was the operative rule in practice regardless of
|
|
1586
|
+
what the prose said. The line now leads with the closing condition and keeps
|
|
1587
|
+
the two ratios beside it as reference values. No rule changed; the order in
|
|
1588
|
+
which a reader meets them did. Operator observation, 2026-08-17: attention
|
|
1589
|
+
failed to land on the P0 criterion round after round.
|
|
1590
|
+
|
|
1591
|
+
- The round dashboard ships with this entry, and its gate states the closing
|
|
1592
|
+
condition (v3.10.2, 2026-08-17): `scripts/render_dashboard.rb` reads a round
|
|
1593
|
+
summary as JSON on stdin, fills `assets/review_dashboard.html`, and writes a
|
|
1594
|
+
self-contained page — the worked example `resource_render` names in its own
|
|
1595
|
+
description and default output derivation (`render_dashboard.rb` →
|
|
1596
|
+
`dashboard.html`). Both files had existed only on one instance, so a fresh
|
|
1597
|
+
install had a tool whose documented example pointed at absent files, and an
|
|
1598
|
+
upgrade of this entry deleted them: the update decision hashes
|
|
1599
|
+
`multi_llm_review_workflow.md` alone, and the apply step replaces the whole
|
|
1600
|
+
entry directory, so anything under `assets/` or `scripts/` that the
|
|
1601
|
+
distribution does not carry is removed without appearing in the report. The
|
|
1602
|
+
input shape is assembled by hand and is not the review tool's payload:
|
|
1603
|
+
`{artifact, rounds:[{round, reviewers:[{id, label, pool: blocking|advisory,
|
|
1604
|
+
verdict, findings:[{class: a|b|c, text, severity}]}]}]}`. Findings may carry
|
|
1605
|
+
`carryover: true`, meaning raised in an earlier round and still open; an
|
|
1606
|
+
absent flag means new. The gate reports a **freeze candidate** when new
|
|
1607
|
+
(a)+(b) is zero, and shows the vote tally as a reference value beside it. It
|
|
1608
|
+
previously required every blocking-pool seat to APPROVE *and* the round's
|
|
1609
|
+
whole (a)+(b) count to be zero, which is unreachable in the state both
|
|
1610
|
+
2026-08 threads actually closed in — driven through the real renderer, a round
|
|
1611
|
+
with zero new and one carryover (a) at 1 of 2 seats approving reports
|
|
1612
|
+
"GATE NOT PASSED" under the old rule and "FREEZE CANDIDATE" under this one.
|
|
1613
|
+
|
|
1577
1614
|
**Key insight**: Design reviews and implementation reviews find
|
|
1578
1615
|
**categorically different bugs**. Both phases are necessary.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# frozen_string_literal: true
|
|
3
|
+
|
|
4
|
+
# render_dashboard.rb — Multi-LLM Review Dashboard renderer
|
|
5
|
+
#
|
|
6
|
+
# Protocol:
|
|
7
|
+
# stdin: JSON review data
|
|
8
|
+
# stdout: self-contained HTML with embedded data
|
|
9
|
+
# stderr: error messages
|
|
10
|
+
# exit 0: success, non-0: failure
|
|
11
|
+
|
|
12
|
+
require 'json'
|
|
13
|
+
|
|
14
|
+
data = $stdin.read
|
|
15
|
+
begin
|
|
16
|
+
parsed = JSON.parse(data)
|
|
17
|
+
rescue JSON::ParserError => e
|
|
18
|
+
$stderr.puts "Invalid JSON: #{e.message}"
|
|
19
|
+
exit 1
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
template_path = File.join(__dir__, '..', 'assets', 'review_dashboard.html')
|
|
23
|
+
unless File.exist?(template_path)
|
|
24
|
+
$stderr.puts "Template not found: #{template_path}"
|
|
25
|
+
exit 1
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
template = File.read(template_path)
|
|
29
|
+
|
|
30
|
+
# Replace the SAMPLE constant with actual data and auto-load on page open
|
|
31
|
+
json_literal = JSON.generate(parsed)
|
|
32
|
+
|
|
33
|
+
# Replace the SAMPLE object with actual data
|
|
34
|
+
html = template.sub(
|
|
35
|
+
/const SAMPLE = \{.+?\n\};/m,
|
|
36
|
+
"const SAMPLE = #{JSON.pretty_generate(parsed)};"
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
# Add auto-load: call loadSample() on DOMContentLoaded
|
|
40
|
+
unless html.include?('auto-loaded')
|
|
41
|
+
html = html.sub(
|
|
42
|
+
'</script>',
|
|
43
|
+
"\n// auto-loaded by render_dashboard.rb\n" \
|
|
44
|
+
"document.addEventListener('DOMContentLoaded', loadSample);\n" \
|
|
45
|
+
'</script>'
|
|
46
|
+
)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
# Update subtitle to show generation timestamp
|
|
50
|
+
html = html.sub(
|
|
51
|
+
'HTML resource (assets/)',
|
|
52
|
+
"Generated #{Time.now.strftime('%Y-%m-%d %H:%M')}"
|
|
53
|
+
)
|
|
54
|
+
|
|
55
|
+
$stdout.write(html)
|
|
@@ -1748,7 +1748,13 @@ module KairosMcp
|
|
|
1748
1748
|
# records still carry 'verdict'; read whichever the record
|
|
1749
1749
|
# speaks.
|
|
1750
1750
|
verdict: parsed['reference_verdict'] || parsed['verdict'],
|
|
1751
|
-
|
|
1751
|
+
# multi_llm_review 0.10.1 renamed this column to vote_tally for
|
|
1752
|
+
# the same reason INV-R2 renamed the one above: the block holds
|
|
1753
|
+
# vote counts and a threshold, and calling it convergence made
|
|
1754
|
+
# the ratio read as the run's closing condition. Records written
|
|
1755
|
+
# before that rename still say 'convergence'; read whichever the
|
|
1756
|
+
# record speaks.
|
|
1757
|
+
vote_tally: parsed['vote_tally'] || parsed['convergence'],
|
|
1752
1758
|
aggregated_findings: (parsed['aggregated_findings'] || []).map { |f|
|
|
1753
1759
|
f.transform_keys(&:to_sym)
|
|
1754
1760
|
},
|
|
@@ -137,7 +137,7 @@ module KairosMcp
|
|
|
137
137
|
# @param reviews [Array<Hash>] from Dispatcher, each with :status, :raw_text, :role_label, etc.
|
|
138
138
|
# @param rule_str [String] e.g., "3/4 APPROVE"
|
|
139
139
|
# @param min_quorum [Integer] minimum successful reviews needed
|
|
140
|
-
# @return [Hash] with :reference_verdict, :
|
|
140
|
+
# @return [Hash] with :reference_verdict, :vote_tally, :reviews, :aggregated_findings
|
|
141
141
|
def self.aggregate(reviews, rule_str = '3/4 APPROVE', min_quorum: 2,
|
|
142
142
|
excluded_slots: [], escalation: nil)
|
|
143
143
|
# Parsing a verdict and deciding whether the reply belongs in the
|
|
@@ -177,7 +177,7 @@ module KairosMcp
|
|
|
177
177
|
findings = aggregate_findings(parsed)
|
|
178
178
|
{
|
|
179
179
|
reference_verdict: overall,
|
|
180
|
-
|
|
180
|
+
vote_tally: {
|
|
181
181
|
approve_count: approve_n,
|
|
182
182
|
reject_count: reject_n,
|
|
183
183
|
skip_count: skipped.size,
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "multi_llm_review",
|
|
3
|
-
"version": "0.10.
|
|
4
|
-
"description": "v0.10.0 (finding weight axis + worker-death recovery, 2026-08-06): findings gain a consequence axis beside the severity axis — the prompt contract requires a [consequence: who is harmed, and how, if this is never fixed] clause on every P0, and aggregation records a P0 without one at P2, keeping the stated severity and the demotion reason on the row (severity_stated / severity_demoted: consequence_missing). Only PRESENCE is checked, mechanically; whether a stated consequence is real or trivial stays the orchestrator's call. The dedup key strips the clause so two reviewers naming one defect still merge, and a group where one member states the harm carries it for the row. Motivating measurement (project_orientation_report checker, R5): 3 of 7 P0s were factually correct findings that cost nobody anything. Reviewer prompts also stop naming the round number — telling a reviewer which round it is in (like telling it its counts are compared) selects for finding-production over finding-weight; the L1 workflow v3.10.0 Reviewer incentive rule states the orchestrator-side half. And a worker death no longer discards completed seats: the worker persists each reply as it arrives (partial_results.json), and collect's crash/timeout branches recover the finished seats, entering every unreached seat in the denominator as a skip row (worker_crashed_seat_lost) with the death named in payload.worker_failure — R3 2026-08-06 lost three completed external seats to one stale heartbeat because the only exit was total loss. v0.9.1 (seat access, frozen 2026-08-06 after a one-round review): reviewer prompts carry a <seat_access> block beside the inline artifact — a seat that cannot read the repository must not attempt tool calls and must not open by saying it will read files; it reviews the artifact text alone, marks unverifiable claims [INFERRED], and still opens with its verdict line. The wording is conditional because seats differ (codex runs --sandbox read-only and can read the repository), and the block is not emitted for by_reference delivery, whose existing cannot-read instruction it would contradict. Root cause fixed: the claude subprocess seat runs with tools disabled in an empty working directory, and on implementation artifacts citing file paths it opened with pseudo-tool-call markup or a cannot-access preamble instead of its verdict line, leaving four consecutive rounds as no_verdict while counting every round on design artifacts. v0.9.0 (evidence fidelity, frozen 2026-08-06 after six review rounds): a finding reaches the record whole. It used to be cut at 201 bytes by an inclusive Range in aggregation and bounded again by the 500-character display limit, so a downstream instance measured 18 of 21 findings arriving at exactly 201 bytes with reviews[].raw_text empty on every row. Findings are now bounded in BYTES at FINDING_RECORD_MAX_LEN = 8000 for the record, with DEFAULT_MAX_LEN = 500 still applied by every path that takes a finding into a prompt. Deduplication still keys on the first 80 characters — widening it would stop two reviewers describing one defect from merging, moving the finding count and the convergence denominator — but the surviving text is no longer arbitrary: issue comes from a member whose severity equals the merged severity, distinct texts survive in issue_variants (capped at MAX_ISSUE_VARIANTS = 8, with issue_variants_omitted naming what the cap dropped). Every row now carries raw_text_excerpt unconditionally (4096 bytes) and the reviewer's reply in raw_text on request (include_raw_text, 65536 bytes). Both are SANITISED TRANSCRIPTIONS, not verbatim records, and both tool schemas say so: the text is byte-clamped, NFKC-normalised, stripped of invisible characters, tag-escaped, then byte-clamped again. No field states whether they hold the whole reply, because nothing in this SkillSet can know — a completeness flag was implemented, measured wrong in both directions, and removed. On delegated runs the pending-state record keeps each subprocess reply as it arrived; on single-phase runs the returned payload is the only form there is. Findings carried into a later round's prompt are sanitised and folded to one line, a path that previously took reviewer text into a prompt with no sanitisation at all. v0.8.0 (v0.7 record schema, design frozen 2026-08-01): the verdict vocabulary is the three canonical words plus tense forms only (INV-R1); the ratio and threshold are recorded reference values, not the run's conclusion — the top-level verdict field became reference_verdict and the run is closed by the operator's declaration outside the record (INV-R2); the persona team occupies one seat, its derivation rule is recorded, and a submission smaller than convened — including empty — is accepted with the shortfall on the record (INV-R3/R4); every run writes an existence marker at dispatch, completed records are never garbage-collected, and expired runs are reduced to a minimal trace instead of erased (INV-R4); a divergence-excluded tally is carried beside the main one (INV-R5); the record names its pre-declared spec and carries transport diagnostics as state tags (INV-R6); artifact delivery is a per-seat attribute (inline | by_reference) and an unreachable delivery is refused rather than dispatched (INV-R7). Parallel multi-LLM review orchestration. Dispatches review prompts to N LLM backends via llm_client, collects verdicts, and computes consensus. v0.6.0: reserve observers (escalate) and a declarable persona execution model; the observer set is built in one pass with explicit precedence (ObserverSet); every slot must name its model and role_label, and duplicate names — including the persona team's own — are refused. A reply's verdict is no longer inferred from its prose: it is read from a declared field, from the header the reply opens with when that header carries a verdict name and nothing else, or not at all, in which case the reply leaves the denominator with no_verdict recorded beside its name. The record says why every observer did or did not count (denominator_composition, five skip_reason values, observers_reporting), and the per-reviewer row is written by one mapping rather than two. v0.5.2: the cursor reviewer pins model composer-2.5 instead of inheriting the cursor CLI default, which is operator-editable and had silently become an Anthropic model. v0.5.1: Fable 5 retired from the roster (five consecutive silent returns), convergence 3/5; orchestrator_model description now states the bare-ID rule so a caller does not review its own output. v0.5.0: adds multi_llm_review_wait (Phase 1.5) for explicit subprocess completion gating with next_action recovery hints, and Path A/B doc disambiguation. v0.4.0 (Phase 12): feedback_text + schema_version, sanitization contract for prompt-injection defense, and multi_llm_review_bundle tool for human-handoff paths without dispatch.",
|
|
3
|
+
"version": "0.10.1",
|
|
4
|
+
"description": "v0.10.1 (2026-08-17): the returned block of vote counts is named vote_tally, not convergence. It holds approve_count, reject_count, skip_count, successful_count, threshold and rule -- vote arithmetic -- while the closing condition the L1 workflow states is 'new (a)+(b) P0 = 0', which this SkillSet does not compute and no returned field carries. INV-R2 had already demoted the ratio to a reference value in a code comment while leaving the field named convergence, so every round handed the orchestrator a block whose name claimed what its contents could not answer. Renaming rather than adding a field is deliberate: with nothing in the payload called convergence, the criterion has to be fetched from the findings. Consumers reading the old name get nil; agent_step reads vote_tally with a fallback to convergence for records written earlier. v0.10.0 (finding weight axis + worker-death recovery, 2026-08-06): findings gain a consequence axis beside the severity axis — the prompt contract requires a [consequence: who is harmed, and how, if this is never fixed] clause on every P0, and aggregation records a P0 without one at P2, keeping the stated severity and the demotion reason on the row (severity_stated / severity_demoted: consequence_missing). Only PRESENCE is checked, mechanically; whether a stated consequence is real or trivial stays the orchestrator's call. The dedup key strips the clause so two reviewers naming one defect still merge, and a group where one member states the harm carries it for the row. Motivating measurement (project_orientation_report checker, R5): 3 of 7 P0s were factually correct findings that cost nobody anything. Reviewer prompts also stop naming the round number — telling a reviewer which round it is in (like telling it its counts are compared) selects for finding-production over finding-weight; the L1 workflow v3.10.0 Reviewer incentive rule states the orchestrator-side half. And a worker death no longer discards completed seats: the worker persists each reply as it arrives (partial_results.json), and collect's crash/timeout branches recover the finished seats, entering every unreached seat in the denominator as a skip row (worker_crashed_seat_lost) with the death named in payload.worker_failure — R3 2026-08-06 lost three completed external seats to one stale heartbeat because the only exit was total loss. v0.9.1 (seat access, frozen 2026-08-06 after a one-round review): reviewer prompts carry a <seat_access> block beside the inline artifact — a seat that cannot read the repository must not attempt tool calls and must not open by saying it will read files; it reviews the artifact text alone, marks unverifiable claims [INFERRED], and still opens with its verdict line. The wording is conditional because seats differ (codex runs --sandbox read-only and can read the repository), and the block is not emitted for by_reference delivery, whose existing cannot-read instruction it would contradict. Root cause fixed: the claude subprocess seat runs with tools disabled in an empty working directory, and on implementation artifacts citing file paths it opened with pseudo-tool-call markup or a cannot-access preamble instead of its verdict line, leaving four consecutive rounds as no_verdict while counting every round on design artifacts. v0.9.0 (evidence fidelity, frozen 2026-08-06 after six review rounds): a finding reaches the record whole. It used to be cut at 201 bytes by an inclusive Range in aggregation and bounded again by the 500-character display limit, so a downstream instance measured 18 of 21 findings arriving at exactly 201 bytes with reviews[].raw_text empty on every row. Findings are now bounded in BYTES at FINDING_RECORD_MAX_LEN = 8000 for the record, with DEFAULT_MAX_LEN = 500 still applied by every path that takes a finding into a prompt. Deduplication still keys on the first 80 characters — widening it would stop two reviewers describing one defect from merging, moving the finding count and the convergence denominator — but the surviving text is no longer arbitrary: issue comes from a member whose severity equals the merged severity, distinct texts survive in issue_variants (capped at MAX_ISSUE_VARIANTS = 8, with issue_variants_omitted naming what the cap dropped). Every row now carries raw_text_excerpt unconditionally (4096 bytes) and the reviewer's reply in raw_text on request (include_raw_text, 65536 bytes). Both are SANITISED TRANSCRIPTIONS, not verbatim records, and both tool schemas say so: the text is byte-clamped, NFKC-normalised, stripped of invisible characters, tag-escaped, then byte-clamped again. No field states whether they hold the whole reply, because nothing in this SkillSet can know — a completeness flag was implemented, measured wrong in both directions, and removed. On delegated runs the pending-state record keeps each subprocess reply as it arrived; on single-phase runs the returned payload is the only form there is. Findings carried into a later round's prompt are sanitised and folded to one line, a path that previously took reviewer text into a prompt with no sanitisation at all. v0.8.0 (v0.7 record schema, design frozen 2026-08-01): the verdict vocabulary is the three canonical words plus tense forms only (INV-R1); the ratio and threshold are recorded reference values, not the run's conclusion — the top-level verdict field became reference_verdict and the run is closed by the operator's declaration outside the record (INV-R2); the persona team occupies one seat, its derivation rule is recorded, and a submission smaller than convened — including empty — is accepted with the shortfall on the record (INV-R3/R4); every run writes an existence marker at dispatch, completed records are never garbage-collected, and expired runs are reduced to a minimal trace instead of erased (INV-R4); a divergence-excluded tally is carried beside the main one (INV-R5); the record names its pre-declared spec and carries transport diagnostics as state tags (INV-R6); artifact delivery is a per-seat attribute (inline | by_reference) and an unreachable delivery is refused rather than dispatched (INV-R7). Parallel multi-LLM review orchestration. Dispatches review prompts to N LLM backends via llm_client, collects verdicts, and computes consensus. v0.6.0: reserve observers (escalate) and a declarable persona execution model; the observer set is built in one pass with explicit precedence (ObserverSet); every slot must name its model and role_label, and duplicate names — including the persona team's own — are refused. A reply's verdict is no longer inferred from its prose: it is read from a declared field, from the header the reply opens with when that header carries a verdict name and nothing else, or not at all, in which case the reply leaves the denominator with no_verdict recorded beside its name. The record says why every observer did or did not count (denominator_composition, five skip_reason values, observers_reporting), and the per-reviewer row is written by one mapping rather than two. v0.5.2: the cursor reviewer pins model composer-2.5 instead of inheriting the cursor CLI default, which is operator-editable and had silently become an Anthropic model. v0.5.1: Fable 5 retired from the roster (five consecutive silent returns), convergence 3/5; orchestrator_model description now states the bare-ID rule so a caller does not review its own output. v0.5.0: adds multi_llm_review_wait (Phase 1.5) for explicit subprocess completion gating with next_action recovery hints, and Path A/B doc disambiguation. v0.4.0 (Phase 12): feedback_text + schema_version, sanitization contract for prompt-injection defense, and multi_llm_review_bundle tool for human-handoff paths without dispatch.",
|
|
5
5
|
"author": "Masaomi Hatakeyama",
|
|
6
6
|
"layer": "L1",
|
|
7
7
|
"depends_on": [
|
|
@@ -51,8 +51,8 @@ module KairosMcp
|
|
|
51
51
|
]
|
|
52
52
|
result = Consensus.aggregate(reviews, '3/4 APPROVE', min_quorum: 2)
|
|
53
53
|
assert_equal 'APPROVE', result[:reference_verdict]
|
|
54
|
-
assert_equal 3, result[:
|
|
55
|
-
assert_equal 0, result[:
|
|
54
|
+
assert_equal 3, result[:vote_tally][:approve_count]
|
|
55
|
+
assert_equal 0, result[:vote_tally][:reject_count]
|
|
56
56
|
end
|
|
57
57
|
|
|
58
58
|
def test_any_reject_means_revise
|
|
@@ -63,7 +63,7 @@ module KairosMcp
|
|
|
63
63
|
]
|
|
64
64
|
result = Consensus.aggregate(reviews, '2/3 APPROVE', min_quorum: 2)
|
|
65
65
|
assert_equal 'REVISE', result[:reference_verdict]
|
|
66
|
-
assert_equal 1, result[:
|
|
66
|
+
assert_equal 1, result[:vote_tally][:reject_count]
|
|
67
67
|
end
|
|
68
68
|
|
|
69
69
|
def test_skip_excluded_from_denominator
|
|
@@ -75,10 +75,10 @@ module KairosMcp
|
|
|
75
75
|
]
|
|
76
76
|
result = Consensus.aggregate(reviews, '3/4 APPROVE', min_quorum: 2)
|
|
77
77
|
assert_equal 'APPROVE', result[:reference_verdict]
|
|
78
|
-
assert_equal 2, result[:
|
|
79
|
-
assert_equal 2, result[:
|
|
78
|
+
assert_equal 2, result[:vote_tally][:successful_count]
|
|
79
|
+
assert_equal 2, result[:vote_tally][:skip_count]
|
|
80
80
|
# threshold = ceil(2 * 0.75) = 2, approve = 2 >= 2
|
|
81
|
-
assert_equal 2, result[:
|
|
81
|
+
assert_equal 2, result[:vote_tally][:threshold]
|
|
82
82
|
end
|
|
83
83
|
|
|
84
84
|
def test_insufficient_quorum
|
|
@@ -137,12 +137,12 @@ module KairosMcp
|
|
|
137
137
|
{ role_label: 'r5', raw_text: body('APPROVE'), status: :success }
|
|
138
138
|
]
|
|
139
139
|
result = Consensus.aggregate(reviews, '3/5 APPROVE', min_quorum: 2)
|
|
140
|
-
excl = result[:
|
|
140
|
+
excl = result[:vote_tally][:excluding_divergent]
|
|
141
141
|
assert_equal 3, excl[:successful_count]
|
|
142
142
|
assert_equal 3, excl[:approve_count]
|
|
143
143
|
# ceil(3 * 0.6) = 2 — NOT ceil(5 * 0.6) = 3.
|
|
144
144
|
assert_equal 2, excl[:threshold]
|
|
145
|
-
assert_equal 3, result[:
|
|
145
|
+
assert_equal 3, result[:vote_tally][:threshold]
|
|
146
146
|
end
|
|
147
147
|
|
|
148
148
|
def test_aggregate_findings_dedup
|
|
@@ -1620,7 +1620,7 @@ module KairosMcp
|
|
|
1620
1620
|
assert_equal ['codex_gpt5.5'], failure['recovered_seats']
|
|
1621
1621
|
assert_equal ['cursor'], failure['lost_seats']
|
|
1622
1622
|
|
|
1623
|
-
observers = payload.dig('
|
|
1623
|
+
observers = payload.dig('vote_tally', 'denominator_composition', 'observers')
|
|
1624
1624
|
lost = observers.find { |o| o['role_label'] == 'cursor' }
|
|
1625
1625
|
refute lost['counted']
|
|
1626
1626
|
assert_equal 'worker_crashed_seat_lost', lost['reason']
|
|
@@ -1931,7 +1931,7 @@ module KairosMcp
|
|
|
1931
1931
|
assert_equal 'LGTM', row['stated_text']
|
|
1932
1932
|
|
|
1933
1933
|
comp = Consensus.aggregate([review], '3/5 APPROVE', min_quorum: 1)
|
|
1934
|
-
observer = comp[:
|
|
1934
|
+
observer = comp[:vote_tally][:denominator_composition][:observers].first
|
|
1935
1935
|
assert_equal 'LGTM', observer[:stated_text]
|
|
1936
1936
|
end
|
|
1937
1937
|
|
|
@@ -360,7 +360,7 @@ module KairosMcp
|
|
|
360
360
|
[{ status: :success, role_label: 'a', substantive: true,
|
|
361
361
|
raw_text: "**Overall Verdict**: APPROVE\n\nthe design reads correctly to me" }],
|
|
362
362
|
'1/1 APPROVE', min_quorum: 1, excluded_slots: excluded
|
|
363
|
-
)[:
|
|
363
|
+
)[:vote_tally][:denominator_composition][:observers]
|
|
364
364
|
end
|
|
365
365
|
|
|
366
366
|
def test_a_slot_that_never_ran_omits_the_fields_it_has_no_value_for
|
|
@@ -1119,7 +1119,7 @@ module KairosMcp
|
|
|
1119
1119
|
out = Consensus.aggregate(reviews, '3/4 APPROVE', min_quorum: 2)
|
|
1120
1120
|
|
|
1121
1121
|
assert_equal 'APPROVE', out[:reference_verdict]
|
|
1122
|
-
assert_equal 3, out[:
|
|
1122
|
+
assert_equal 3, out[:vote_tally][:approve_count]
|
|
1123
1123
|
end
|
|
1124
1124
|
end
|
|
1125
1125
|
end
|
|
@@ -420,8 +420,8 @@ module KairosMcp
|
|
|
420
420
|
'3/5 APPROVE', min_quorum: 2
|
|
421
421
|
)
|
|
422
422
|
|
|
423
|
-
assert_equal 2, out[:
|
|
424
|
-
assert_equal 1, out[:
|
|
423
|
+
assert_equal 2, out[:vote_tally][:successful_count]
|
|
424
|
+
assert_equal 1, out[:vote_tally][:skip_count]
|
|
425
425
|
hollow = out[:reviews].find { |r| r[:role_label] == 'c' }
|
|
426
426
|
assert_equal 'SKIP', hollow[:verdict]
|
|
427
427
|
assert_equal Consensus::SKIP_REASON_INSUBSTANTIAL, hollow[:skip_reason]
|
|
@@ -440,8 +440,8 @@ module KairosMcp
|
|
|
440
440
|
out = Consensus.aggregate([review('a', REAL), review('b', short)],
|
|
441
441
|
'3/5 APPROVE', min_quorum: 1)
|
|
442
442
|
|
|
443
|
-
assert_equal 2, out[:
|
|
444
|
-
assert_equal 1, out[:
|
|
443
|
+
assert_equal 2, out[:vote_tally][:successful_count]
|
|
444
|
+
assert_equal 1, out[:vote_tally][:reject_count]
|
|
445
445
|
end
|
|
446
446
|
|
|
447
447
|
# And a review that says something real without stating its verdict in
|
|
@@ -454,7 +454,7 @@ module KairosMcp
|
|
|
454
454
|
out = Consensus.aggregate([review('a', REAL), review('b', headerless)],
|
|
455
455
|
'3/5 APPROVE', min_quorum: 1)
|
|
456
456
|
|
|
457
|
-
assert_equal 1, out[:
|
|
457
|
+
assert_equal 1, out[:vote_tally][:successful_count]
|
|
458
458
|
left = out[:reviews].find { |r| r[:role_label] == 'b' }
|
|
459
459
|
assert_equal Consensus::SKIP_REASON_NO_VERDICT, left[:skip_reason]
|
|
460
460
|
end
|
|
@@ -478,7 +478,7 @@ module KairosMcp
|
|
|
478
478
|
'3/5 APPROVE', min_quorum: 2
|
|
479
479
|
)
|
|
480
480
|
|
|
481
|
-
assert_equal 2, out[:
|
|
481
|
+
assert_equal 2, out[:vote_tally][:threshold]
|
|
482
482
|
assert_equal 'APPROVE', out[:reference_verdict]
|
|
483
483
|
end
|
|
484
484
|
|
|
@@ -493,7 +493,7 @@ module KairosMcp
|
|
|
493
493
|
escalation: { 'escalated' => true, 'slots' => ['cli_fable5'] }
|
|
494
494
|
)
|
|
495
495
|
|
|
496
|
-
comp = out[:
|
|
496
|
+
comp = out[:vote_tally][:denominator_composition]
|
|
497
497
|
assert_equal %w[a c cli_opus5], comp[:observers].map { |o| o[:role_label] }
|
|
498
498
|
# A record that exists is carried as it is. Filling its gaps wrote
|
|
499
499
|
# `requested: false` beside `escalated: true`, a pair the producing
|
|
@@ -517,7 +517,7 @@ module KairosMcp
|
|
|
517
517
|
out = Consensus.aggregate([review('a', REAL), hollow],
|
|
518
518
|
'3/5 APPROVE', min_quorum: 1)
|
|
519
519
|
|
|
520
|
-
assert_equal 1, out[:
|
|
520
|
+
assert_equal 1, out[:vote_tally][:successful_count]
|
|
521
521
|
assert_equal Consensus::SKIP_REASON_INSUBSTANTIAL,
|
|
522
522
|
out[:reviews].find { |r| r[:role_label] == 'team' }[:skip_reason]
|
|
523
523
|
end
|
|
@@ -527,7 +527,7 @@ module KairosMcp
|
|
|
527
527
|
out = Consensus.aggregate([review('a', REAL), terse],
|
|
528
528
|
'3/5 APPROVE', min_quorum: 1)
|
|
529
529
|
|
|
530
|
-
assert_equal 2, out[:
|
|
530
|
+
assert_equal 2, out[:vote_tally][:successful_count]
|
|
531
531
|
end
|
|
532
532
|
|
|
533
533
|
# INV-E5 / INV-P1: a record says whether the model name was observed or
|
|
@@ -538,7 +538,7 @@ module KairosMcp
|
|
|
538
538
|
declared = review('b', REAL).merge(model_source: 'declared')
|
|
539
539
|
out = Consensus.aggregate([observed, declared], '3/5 APPROVE', min_quorum: 1)
|
|
540
540
|
|
|
541
|
-
by_label = out[:
|
|
541
|
+
by_label = out[:vote_tally][:denominator_composition][:observers]
|
|
542
542
|
.map { |o| [o[:role_label], o] }.to_h
|
|
543
543
|
assert_equal 'observed', by_label['a'][:model_source]
|
|
544
544
|
assert_equal true, by_label['a'][:model_divergence]
|
|
@@ -294,7 +294,7 @@ module KairosMcp
|
|
|
294
294
|
# Nothing was lost in transport, and the row has to say so on its own.
|
|
295
295
|
assert_equal 'insubstantial', hollow['skip_reason']
|
|
296
296
|
assert_nil hollow['error']
|
|
297
|
-
assert_equal 3, payload['
|
|
297
|
+
assert_equal 3, payload['vote_tally']['successful_count']
|
|
298
298
|
|
|
299
299
|
# A row that did not leave the denominator has no reason to give, and
|
|
300
300
|
# omits the field rather than carrying an explicit null — the shape
|
|
@@ -324,7 +324,7 @@ module KairosMcp
|
|
|
324
324
|
# INV-E4: the slots that never ran, and the escalation, survive into
|
|
325
325
|
# the final record.
|
|
326
326
|
def test_composition_survives_the_round_trip
|
|
327
|
-
comp = collect(write_state)['
|
|
327
|
+
comp = collect(write_state)['vote_tally']['denominator_composition']
|
|
328
328
|
|
|
329
329
|
labels = comp['observers'].map { |o| o['role_label'] }
|
|
330
330
|
assert_includes labels, 'cli_opus5'
|
|
@@ -386,7 +386,7 @@ module KairosMcp
|
|
|
386
386
|
token = write_state('escalation' => { 'requested' => true })
|
|
387
387
|
|
|
388
388
|
assert_equal({ 'requested' => true },
|
|
389
|
-
collect(token)['
|
|
389
|
+
collect(token)['vote_tally']['denominator_composition']['escalation'])
|
|
390
390
|
end
|
|
391
391
|
|
|
392
392
|
# The one case that is answered rather than left silent, because it can
|
|
@@ -400,7 +400,7 @@ module KairosMcp
|
|
|
400
400
|
|
|
401
401
|
assert_equal({ 'requested' => false, 'escalated' => false,
|
|
402
402
|
'slots' => [], 'dispatched' => [] },
|
|
403
|
-
collect(token)['
|
|
403
|
+
collect(token)['vote_tally']['denominator_composition']['escalation'])
|
|
404
404
|
end
|
|
405
405
|
|
|
406
406
|
# The filled record is built fresh each time. Returning copies that
|
|
@@ -417,7 +417,7 @@ module KairosMcp
|
|
|
417
417
|
# INV-E5: a divergence recorded by the dispatcher is still legible at
|
|
418
418
|
# the end of the round trip.
|
|
419
419
|
def test_divergence_survives_the_round_trip
|
|
420
|
-
comp = collect(write_state)['
|
|
420
|
+
comp = collect(write_state)['vote_tally']['denominator_composition']
|
|
421
421
|
diverged = comp['observers'].find { |o| o['role_label'] == 'cli_opus46' }
|
|
422
422
|
|
|
423
423
|
assert_equal 'observed', diverged['model_source']
|
|
@@ -449,14 +449,14 @@ module KairosMcp
|
|
|
449
449
|
lenient = split.call('2/3 APPROVE')
|
|
450
450
|
strict = split.call('3/3 APPROVE')
|
|
451
451
|
|
|
452
|
-
assert_equal 3, lenient['
|
|
453
|
-
assert_equal 2, lenient['
|
|
452
|
+
assert_equal 3, lenient['vote_tally']['successful_count']
|
|
453
|
+
assert_equal 2, lenient['vote_tally']['approve_count']
|
|
454
454
|
|
|
455
455
|
assert_equal 'APPROVE', lenient['reference_verdict']
|
|
456
|
-
assert_equal '2/3 APPROVE', lenient['
|
|
456
|
+
assert_equal '2/3 APPROVE', lenient['vote_tally']['rule']
|
|
457
457
|
|
|
458
458
|
assert_equal 'REVISE', strict['reference_verdict']
|
|
459
|
-
assert_equal '3/3 APPROVE', strict['
|
|
459
|
+
assert_equal '3/3 APPROVE', strict['vote_tally']['rule']
|
|
460
460
|
end
|
|
461
461
|
|
|
462
462
|
# The other half of the verdict decision travels the same boundary and
|
|
@@ -467,10 +467,10 @@ module KairosMcp
|
|
|
467
467
|
high = collect(write_state('min_quorum' => 9))
|
|
468
468
|
|
|
469
469
|
assert_equal 'APPROVE', low['reference_verdict']
|
|
470
|
-
assert_equal 1, low['
|
|
470
|
+
assert_equal 1, low['vote_tally']['min_quorum']
|
|
471
471
|
|
|
472
472
|
assert_equal 'INSUFFICIENT', high['reference_verdict']
|
|
473
|
-
assert_equal 9, high['
|
|
473
|
+
assert_equal 9, high['vote_tally']['min_quorum']
|
|
474
474
|
end
|
|
475
475
|
|
|
476
476
|
# INV-E5 on the rows a reader reads, not only in the composition. The
|
|
@@ -496,7 +496,7 @@ module KairosMcp
|
|
|
496
496
|
assert_equal true, persona['synthetic']
|
|
497
497
|
assert_equal false, executed['synthetic']
|
|
498
498
|
|
|
499
|
-
observers = payload['
|
|
499
|
+
observers = payload['vote_tally']['denominator_composition']['observers']
|
|
500
500
|
assert_equal true, observers.find { |o| o['role_label'] == persona['role_label'] }['synthetic']
|
|
501
501
|
assert_equal false, observers.find { |o| o['role_label'] == 'codex' }['synthetic']
|
|
502
502
|
end
|
|
@@ -862,7 +862,7 @@ module KairosMcp
|
|
|
862
862
|
{ status: :success, role_label: 'b', verdict: 'SKIP', raw_text: 'x',
|
|
863
863
|
skip_reason: nil }],
|
|
864
864
|
'3/5 APPROVE', min_quorum: 1
|
|
865
|
-
)[:
|
|
865
|
+
)[:vote_tally][:denominator_composition]
|
|
866
866
|
|
|
867
867
|
by_label = comp[:observers].map { |o| [o[:role_label], o] }.to_h
|
|
868
868
|
assert_equal Consensus::SKIP_REASON_NO_VERDICT, by_label['b'][:reason]
|
|
@@ -148,7 +148,7 @@ module KairosMcp
|
|
|
148
148
|
'orchestrator_strategy' => 'exclude',
|
|
149
149
|
'persona_model' => 'claude-fable-5')
|
|
150
150
|
|
|
151
|
-
assert_equal '3/4 APPROVE', lost_one['
|
|
151
|
+
assert_equal '3/4 APPROVE', lost_one['vote_tally']['rule']
|
|
152
152
|
|
|
153
153
|
# The persona answers in the excluded slot's place, so the roster is
|
|
154
154
|
# whole and the rule it is judged by is the whole-roster one.
|
|
@@ -247,17 +247,17 @@ module KairosMcp
|
|
|
247
247
|
def test_exclude_applies_the_post_exclusion_rule
|
|
248
248
|
out = run_review(tool, 'orchestrator_model' => 'claude-opus-5',
|
|
249
249
|
'orchestrator_strategy' => 'exclude')
|
|
250
|
-
assert_equal '3/4 APPROVE', out['
|
|
250
|
+
assert_equal '3/4 APPROVE', out['vote_tally']['rule']
|
|
251
251
|
end
|
|
252
252
|
|
|
253
253
|
def test_the_post_exclusion_rule_does_not_apply_without_an_exclusion
|
|
254
254
|
out = run_review(tool, 'orchestrator_strategy' => 'exclude')
|
|
255
|
-
assert_equal '3/5 APPROVE', out['
|
|
255
|
+
assert_equal '3/5 APPROVE', out['vote_tally']['rule']
|
|
256
256
|
end
|
|
257
257
|
|
|
258
258
|
def test_the_default_strategy_keeps_the_full_roster_rule
|
|
259
259
|
out = run_review(tool)
|
|
260
|
-
assert_equal '3/5 APPROVE', out['
|
|
260
|
+
assert_equal '3/5 APPROVE', out['vote_tally']['rule']
|
|
261
261
|
end
|
|
262
262
|
|
|
263
263
|
# --- INV-E3: the container, through the tool ---
|
|
@@ -269,8 +269,8 @@ module KairosMcp
|
|
|
269
269
|
refute_includes off['reviews'].map { |r| r['role_label'] }, 'r'
|
|
270
270
|
assert_includes on['reviews'].map { |r| r['role_label'] }, 'r'
|
|
271
271
|
|
|
272
|
-
off_rec = off['
|
|
273
|
-
on_rec = on['
|
|
272
|
+
off_rec = off['vote_tally']['denominator_composition']['escalation']
|
|
273
|
+
on_rec = on['vote_tally']['denominator_composition']['escalation']
|
|
274
274
|
assert_equal({ 'requested' => false, 'escalated' => false, 'slots' => [], 'dispatched' => [] }, off_rec)
|
|
275
275
|
assert_equal({ 'requested' => true, 'escalated' => true, 'slots' => ['r'], 'dispatched' => ['r'] }, on_rec)
|
|
276
276
|
end
|
|
@@ -283,7 +283,7 @@ module KairosMcp
|
|
|
283
283
|
out = run_review(tool, 'escalate' => true,
|
|
284
284
|
'orchestrator_model' => 'claude-fable-5',
|
|
285
285
|
'orchestrator_strategy' => 'exclude')
|
|
286
|
-
rec = out['
|
|
286
|
+
rec = out['vote_tally']['denominator_composition']['escalation']
|
|
287
287
|
|
|
288
288
|
assert_equal({ 'requested' => true, 'escalated' => true, 'slots' => ['r'], 'dispatched' => [] }, rec)
|
|
289
289
|
refute_includes out['reviews'].map { |r| r['role_label'] }, 'r'
|
|
@@ -301,10 +301,10 @@ module KairosMcp
|
|
|
301
301
|
assert_equal %w[a b c], out['reviews'].map { |r| r['role_label'] }
|
|
302
302
|
assert_equal({ 'requested' => true, 'escalated' => false,
|
|
303
303
|
'slots' => [], 'dispatched' => [] },
|
|
304
|
-
out['
|
|
304
|
+
out['vote_tally']['denominator_composition']['escalation'])
|
|
305
305
|
|
|
306
306
|
# Distinguishable from the run that never asked.
|
|
307
|
-
never_asked = run_review(t)['
|
|
307
|
+
never_asked = run_review(t)['vote_tally']['denominator_composition']['escalation']
|
|
308
308
|
assert_equal false, never_asked['requested']
|
|
309
309
|
end
|
|
310
310
|
|
|
@@ -321,7 +321,7 @@ module KairosMcp
|
|
|
321
321
|
|
|
322
322
|
def test_a_divergent_answer_names_both_models_in_the_record
|
|
323
323
|
out = run_review(tool(observed: 'claude-opus-4-8'))
|
|
324
|
-
row = out['
|
|
324
|
+
row = out['vote_tally']['denominator_composition']['observers']
|
|
325
325
|
.find { |o| o['role_label'] == 'b' }
|
|
326
326
|
|
|
327
327
|
assert_equal 'observed', row['model_source']
|
|
@@ -491,7 +491,7 @@ module KairosMcp
|
|
|
491
491
|
def test_the_single_phase_record_keeps_the_slot_that_did_not_run
|
|
492
492
|
out = run_review(tool, 'orchestrator_model' => 'claude-opus-5',
|
|
493
493
|
'orchestrator_strategy' => 'exclude')
|
|
494
|
-
observers = out['
|
|
494
|
+
observers = out['vote_tally']['denominator_composition']['observers']
|
|
495
495
|
dropped = observers.find { |o| o['role_label'] == 'a' }
|
|
496
496
|
|
|
497
497
|
refute_nil dropped, 'the excluded slot vanished from the composition'
|
|
@@ -557,17 +557,17 @@ module KairosMcp
|
|
|
557
557
|
hollow = out['reviews'].find { |r| r['role_label'] == 'c' }
|
|
558
558
|
assert_equal 'SKIP', hollow['verdict']
|
|
559
559
|
assert_equal 'insubstantial', hollow['skip_reason']
|
|
560
|
-
assert_equal 2, out['
|
|
560
|
+
assert_equal 2, out['vote_tally']['successful_count']
|
|
561
561
|
# Three observers answered; one of them said nothing. What the
|
|
562
562
|
# configuration named is a different number and a different field.
|
|
563
|
-
assert_equal 3, out['
|
|
563
|
+
assert_equal 3, out['vote_tally']['observers_reporting']
|
|
564
564
|
end
|
|
565
565
|
|
|
566
566
|
# The denominator shrank, so the record has to say why. A transport
|
|
567
567
|
# failure and an empty reply both remove a slot and must stay apart.
|
|
568
568
|
def test_the_record_separates_a_hollow_reply_from_a_lost_one
|
|
569
569
|
comp = run_review(tool_answering_hollow_for('gpt-5.5'))
|
|
570
|
-
.dig('
|
|
570
|
+
.dig('vote_tally', 'denominator_composition', 'observers')
|
|
571
571
|
entry = comp.find { |o| o['role_label'] == 'c' }
|
|
572
572
|
|
|
573
573
|
assert_equal false, entry['counted']
|
|
@@ -584,9 +584,9 @@ module KairosMcp
|
|
|
584
584
|
plain = run_review(tool)
|
|
585
585
|
stale = run_review(tool({ 'substance_min_chars' => 400 }))
|
|
586
586
|
|
|
587
|
-
assert_operator plain['
|
|
588
|
-
assert_equal plain['
|
|
589
|
-
stale['
|
|
587
|
+
assert_operator plain['vote_tally']['successful_count'], :>, 0
|
|
588
|
+
assert_equal plain['vote_tally']['successful_count'],
|
|
589
|
+
stale['vote_tally']['successful_count']
|
|
590
590
|
assert_equal plain['reviews'].map { |r| r['verdict'] },
|
|
591
591
|
stale['reviews'].map { |r| r['verdict'] }
|
|
592
592
|
end
|
|
@@ -649,7 +649,7 @@ module KairosMcp
|
|
|
649
649
|
raw_text: 'APPROVE. A record written before provenance existed.' }],
|
|
650
650
|
'3/5 APPROVE', min_quorum: 1
|
|
651
651
|
)
|
|
652
|
-
row = parsed[:
|
|
652
|
+
row = parsed[:vote_tally][:denominator_composition][:observers].first
|
|
653
653
|
|
|
654
654
|
assert_equal 'declared', row[:model_source]
|
|
655
655
|
end
|
|
@@ -881,7 +881,7 @@ module KairosMcp
|
|
|
881
881
|
assert_equal 'inline', deliveries['d']
|
|
882
882
|
|
|
883
883
|
# The same column on the composition's dispatched rows (INV-R7).
|
|
884
|
-
observers = out['
|
|
884
|
+
observers = out['vote_tally']['denominator_composition']['observers']
|
|
885
885
|
by_label = observers.map { |o| [o['role_label'], o] }.to_h
|
|
886
886
|
assert_equal 'by_reference', by_label['c']['artifact_delivery']
|
|
887
887
|
assert_equal 'inline', by_label['d']['artifact_delivery']
|
|
@@ -901,7 +901,7 @@ module KairosMcp
|
|
|
901
901
|
|
|
902
902
|
assert_equal 'ok', out['status']
|
|
903
903
|
assert_equal ['c'], out['reviews'].map { |r| r['role_label'] }
|
|
904
|
-
observers = out['
|
|
904
|
+
observers = out['vote_tally']['denominator_composition']['observers']
|
|
905
905
|
refused = observers.find { |o| o['role_label'] == 'cli' }
|
|
906
906
|
assert_equal false, refused['counted']
|
|
907
907
|
assert_equal 'by_reference_unreachable_in_sandbox', refused['reason']
|
|
@@ -919,7 +919,7 @@ module KairosMcp
|
|
|
919
919
|
]
|
|
920
920
|
out = run_review(tool(roster))
|
|
921
921
|
|
|
922
|
-
observers = out['
|
|
922
|
+
observers = out['vote_tally']['denominator_composition']['observers']
|
|
923
923
|
refused = observers.find { |o| o['role_label'] == 'c' }
|
|
924
924
|
assert_equal 'by_reference_without_artifact_path', refused['reason']
|
|
925
925
|
end
|
|
@@ -946,8 +946,8 @@ module KairosMcp
|
|
|
946
946
|
'gpt-5.5' => { 'model_observed' => 'gpt-5-mini' }
|
|
947
947
|
}))
|
|
948
948
|
|
|
949
|
-
assert_equal 2, out['
|
|
950
|
-
excl = out['
|
|
949
|
+
assert_equal 2, out['vote_tally']['approve_count']
|
|
950
|
+
excl = out['vote_tally']['excluding_divergent']
|
|
951
951
|
assert_equal 1, excl['approve_count']
|
|
952
952
|
assert_equal 1, excl['successful_count']
|
|
953
953
|
|
|
@@ -1011,7 +1011,7 @@ module KairosMcp
|
|
|
1011
1011
|
def test_composition_rows_carry_the_seat_mark
|
|
1012
1012
|
out = run_review(tool(roster_one))
|
|
1013
1013
|
|
|
1014
|
-
observers = out['
|
|
1014
|
+
observers = out['vote_tally']['denominator_composition']['observers']
|
|
1015
1015
|
assert observers.all? { |o| o['seat'] == true }
|
|
1016
1016
|
end
|
|
1017
1017
|
end
|
|
@@ -545,7 +545,7 @@ module KairosMcp
|
|
|
545
545
|
case consensus[:reference_verdict]
|
|
546
546
|
when 'APPROVE' then nil
|
|
547
547
|
when 'INSUFFICIENT'
|
|
548
|
-
FeedbackFormatter.build_insufficient(consensus[:
|
|
548
|
+
FeedbackFormatter.build_insufficient(consensus[:vote_tally][:reason] || 'quorum not met')
|
|
549
549
|
else
|
|
550
550
|
FeedbackFormatter.build(sanitized_findings)
|
|
551
551
|
end
|
|
@@ -558,7 +558,7 @@ module KairosMcp
|
|
|
558
558
|
# closed by the operator's declaration, outside this record.
|
|
559
559
|
'reference_verdict' => consensus[:reference_verdict],
|
|
560
560
|
'feedback_text' => feedback_text,
|
|
561
|
-
'
|
|
561
|
+
'vote_tally' => hash_to_string_keys(consensus[:vote_tally]),
|
|
562
562
|
'reviews' => consensus[:reviews].map { |r|
|
|
563
563
|
ReviewSerializer.payload_row(r, include_raw_text: include_raw_text)
|
|
564
564
|
},
|
|
@@ -428,7 +428,7 @@ module KairosMcp
|
|
|
428
428
|
case consensus[:reference_verdict]
|
|
429
429
|
when 'APPROVE' then nil
|
|
430
430
|
when 'INSUFFICIENT'
|
|
431
|
-
FeedbackFormatter.build_insufficient(consensus[:
|
|
431
|
+
FeedbackFormatter.build_insufficient(consensus[:vote_tally][:reason] || 'quorum not met')
|
|
432
432
|
else
|
|
433
433
|
FeedbackFormatter.build(sanitized_findings)
|
|
434
434
|
end
|
|
@@ -441,7 +441,7 @@ module KairosMcp
|
|
|
441
441
|
# closed by the operator's declaration, outside this record.
|
|
442
442
|
'reference_verdict' => consensus[:reference_verdict],
|
|
443
443
|
'feedback_text' => feedback_text,
|
|
444
|
-
'
|
|
444
|
+
'vote_tally' => hash_to_string_keys(consensus[:vote_tally]),
|
|
445
445
|
'reviews' => consensus[:reviews].map { |r|
|
|
446
446
|
ReviewSerializer.payload_row(r, include_raw_text: include_raw_text)
|
|
447
447
|
},
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kairos-chain
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 3.
|
|
4
|
+
version: 3.71.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Masaomi Hatakeyama
|
|
@@ -303,7 +303,9 @@ files:
|
|
|
303
303
|
- templates/knowledge/mcp_to_saas_development_workflow/mcp_to_saas_development_workflow.md
|
|
304
304
|
- templates/knowledge/multi_agent_design_workflow/multi_agent_design_workflow.md
|
|
305
305
|
- templates/knowledge/multi_agent_design_workflow_jp/multi_agent_design_workflow_jp.md
|
|
306
|
+
- templates/knowledge/multi_llm_review_workflow/assets/review_dashboard.html
|
|
306
307
|
- templates/knowledge/multi_llm_review_workflow/multi_llm_review_workflow.md
|
|
308
|
+
- templates/knowledge/multi_llm_review_workflow/scripts/render_dashboard.rb
|
|
307
309
|
- templates/knowledge/multi_llm_reviewer_evaluation/multi_llm_reviewer_evaluation.md
|
|
308
310
|
- templates/knowledge/persona_definitions/persona_definitions.md
|
|
309
311
|
- templates/knowledge/project_orientation_report/assets/report_template.html
|