loki-mode 7.10.1 → 7.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +2 -2
- package/VERSION +1 -1
- package/autonomy/loki +297 -0
- package/autonomy/run.sh +22 -0
- package/dashboard/__init__.py +1 -1
- package/dashboard/server.py +242 -0
- package/dashboard/static/cost.html +274 -0
- package/dashboard/static/index.html +94 -0
- package/docs/INSTALLATION.md +1 -1
- package/docs/R3-COST-OBSERVABILITY-DESIGN.md +147 -0
- package/loki-ts/dist/loki.js +144 -144
- package/mcp/__init__.py +1 -1
- package/package.json +1 -1
|
@@ -0,0 +1,274 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<!--
|
|
3
|
+
Loki Mode - Cost + observability panel (R3, zero-build standalone).
|
|
4
|
+
|
|
5
|
+
Self-contained: all CSS + JS inlined, no external resources. Fetches
|
|
6
|
+
/api/cost/timeline and renders project total, a budget gauge that warns at
|
|
7
|
+
80% (before the hard cap at 100%), per-run cost history, model-routing
|
|
8
|
+
breakdown, and an inline-SVG cumulative-cost line for the current run.
|
|
9
|
+
|
|
10
|
+
Anti-surprise-cost wedge: cost is shown transparently. Uncollected cost is
|
|
11
|
+
shown as "not recorded", never a fabricated $0.00.
|
|
12
|
+
-->
|
|
13
|
+
<html lang="en">
|
|
14
|
+
<head>
|
|
15
|
+
<meta charset="utf-8">
|
|
16
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
17
|
+
<title>Loki Mode - Cost and Observability</title>
|
|
18
|
+
<style>
|
|
19
|
+
:root {
|
|
20
|
+
--bg: #0f1115; --panel: #171a21; --panel-2: #1d2129; --border: #2a2f3a;
|
|
21
|
+
--text: #e7e9ee; --muted: #9aa1ad; --faint: #6b7280; --accent: #6f7bf7;
|
|
22
|
+
--green: #34d399; --red: #f87171; --amber: #fbbf24;
|
|
23
|
+
--mono: ui-monospace, "SF Mono", "Menlo", "Consolas", monospace;
|
|
24
|
+
--sans: 'Inter', system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
|
25
|
+
}
|
|
26
|
+
* { box-sizing: border-box; }
|
|
27
|
+
body { margin: 0; background: var(--bg); color: var(--text); font-family: var(--sans); line-height: 1.5; }
|
|
28
|
+
a { color: var(--accent); text-decoration: none; }
|
|
29
|
+
a:hover { text-decoration: underline; }
|
|
30
|
+
.wrap { max-width: 960px; margin: 0 auto; padding: 40px 20px 80px; }
|
|
31
|
+
.head { display: flex; align-items: baseline; justify-content: space-between; margin-bottom: 8px; }
|
|
32
|
+
h1 { font-size: 24px; font-weight: 650; letter-spacing: -0.3px; margin: 0; }
|
|
33
|
+
h2 { font-size: 15px; font-weight: 600; color: var(--muted); margin: 30px 0 12px; text-transform: uppercase; letter-spacing: 0.5px; }
|
|
34
|
+
.head a { font-size: 13px; }
|
|
35
|
+
.sub { color: var(--muted); font-size: 14px; margin: 0 0 26px; }
|
|
36
|
+
.cards { display: flex; gap: 14px; flex-wrap: wrap; }
|
|
37
|
+
.card { flex: 1 1 200px; background: var(--panel); border: 1px solid var(--border); border-radius: 12px; padding: 16px 18px; }
|
|
38
|
+
.card .label { color: var(--muted); font-size: 12px; text-transform: uppercase; letter-spacing: 0.5px; }
|
|
39
|
+
.card .val { font-family: var(--mono); font-size: 26px; font-weight: 650; margin-top: 6px; }
|
|
40
|
+
.card .note { color: var(--faint); font-size: 12px; margin-top: 4px; }
|
|
41
|
+
.gauge { background: var(--panel); border: 1px solid var(--border); border-radius: 12px; padding: 18px; margin-top: 14px; }
|
|
42
|
+
.gauge .top { display: flex; justify-content: space-between; align-items: baseline; }
|
|
43
|
+
.gauge .pct { font-family: var(--mono); font-weight: 650; font-size: 18px; }
|
|
44
|
+
.bar { position: relative; height: 14px; background: var(--panel-2); border-radius: 8px; margin: 12px 0 6px; overflow: hidden; }
|
|
45
|
+
.bar .fill { height: 100%; border-radius: 8px; transition: width .3s; }
|
|
46
|
+
.bar .warnline { position: absolute; top: -3px; bottom: -3px; width: 2px; background: var(--amber); left: 80%; }
|
|
47
|
+
.fill.ok { background: var(--green); }
|
|
48
|
+
.fill.warn { background: var(--amber); }
|
|
49
|
+
.fill.exceeded { background: var(--red); }
|
|
50
|
+
.status { font-size: 13px; margin-top: 8px; }
|
|
51
|
+
.status.ok { color: var(--green); }
|
|
52
|
+
.status.warn { color: var(--amber); }
|
|
53
|
+
.status.exceeded { color: var(--red); }
|
|
54
|
+
.status.none { color: var(--muted); }
|
|
55
|
+
.cap-note { color: var(--faint); font-size: 12px; margin-top: 6px; }
|
|
56
|
+
table { width: 100%; border-collapse: collapse; font-size: 13px; }
|
|
57
|
+
th, td { text-align: left; padding: 8px 10px; border-bottom: 1px solid var(--border); }
|
|
58
|
+
th { color: var(--muted); font-weight: 600; font-size: 12px; text-transform: uppercase; letter-spacing: 0.4px; }
|
|
59
|
+
td.num, th.num { text-align: right; font-family: var(--mono); }
|
|
60
|
+
.badge { font-size: 12px; font-weight: 600; padding: 2px 8px; border-radius: 6px; border: 1px solid var(--border); }
|
|
61
|
+
.b-approve { color: var(--green); border-color: rgba(52,211,153,0.4); }
|
|
62
|
+
.b-reject { color: var(--red); border-color: rgba(248,113,113,0.4); }
|
|
63
|
+
.b-concern { color: var(--amber); border-color: rgba(251,191,36,0.4); }
|
|
64
|
+
.panel { background: var(--panel); border: 1px solid var(--border); border-radius: 12px; padding: 18px; }
|
|
65
|
+
.empty { color: var(--muted); background: var(--panel); border: 1px solid var(--border); border-radius: 12px; padding: 24px; text-align: center; }
|
|
66
|
+
.empty code { font-family: var(--mono); color: var(--text); background: var(--panel-2); padding: 2px 6px; border-radius: 5px; }
|
|
67
|
+
.mono { font-family: var(--mono); }
|
|
68
|
+
.muted { color: var(--muted); }
|
|
69
|
+
svg { display: block; width: 100%; height: 140px; }
|
|
70
|
+
.modelrow { display: flex; align-items: center; gap: 10px; margin: 6px 0; }
|
|
71
|
+
.modelrow .name { width: 130px; font-size: 13px; }
|
|
72
|
+
.modelrow .mbar { flex: 1; height: 10px; background: var(--panel-2); border-radius: 6px; overflow: hidden; }
|
|
73
|
+
.modelrow .mbar .mfill { height: 100%; background: var(--accent); border-radius: 6px; }
|
|
74
|
+
.modelrow .mval { width: 90px; text-align: right; font-family: var(--mono); font-size: 13px; }
|
|
75
|
+
</style>
|
|
76
|
+
</head>
|
|
77
|
+
<body>
|
|
78
|
+
<div class="wrap">
|
|
79
|
+
<div class="head">
|
|
80
|
+
<h1>Cost and Observability</h1>
|
|
81
|
+
<a href="/">Back to dashboard</a>
|
|
82
|
+
</div>
|
|
83
|
+
<p class="sub">Transparent cost: per-run and per-project spend, model routing, token burn, and budget caps that warn before they stop. No surprise bills.</p>
|
|
84
|
+
<div id="content"><p class="sub">Loading...</p></div>
|
|
85
|
+
</div>
|
|
86
|
+
<script>
|
|
87
|
+
(function () {
|
|
88
|
+
"use strict";
|
|
89
|
+
function esc(s) {
|
|
90
|
+
s = (s === null || s === undefined) ? "" : String(s);
|
|
91
|
+
return s.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">")
|
|
92
|
+
.replace(/"/g, """).replace(/'/g, "'");
|
|
93
|
+
}
|
|
94
|
+
function fmtUsd(n) {
|
|
95
|
+
if (n === null || n === undefined) return "not recorded";
|
|
96
|
+
n = Number(n);
|
|
97
|
+
if (!isFinite(n)) return "not recorded";
|
|
98
|
+
var s = n.toFixed(4).replace(/0+$/, "").replace(/\.$/, "");
|
|
99
|
+
if (s.indexOf(".") === -1) s += ".00";
|
|
100
|
+
else if (s.split(".")[1].length === 1) s += "0";
|
|
101
|
+
return "$" + s;
|
|
102
|
+
}
|
|
103
|
+
function badgeClass(v) {
|
|
104
|
+
v = String(v || "").toUpperCase();
|
|
105
|
+
if (v.indexOf("APPROVE") === 0 || v === "PASS" || v === "PASSED") return "b-approve";
|
|
106
|
+
if (v.indexOf("REJECT") === 0 || v.indexOf("BLOCK") === 0 || v === "FAIL") return "b-reject";
|
|
107
|
+
if (v.indexOf("CONCERN") === 0) return "b-concern";
|
|
108
|
+
return "";
|
|
109
|
+
}
|
|
110
|
+
function statusText(st) {
|
|
111
|
+
if (st === "exceeded") return "Budget cap reached. The run is paused to prevent a surprise bill.";
|
|
112
|
+
if (st === "warn") return "Approaching budget cap (80% or more used). Warning only, the run continues.";
|
|
113
|
+
if (st === "ok") return "Within budget.";
|
|
114
|
+
return "No budget cap set. Set LOKI_BUDGET_LIMIT to cap spend.";
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
function renderBudget(b) {
|
|
118
|
+
if (!b) return "";
|
|
119
|
+
var html = '<h2>Budget</h2>';
|
|
120
|
+
if (b.limit === null || b.limit === undefined) {
|
|
121
|
+
html += '<div class="panel"><div class="status none">' + statusText("none") + '</div>' +
|
|
122
|
+
'<div class="cap-note">When a cap is set, Loki warns at 80% and hard-stops at 100%.</div></div>';
|
|
123
|
+
return html;
|
|
124
|
+
}
|
|
125
|
+
var pct = (b.percent_used === null || b.percent_used === undefined) ? 0 : Number(b.percent_used);
|
|
126
|
+
var fillPct = Math.max(0, Math.min(100, pct));
|
|
127
|
+
var st = esc(b.status || "ok");
|
|
128
|
+
html += '<div class="gauge">' +
|
|
129
|
+
'<div class="top"><span class="muted">' + fmtUsd(b.used) + ' of ' + fmtUsd(b.limit) + ' used</span>' +
|
|
130
|
+
'<span class="pct">' + pct.toFixed(1) + '%</span></div>' +
|
|
131
|
+
'<div class="bar"><div class="fill ' + st + '" style="width:' + fillPct + '%"></div>' +
|
|
132
|
+
'<div class="warnline" title="80% warn threshold"></div></div>' +
|
|
133
|
+
'<div class="status ' + st + '">' + statusText(b.status) + '</div>' +
|
|
134
|
+
'<div class="cap-note">Remaining: ' + fmtUsd(b.remaining) +
|
|
135
|
+
'. Warns at ' + esc(b.warn_threshold_percent) + '%, hard-stops at 100%.</div>' +
|
|
136
|
+
'</div>';
|
|
137
|
+
return html;
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
function renderModelBreakdown(runs) {
|
|
141
|
+
// Aggregate per-run cost by model for routing visibility.
|
|
142
|
+
var byModel = {};
|
|
143
|
+
var total = 0;
|
|
144
|
+
for (var i = 0; i < runs.length; i++) {
|
|
145
|
+
var m = runs[i].model || "unknown";
|
|
146
|
+
var c = runs[i].cost_usd;
|
|
147
|
+
if (c === null || c === undefined) continue;
|
|
148
|
+
c = Number(c);
|
|
149
|
+
if (!isFinite(c)) continue;
|
|
150
|
+
byModel[m] = (byModel[m] || 0) + c;
|
|
151
|
+
total += c;
|
|
152
|
+
}
|
|
153
|
+
var keys = Object.keys(byModel);
|
|
154
|
+
if (keys.length === 0) return "";
|
|
155
|
+
keys.sort(function (a, b) { return byModel[b] - byModel[a]; });
|
|
156
|
+
var html = '<h2>Model routing (by spend)</h2><div class="panel">';
|
|
157
|
+
for (var k = 0; k < keys.length; k++) {
|
|
158
|
+
var name = keys[k];
|
|
159
|
+
var val = byModel[name];
|
|
160
|
+
var w = total > 0 ? (val / total * 100) : 0;
|
|
161
|
+
html += '<div class="modelrow"><span class="name mono">' + esc(name) + '</span>' +
|
|
162
|
+
'<span class="mbar"><span class="mfill" style="width:' + w.toFixed(1) + '%"></span></span>' +
|
|
163
|
+
'<span class="mval">' + fmtUsd(val) + '</span></div>';
|
|
164
|
+
}
|
|
165
|
+
html += '</div>';
|
|
166
|
+
return html;
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
function renderCurrentRun(cr) {
|
|
170
|
+
var html = '<h2>Current run (cost over iterations)</h2>';
|
|
171
|
+
if (!cr || !cr.cost_recorded || !cr.iterations || cr.iterations.length === 0) {
|
|
172
|
+
html += '<div class="empty">No iteration cost recorded for the current run yet.' +
|
|
173
|
+
' Cost appears once a run produces efficiency records.</div>';
|
|
174
|
+
return html;
|
|
175
|
+
}
|
|
176
|
+
var its = cr.iterations;
|
|
177
|
+
// Inline SVG cumulative line.
|
|
178
|
+
var W = 900, H = 140, pad = 8;
|
|
179
|
+
var maxCum = 0;
|
|
180
|
+
for (var i = 0; i < its.length; i++) {
|
|
181
|
+
var cv = Number(its[i].cumulative_usd) || 0;
|
|
182
|
+
if (cv > maxCum) maxCum = cv;
|
|
183
|
+
}
|
|
184
|
+
if (maxCum <= 0) maxCum = 1;
|
|
185
|
+
var pts = [];
|
|
186
|
+
for (var j = 0; j < its.length; j++) {
|
|
187
|
+
var x = its.length === 1 ? W / 2 : pad + (j / (its.length - 1)) * (W - 2 * pad);
|
|
188
|
+
var y = H - pad - ((Number(its[j].cumulative_usd) || 0) / maxCum) * (H - 2 * pad);
|
|
189
|
+
pts.push(x.toFixed(1) + "," + y.toFixed(1));
|
|
190
|
+
}
|
|
191
|
+
var poly = '<svg viewBox="0 0 ' + W + ' ' + H + '" preserveAspectRatio="none">' +
|
|
192
|
+
'<polyline fill="none" stroke="#6f7bf7" stroke-width="2" points="' + pts.join(" ") + '"/>' +
|
|
193
|
+
'</svg>';
|
|
194
|
+
html += '<div class="panel"><div class="muted" style="font-size:13px;margin-bottom:6px;">' +
|
|
195
|
+
'Cumulative spend this run: <span class="mono">' + fmtUsd(cr.total_usd) + '</span></div>' +
|
|
196
|
+
poly + '</div>';
|
|
197
|
+
// Per-iteration table.
|
|
198
|
+
var rows = "";
|
|
199
|
+
for (var r = 0; r < its.length; r++) {
|
|
200
|
+
var it = its[r];
|
|
201
|
+
rows += '<tr><td class="num mono">' + esc(it.iteration) + '</td>' +
|
|
202
|
+
'<td class="mono">' + esc(it.model) + '</td>' +
|
|
203
|
+
'<td>' + esc(it.phase) + '</td>' +
|
|
204
|
+
'<td class="num">' + esc(it.input_tokens) + '</td>' +
|
|
205
|
+
'<td class="num">' + esc(it.output_tokens) + '</td>' +
|
|
206
|
+
'<td class="num mono">' + fmtUsd(it.cost_usd) + '</td>' +
|
|
207
|
+
'<td class="num mono">' + fmtUsd(it.cumulative_usd) + '</td></tr>';
|
|
208
|
+
}
|
|
209
|
+
html += '<table style="margin-top:14px;"><thead><tr>' +
|
|
210
|
+
'<th class="num">Iter</th><th>Model</th><th>Phase</th>' +
|
|
211
|
+
'<th class="num">Input tok</th><th class="num">Output tok</th>' +
|
|
212
|
+
'<th class="num">Cost</th><th class="num">Cumulative</th>' +
|
|
213
|
+
'</tr></thead><tbody>' + rows + '</tbody></table>';
|
|
214
|
+
return html;
|
|
215
|
+
}
|
|
216
|
+
|
|
217
|
+
function renderRuns(runs) {
|
|
218
|
+
var html = '<h2>Run history (per-run cost)</h2>';
|
|
219
|
+
if (!runs || runs.length === 0) {
|
|
220
|
+
html += '<div class="empty">No completed runs yet. Per-run cost history' +
|
|
221
|
+
' comes from proof-of-run artifacts (<code>.loki/proofs/</code>).</div>';
|
|
222
|
+
return html;
|
|
223
|
+
}
|
|
224
|
+
var rows = "";
|
|
225
|
+
for (var i = 0; i < runs.length; i++) {
|
|
226
|
+
var p = runs[i];
|
|
227
|
+
var verdict = p.final_verdict ?
|
|
228
|
+
'<span class="badge ' + badgeClass(p.final_verdict) + '">' + esc(p.final_verdict) + '</span>' : '';
|
|
229
|
+
rows += '<tr><td class="mono">' + esc(p.run_id) + '</td>' +
|
|
230
|
+
'<td class="muted">' + esc(p.generated_at || "") + '</td>' +
|
|
231
|
+
'<td class="mono">' + esc(p.model || "") + '</td>' +
|
|
232
|
+
'<td class="num">' + (p.files_changed === null || p.files_changed === undefined ? "" : esc(p.files_changed)) + '</td>' +
|
|
233
|
+
'<td>' + verdict + '</td>' +
|
|
234
|
+
'<td class="num mono">' + fmtUsd(p.cost_usd) + '</td></tr>';
|
|
235
|
+
}
|
|
236
|
+
html += '<table><thead><tr>' +
|
|
237
|
+
'<th>Run</th><th>When</th><th>Model</th><th class="num">Files</th>' +
|
|
238
|
+
'<th>Verdict</th><th class="num">Cost</th>' +
|
|
239
|
+
'</tr></thead><tbody>' + rows + '</tbody></table>';
|
|
240
|
+
return html;
|
|
241
|
+
}
|
|
242
|
+
|
|
243
|
+
function render(d) {
|
|
244
|
+
var c = document.getElementById("content");
|
|
245
|
+
var b = d.budget || {};
|
|
246
|
+
var cards = '<div class="cards">' +
|
|
247
|
+
'<div class="card"><div class="label">Project total</div>' +
|
|
248
|
+
'<div class="val">' + fmtUsd(d.project_total_usd) + '</div>' +
|
|
249
|
+
'<div class="note">' + esc(d.runs_count) + ' run(s) recorded</div></div>' +
|
|
250
|
+
'<div class="card"><div class="label">Current run</div>' +
|
|
251
|
+
'<div class="val">' + fmtUsd(d.current_run ? d.current_run.total_usd : null) + '</div>' +
|
|
252
|
+
'<div class="note">' + (d.current_run && d.current_run.iterations ? d.current_run.iterations.length : 0) + ' iteration(s)</div></div>' +
|
|
253
|
+
'<div class="card"><div class="label">Budget status</div>' +
|
|
254
|
+
'<div class="val ' + esc(b.status || "none") + '" style="font-size:20px;text-transform:capitalize;">' + esc(b.status || "none") + '</div>' +
|
|
255
|
+
'<div class="note">' + (b.limit ? fmtUsd(b.used) + ' / ' + fmtUsd(b.limit) : "no cap set") + '</div></div>' +
|
|
256
|
+
'</div>';
|
|
257
|
+
c.innerHTML = cards +
|
|
258
|
+
renderBudget(d.budget) +
|
|
259
|
+
renderCurrentRun(d.current_run) +
|
|
260
|
+
renderModelBreakdown(d.runs || []) +
|
|
261
|
+
renderRuns(d.runs);
|
|
262
|
+
}
|
|
263
|
+
function renderError(msg) {
|
|
264
|
+
document.getElementById("content").innerHTML =
|
|
265
|
+
'<div class="empty">Could not load cost data. ' + esc(msg || "") + "</div>";
|
|
266
|
+
}
|
|
267
|
+
fetch("/api/cost/timeline", { headers: { "Accept": "application/json" } })
|
|
268
|
+
.then(function (r) { if (!r.ok) throw new Error("HTTP " + r.status); return r.json(); })
|
|
269
|
+
.then(function (d) { render(d || {}); })
|
|
270
|
+
.catch(function (e) { renderError(e && e.message); });
|
|
271
|
+
})();
|
|
272
|
+
</script>
|
|
273
|
+
</body>
|
|
274
|
+
</html>
|
|
@@ -409,6 +409,40 @@
|
|
|
409
409
|
display: block;
|
|
410
410
|
}
|
|
411
411
|
|
|
412
|
+
.budget-banner {
|
|
413
|
+
position: fixed;
|
|
414
|
+
top: 0;
|
|
415
|
+
left: 0;
|
|
416
|
+
right: 0;
|
|
417
|
+
padding: 8px 16px;
|
|
418
|
+
text-align: center;
|
|
419
|
+
font-size: 13px;
|
|
420
|
+
font-weight: 600;
|
|
421
|
+
display: none;
|
|
422
|
+
z-index: 1000;
|
|
423
|
+
color: #201515;
|
|
424
|
+
}
|
|
425
|
+
|
|
426
|
+
.budget-banner.show {
|
|
427
|
+
display: block;
|
|
428
|
+
}
|
|
429
|
+
|
|
430
|
+
.budget-banner.warn {
|
|
431
|
+
background: var(--loki-warning);
|
|
432
|
+
}
|
|
433
|
+
|
|
434
|
+
.budget-banner.exceeded {
|
|
435
|
+
background: var(--loki-red);
|
|
436
|
+
color: #fff;
|
|
437
|
+
}
|
|
438
|
+
|
|
439
|
+
.budget-banner a {
|
|
440
|
+
color: inherit;
|
|
441
|
+
text-decoration: underline;
|
|
442
|
+
margin-left: 10px;
|
|
443
|
+
font-weight: 600;
|
|
444
|
+
}
|
|
445
|
+
|
|
412
446
|
/* Loading state */
|
|
413
447
|
.loading {
|
|
414
448
|
display: flex;
|
|
@@ -581,6 +615,15 @@
|
|
|
581
615
|
Offline - showing cached data
|
|
582
616
|
</div>
|
|
583
617
|
|
|
618
|
+
<!-- Budget Banner (R3 anti-surprise-cost): persistent, visible on every
|
|
619
|
+
page without opening the Cost panel. Amber at >=80% (warn), red at
|
|
620
|
+
>=100% (exceeded). Driven by the existing WebSocket budget_status push
|
|
621
|
+
and a polling fallback against /api/cost/timeline. -->
|
|
622
|
+
<div class="budget-banner" id="budget-banner" role="status" aria-live="polite">
|
|
623
|
+
<span id="budget-banner-text"></span>
|
|
624
|
+
<a href="/cost" id="budget-banner-link">View cost</a>
|
|
625
|
+
</div>
|
|
626
|
+
|
|
584
627
|
<!-- Dashboard Layout -->
|
|
585
628
|
<div class="dashboard-layout">
|
|
586
629
|
<!-- Sidebar -->
|
|
@@ -13609,6 +13652,57 @@ document.addEventListener('DOMContentLoaded', function() {
|
|
|
13609
13652
|
document.getElementById('offline-banner').classList.add('show');
|
|
13610
13653
|
}
|
|
13611
13654
|
|
|
13655
|
+
// R3 budget banner: a persistent, page-wide indicator so a user running an
|
|
13656
|
+
// overnight job sees the 80% budget warning WITHOUT opening the Cost panel.
|
|
13657
|
+
// It reuses the existing WebSocket push (budget_status -> api:budget_status
|
|
13658
|
+
// on the shared API client) and falls back to polling /api/cost/timeline.
|
|
13659
|
+
(function initBudgetBanner() {
|
|
13660
|
+
var banner = document.getElementById('budget-banner');
|
|
13661
|
+
var textEl = document.getElementById('budget-banner-text');
|
|
13662
|
+
if (!banner || !textEl) return;
|
|
13663
|
+
|
|
13664
|
+
function renderBudget(b) {
|
|
13665
|
+
if (!b || (b.status !== 'warn' && b.status !== 'exceeded')) {
|
|
13666
|
+
banner.classList.remove('show', 'warn', 'exceeded');
|
|
13667
|
+
return;
|
|
13668
|
+
}
|
|
13669
|
+
// Honest copy: "Budget at 82% - hard stop at 100%."
|
|
13670
|
+
var pct = (b.percent_used === null || b.percent_used === undefined)
|
|
13671
|
+
? null : Number(b.percent_used);
|
|
13672
|
+
var pctTxt = (pct === null || !isFinite(pct)) ? '' : Math.round(pct) + '%';
|
|
13673
|
+
var msg;
|
|
13674
|
+
if (b.status === 'exceeded') {
|
|
13675
|
+
msg = 'Budget cap reached' + (pctTxt ? ' (' + pctTxt + ')' : '') +
|
|
13676
|
+
'. The run is paused to prevent a surprise bill.';
|
|
13677
|
+
} else {
|
|
13678
|
+
msg = 'Budget at ' + (pctTxt || 'over 80%') + ' - hard stop at 100%.';
|
|
13679
|
+
}
|
|
13680
|
+
textEl.textContent = msg;
|
|
13681
|
+
banner.classList.remove('warn', 'exceeded');
|
|
13682
|
+
banner.classList.add('show', b.status);
|
|
13683
|
+
}
|
|
13684
|
+
|
|
13685
|
+
// Polling fallback (the WS push is best-effort; polling guarantees the
|
|
13686
|
+
// banner is correct even on a freshly opened page or after a reconnect).
|
|
13687
|
+
function poll() {
|
|
13688
|
+
fetch('/api/cost/timeline', { headers: { 'Accept': 'application/json' } })
|
|
13689
|
+
.then(function (r) { return r.ok ? r.json() : null; })
|
|
13690
|
+
.then(function (d) { if (d && d.budget) renderBudget(d.budget); })
|
|
13691
|
+
.catch(function () { /* offline / no endpoint: leave banner as-is */ });
|
|
13692
|
+
}
|
|
13693
|
+
poll();
|
|
13694
|
+
setInterval(poll, 15000);
|
|
13695
|
+
|
|
13696
|
+
// Reuse the existing shared WebSocket client for the proactive push.
|
|
13697
|
+
try {
|
|
13698
|
+
var api = LokiDashboard.getApiClient({ baseUrl: window.location.origin });
|
|
13699
|
+
api.addEventListener('api:budget_status', function (e) {
|
|
13700
|
+
renderBudget(e && e.detail);
|
|
13701
|
+
});
|
|
13702
|
+
api.connect().catch(function () { /* polling fallback still covers it */ });
|
|
13703
|
+
} catch (err) { /* polling fallback still covers it */ }
|
|
13704
|
+
})();
|
|
13705
|
+
|
|
13612
13706
|
// Mobile menu toggle
|
|
13613
13707
|
var mobileMenuBtn = document.getElementById('mobile-menu-btn');
|
|
13614
13708
|
var sidebar = document.getElementById('sidebar');
|
package/docs/INSTALLATION.md
CHANGED
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
# R3: Cost + Observability Dashboard (anti-surprise-cost wedge)
|
|
2
|
+
|
|
3
|
+
Design note. Verified against live source on 2026-06-03 (v7.8.3 worktree).
|
|
4
|
+
No version bumps, no commits to main. This file is a design artifact for the
|
|
5
|
+
integrator; cherry-pick the implementation files listed at the bottom.
|
|
6
|
+
|
|
7
|
+
## Goal
|
|
8
|
+
|
|
9
|
+
Counter the #1 competitor churn driver (surprise cost) with TRANSPARENT cost:
|
|
10
|
+
per-run and per-project cost USD over time, model-routing visibility, token
|
|
11
|
+
burn, and budget caps that WARN before the cap (at 80%) rather than surprise
|
|
12
|
+
the user, while preserving the existing hard-stop at 100%.
|
|
13
|
+
|
|
14
|
+
## What already exists (reuse, do NOT duplicate)
|
|
15
|
+
|
|
16
|
+
| Surface | Location | Has | Missing for R3 |
|
|
17
|
+
|---|---|---|---|
|
|
18
|
+
| Aggregate cost | `dashboard/server.py` `GET /api/cost` (~4391) | totals, by_phase, by_model, basic budget | per-run history, time-series, warn status |
|
|
19
|
+
| Budget status | `dashboard/server.py` `GET /api/budget` (~4498) | limit, used, exceeded, remaining | warn-at-80% status field |
|
|
20
|
+
| Pricing | `dashboard/server.py` `GET /api/pricing` (~4575) | model price table | -- |
|
|
21
|
+
| Hard cap | `autonomy/run.sh` `check_budget_limit()` (8333) | pause + signal at >=100% | warn at 80% (no pause) |
|
|
22
|
+
| Bun cap | `loki-ts/src/runner/budget.ts` `checkBudgetLimit()` | parity of hard cap | warn at 80% |
|
|
23
|
+
| Cost lib | `autonomy/lib/efficiency_cost.py` `collect_efficiency` | sum cost_usd + tokens, honest None | (this is the shared lib to reuse) |
|
|
24
|
+
| Per-run proof | `.loki/proofs/<run_id>/proof.json` via proof-generator.py | run_id, generated_at, cost.usd, files_changed.count, council.final_verdict, provider.model | (source for per-run history) |
|
|
25
|
+
| Productivity CLI | `autonomy/loki` `cmd_metrics()` (17837) | session productivity report | dedicated cost view |
|
|
26
|
+
| Estimate CLI | `autonomy/loki` `cmd_plan()` | pre-run cost ESTIMATE | actuals |
|
|
27
|
+
|
|
28
|
+
## Critical data-source fact (verified)
|
|
29
|
+
|
|
30
|
+
`autonomy/run.sh:3186` wipes `.loki/metrics/efficiency/iteration-*.json` at the
|
|
31
|
+
start of every run. Therefore:
|
|
32
|
+
|
|
33
|
+
- `.loki/metrics/efficiency/` only ever holds the CURRENT run's iterations. It
|
|
34
|
+
is the source for the INTRA-RUN time-series (per-iteration, now carries a
|
|
35
|
+
`timestamp` field, run.sh:4246).
|
|
36
|
+
- Per-RUN and per-PROJECT cost OVER TIME must come from
|
|
37
|
+
`.loki/proofs/<run_id>/proof.json` (persistent, one dir per run, carries
|
|
38
|
+
`cost.usd` + `generated_at` + `provider.model`). This is the real
|
|
39
|
+
"cost over time" series. Using efficiency/ for it would silently show one run.
|
|
40
|
+
|
|
41
|
+
## Deliverables
|
|
42
|
+
|
|
43
|
+
### 1. New endpoint: `GET /api/cost/timeline` (dashboard/server.py)
|
|
44
|
+
|
|
45
|
+
Read-only. Returns two honest series plus a budget block:
|
|
46
|
+
|
|
47
|
+
```json
|
|
48
|
+
{
|
|
49
|
+
"current_run": {
|
|
50
|
+
"iterations": [
|
|
51
|
+
{"iteration": 1, "timestamp": "...", "model": "sonnet",
|
|
52
|
+
"phase": "build", "input_tokens": 1500, "output_tokens": 500,
|
|
53
|
+
"cost_usd": 0.05, "cumulative_usd": 0.05}
|
|
54
|
+
],
|
|
55
|
+
"total_usd": 0.05,
|
|
56
|
+
"cost_recorded": true
|
|
57
|
+
},
|
|
58
|
+
"runs": [
|
|
59
|
+
{"run_id": "...", "generated_at": "...", "model": "sonnet",
|
|
60
|
+
"cost_usd": 1.84, "files_changed": 3, "final_verdict": "APPROVE"}
|
|
61
|
+
],
|
|
62
|
+
"project_total_usd": 1.89,
|
|
63
|
+
"runs_count": 1,
|
|
64
|
+
"budget": {
|
|
65
|
+
"limit": 50.0, "used": 1.89, "remaining": 48.11,
|
|
66
|
+
"percent_used": 3.78, "status": "ok",
|
|
67
|
+
"warn_threshold_percent": 80, "exceeded": false
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
- `current_run.iterations` from `.loki/metrics/efficiency/iteration-*.json`,
|
|
73
|
+
sorted by iteration, with a running `cumulative_usd`. Cost per record:
|
|
74
|
+
prefer `cost_usd`; if null, price from tokens via the EXISTING
|
|
75
|
+
`_calculate_model_cost` helper (do not add a new pricer).
|
|
76
|
+
- `runs` from `.loki/proofs/*/proof.json` (reuse `_proofs_dir` + `_safe_json_read`).
|
|
77
|
+
- `project_total_usd` = sum of per-run proof costs (the persistent history).
|
|
78
|
+
- `budget.status`: "ok" (<80%), "warn" (>=80% and <100%), "exceeded" (>=100%).
|
|
79
|
+
Computed at read time. No budget.json schema change (avoids the
|
|
80
|
+
byte-identical-JSON parity trap with run.sh heredoc / budget.ts).
|
|
81
|
+
- `cost_recorded` distinguishes "recorded but $0" (records exist, sum 0.0) from
|
|
82
|
+
"not recorded" (no records) -- mirrors efficiency_cost.py honesty contract.
|
|
83
|
+
|
|
84
|
+
`/api/cost` and `/api/budget` are left UNCHANGED (existing frontend + tests
|
|
85
|
+
depend on them). The new endpoint is additive.
|
|
86
|
+
|
|
87
|
+
### 2. Dashboard panel: `dashboard/static/cost.html`
|
|
88
|
+
|
|
89
|
+
Self-contained, zero-build, all CSS+JS inlined (mirrors `proofs.html`). Fetches
|
|
90
|
+
`/api/cost/timeline`. Shows: project total, budget gauge with a colored
|
|
91
|
+
warn/exceeded state and an explicit "warns at 80%, hard-stops at 100%" caption,
|
|
92
|
+
per-run history table, model-routing breakdown, and a simple inline-SVG
|
|
93
|
+
cumulative-cost line for the current run. Linked from "/".
|
|
94
|
+
|
|
95
|
+
### 3. CLI cost view: `loki cost`
|
|
96
|
+
|
|
97
|
+
New `cmd_cost()` in `autonomy/loki` (no existing `cost` command -> free to add).
|
|
98
|
+
Wired into dispatch + help. Reads the same two sources via a single embedded
|
|
99
|
+
python3 block that imports `autonomy/lib/efficiency_cost.collect_efficiency`
|
|
100
|
+
for the current-run aggregate (REUSE, not a 5th copy), and reads proofs/ for
|
|
101
|
+
per-run history. Flags: `--json`, `--last N` (limit run history). Shows budget
|
|
102
|
+
status with the 80% warn line. Honest: prints "cost not recorded for this run"
|
|
103
|
+
when efficiency returns usd=None.
|
|
104
|
+
|
|
105
|
+
`loki cost` is chosen over `loki metrics --cost` because cost is the headline
|
|
106
|
+
R3 wedge and deserves a first-class verb; `loki metrics` stays a productivity
|
|
107
|
+
report. Bun parity for `loki cost` is OUT of scope for this slice (documented
|
|
108
|
+
gap; the bash route is canonical and the budget runtime warn below has Bun
|
|
109
|
+
parity which is the load-bearing part).
|
|
110
|
+
|
|
111
|
+
### 4. Budget warn-at-80% (runtime, both routes)
|
|
112
|
+
|
|
113
|
+
Add a non-pausing warn when crossing 80%, keep the 100% pause:
|
|
114
|
+
- `autonomy/run.sh` `check_budget_limit()`: when `0.80*limit <= cost < limit`,
|
|
115
|
+
`log_warn` + `emit_event_json budget_warning`. Does NOT pause.
|
|
116
|
+
- `loki-ts/src/runner/budget.ts` `checkBudgetLimit()`: same warn semantics via
|
|
117
|
+
the returned result (add `warn: boolean` to `CheckBudgetResult`); orchestrator
|
|
118
|
+
logs it. No budget.json schema change.
|
|
119
|
+
|
|
120
|
+
## Tests
|
|
121
|
+
|
|
122
|
+
- `tests/dashboard/test_cost_timeline_endpoint.py` (pytest, `_ForceLokiDir`
|
|
123
|
+
pattern): empty dirs -> 200 with honest nulls; current-run aggregation +
|
|
124
|
+
cumulative; per-run history from proofs; budget status thresholds
|
|
125
|
+
(ok/warn/exceeded) at 79/80/100%; recorded-but-zero vs not-recorded; corrupt
|
|
126
|
+
JSON skipped; no-PII (no absolute paths leaked).
|
|
127
|
+
- `loki-ts/tests/runner/budget.test.ts` (extended, bun test): warn flag true in
|
|
128
|
+
[80%,100%), false below 80% and at/above 100% (exceeded path), no pause file
|
|
129
|
+
written on warn.
|
|
130
|
+
|
|
131
|
+
## No-PII / honesty constraints
|
|
132
|
+
|
|
133
|
+
- Endpoints return only aggregates + run_ids + model names + timestamps. No file
|
|
134
|
+
paths, no prompt text, no token strings. proof.json is already redacted by the
|
|
135
|
+
R1 generator before it lands.
|
|
136
|
+
- `$0.00` is never fabricated: uncollected cost surfaces as null / "not recorded".
|
|
137
|
+
|
|
138
|
+
## Files (for the integrator to cherry-pick)
|
|
139
|
+
|
|
140
|
+
- `dashboard/server.py` (add `/api/cost/timeline`)
|
|
141
|
+
- `dashboard/static/cost.html` (new)
|
|
142
|
+
- `autonomy/loki` (add `cmd_cost` + dispatch + help)
|
|
143
|
+
- `autonomy/run.sh` (warn-at-80% in `check_budget_limit`)
|
|
144
|
+
- `loki-ts/src/runner/budget.ts` (warn flag)
|
|
145
|
+
- `tests/dashboard/test_cost_timeline_endpoint.py` (new)
|
|
146
|
+
- `loki-ts/tests/runner/budget.test.ts` (extended: warn-at-80% describe block)
|
|
147
|
+
- `docs/R3-COST-OBSERVABILITY-DESIGN.md` (this file)
|