ai-agentmanager 1.9.0 → 1.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.10.0 (2026-10-31)
4
+
5
+ - **Review scorecard:** an agent's changes scored 0-100 for Intent, Spec (each acceptance criterion met / not met / unverified, with evidence), Tests, Safety, Scope and Plan, with an overall readiness, a recommendation (Ready to approve / Look at these first / Not ready) and **Where to look**: the few hunks a person should read. Facts the app measures cap the model's scores (failing CI, failing checks, code without tests, plan steps ticked, unassessed criteria) and the recommendation follows fixed rules. At the top of Review changes, as a **Scorecard** document, as a badge in Land, in the PR description, and posted on the PR with **Post to PR** (posting again updates the comment). Runs by itself when the agent marks its work done (`"scorecard": { "auto": false }` turns that off); `agentmgr doc <agent> scorecard`.
6
+
3
7
  ## 1.9.0 (2026-10-31)
4
8
 
5
9
  - **Intent, spec and plan:** each task agent now has INTENT.md (the request as given, its Jira summary, attachments and who started it; written by AI Agent Manager), SPEC.md (requirements, acceptance criteria, out of scope; the agent writes it before changing code) and PLAN.md (steps it ticks off), next to its brief. **Intent · Spec · Plan** buttons and the split view's Docs menu show them; Explain, the documents and AI review read the spec and plan; **Push & open PR** puts them in the pull request's description (intent in full, spec and plan folded). `"specs": false` in config.json turns the spec and plan instructions off.
package/README.md CHANGED
@@ -85,13 +85,13 @@ Runs on macOS and Linux; on Windows, use WSL2. Requires Node 20+, `git`, and the
85
85
 
86
86
  ```sh
87
87
  npm install -g ai-agentmanager # the latest release
88
- npm install -g ai-agentmanager@1.9.0 # or a specific one
88
+ npm install -g ai-agentmanager@1.10.0 # or a specific one
89
89
  ```
90
90
 
91
91
  Or from the repository, at a release tag:
92
92
 
93
93
  ```sh
94
- git clone --branch v1.9.0 --depth 1 https://github.com/jdmunta/ai-agentmanager-core.git
94
+ git clone --branch v1.10.0 --depth 1 https://github.com/jdmunta/ai-agentmanager-core.git
95
95
  cd ai-agentmanager-core
96
96
  npm install
97
97
  npm link # optional: puts `agentmgr` on your PATH
@@ -111,7 +111,7 @@ npm link
111
111
  | Installed with | Update |
112
112
  | --- | --- |
113
113
  | npm | `npm install -g ai-agentmanager@latest` |
114
- | A clone at a release tag | `git fetch --tags && git checkout v1.9.0 && npm install` (the tag you want) |
114
+ | A clone at a release tag | `git fetch --tags && git checkout v1.10.0 && npm install` (the tag you want) |
115
115
  | A clone of `main` | `git pull && npm install` |
116
116
 
117
117
  **Check your setup.** `agentmgr doctor` checks Node.js, Claude Code and its sign-in, git, `gh`, the data folder, the daemon's port and Chrome, and says how to fix anything missing.
@@ -387,6 +387,7 @@ Each card has at most **three points of about 12 words each**, so they're easy t
387
387
 
388
388
  - **Files and diff.** The changed files on the left with lines added and removed, and the diff file by file on the right. Click a file header to fold it; very large files start folded.
389
389
  - **Line comments.** Click **+** next to a line (or its line number) to comment on it, and add an overall comment if you like. **Send to agent** types them into the agent as one message, each with its file, line and the code it is about, and asks it to address them. Comments not sent yet are kept in this browser.
390
+ - **Review scorecard.** **📊 Score changes** (it also runs by itself when the agent marks its work done) scores the changes 0-100 for **Intent**, **Spec** (each acceptance criterion met, not met or unverified, with evidence), **Tests**, **Safety**, **Scope** and **Plan**, and gives an overall readiness with a recommendation: *Ready to approve*, *Look at these first* or *Not ready*, plus **Where to look**, the few hunks worth a person's time. What the app measures caps the model: failing CI or checks, code changed without tests, plan steps not ticked, criteria not assessed. **Post to PR** puts it on the pull request as a comment (again: updates it), and Push & open PR includes it in the description. It also shows as a badge in Land. Turn the automatic run off with `"scorecard": { "auto": false }`.
390
391
  - **Explain changes** explains each change one by one: a title, what it changes, why (from the request and the conversation), the risk and what to check, in the order a reviewer should read them. Each explanation sits above its hunk, and **◀ / ▶** (or **[** and **]**) step through them, opening folded files on the way. When the agent changes the code afterwards, the page says so and **Explain again** brings it up to date. From the CLI: `agentmgr doc <agent> walkthrough`.
391
392
  - **Commit all** commits everything in the worktree, new files included, with your git name and email.
392
393
  - **Intent, spec and plan.** Each agent keeps INTENT.md (what was asked, written by AI Agent Manager), SPEC.md (requirements and acceptance criteria, written by the agent before it changes code) and PLAN.md (steps, ticked off as it goes) next to its brief. Open them with **Intent · Spec · Plan** or in the split view's Docs; Explain, the documents and AI review read them too. `"specs": false` in config.json stops asking agents for the spec and plan.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ai-agentmanager",
3
- "version": "1.9.0",
3
+ "version": "1.10.0",
4
4
  "description": "Run background coding agents (Claude Code, Codex) per intent, and keep track of them from a web UI, your phone or the CLI",
5
5
  "keywords": [
6
6
  "claude",
package/public/app.js CHANGED
@@ -7,6 +7,7 @@ import { openStream, post, REMOTE } from '/transport.js';
7
7
  import { attachBox } from '/attach.js';
8
8
  import { initSide } from '/side.js';
9
9
  import { addTools, download, htmlFile, initExport } from '/export.js';
10
+ import { badgeHtml as scoreBadge, scorecardHtml, scorecardSpeech } from '/scorecard.js';
10
11
  import { bindListen, listenHtml, play, speakable, now as speaking, stop as stopSpeaking } from '/speak.js';
11
12
  import { initReview } from '/review.js';
12
13
  import { initLand, prChips } from '/land.js';
@@ -123,7 +124,7 @@ $('#new-playbook').onclick = () => {
123
124
  playbooksUi.open({ intent: f.intents.value.trim(), projects: f.project.value.trim() ? [f.project.value.trim()] : [], model: f.model.value.trim(), permissionMode: f.permissionMode.value });
124
125
  };
125
126
  $('#triggers-open').onclick = () => triggersUi.open();
126
- const landUi = initLand({ api, esc, toast, agents, $, select: (id) => (showTab('agents'), select(id)), openReview: (id) => reviewUi.open(id), isOpen: () => sideTab === 'land' });
127
+ const landUi = initLand({ api, esc, toast, agents, $, scoreBadge, select: (id) => (showTab('agents'), select(id)), openReview: (id) => reviewUi.open(id), isOpen: () => sideTab === 'land' });
127
128
 
128
129
  function statusOf(a) {
129
130
  if (a.runtime.planning) return a.runtime.attention ? 'attention' : 'planning';
@@ -267,9 +268,9 @@ const INSIGHT_ICONS = { why: '🎯', what: '📦', how: '🛠', now: '📍' };
267
268
 
268
269
  // ------------------------------------------------------------------ stakeholder documents
269
270
 
270
- const DOCS = { analysis: 'Analysis', design: 'Design', flow: 'Flow', proposal: 'Proposal', humanai: 'Human vs AI', diagrams: 'Diagrams' };
271
+ const DOCS = { analysis: 'Analysis', design: 'Design', flow: 'Flow', proposal: 'Proposal', humanai: 'Human vs AI', diagrams: 'Diagrams', scorecard: 'Scorecard' };
271
272
  // Optional extras that "Write all" leaves out.
272
- const DOC_EXTRAS = new Set(['diagrams']);
273
+ const DOC_EXTRAS = new Set(['diagrams', 'scorecard']);
273
274
  const DOC_TITLES = { ...DOCS, plan: 'Plan', intent: 'Intent', spec: 'Spec' };
274
275
  // The agent's intent, spec and plan files (read from the daemon when opened; kept here by agent and kind).
275
276
  const SPEC_KINDS = { intent: 'Intent', spec: 'Spec', plan: 'Plan' };
@@ -350,7 +351,7 @@ function docButtons(a) {
350
351
  const d = a.docs?.[k];
351
352
  const stale = d?.markdown && a.brief?.updatedAt && d.briefUpdatedAt !== a.brief.updatedAt;
352
353
  return `<button class="small doc-btn ${d?.markdown ? 'has' : ''}" data-doc="${k}" ${d?.pending ? 'disabled' : ''}
353
- title="${k === 'diagrams' ? (d?.markdown ? `Open the diagrams${stale ? ' (the brief changed since)' : ''}` : 'Draw the problem and its solution for a quick grasp: architecture, design, data flow (animated) and the key interaction, diagrams only. Built from the documents already written and the agent\'s work') : d?.markdown ? `Open the ${label.toLowerCase()} document${stale ? ' (the brief changed since)' : ''}` : `Write a ${label.toLowerCase()} document for stakeholders`}">${d?.pending ? `${label}…` : label}${d?.markdown ? (stale ? ' •' : ' ✓') : ''}</button>`;
354
+ title="${k === 'scorecard' ? (d?.markdown ? `Open the review scorecard: ${d.data?.overall ?? '—'}/100` : 'Score the changes for review: intent, spec, scope, plan, tests and safety, with an overall readiness and where to look') : k === 'diagrams' ? (d?.markdown ? `Open the diagrams${stale ? ' (the brief changed since)' : ''}` : 'Draw the problem and its solution for a quick grasp: architecture, design, data flow (animated) and the key interaction, diagrams only. Built from the documents already written and the agent\'s work') : d?.markdown ? `Open the ${label.toLowerCase()} document${stale ? ' (the brief changed since)' : ''}` : `Write a ${label.toLowerCase()} document for stakeholders`}">${d?.pending ? `${label}…` : label}${d?.markdown ? (stale ? ' •' : ' ✓') : ''}</button>`;
354
355
  })
355
356
  .join('')}</span>${specButtons(a)}`;
356
357
  }
@@ -388,7 +389,7 @@ function renderDocDialog() {
388
389
  else if (specKind && !d?.markdown) body.innerHTML = `<p class="hint">${docView.kind === 'intent' ? 'No intent file for this agent (agents created before 1.9 have none).' : `The agent hasn't written its ${label.toLowerCase()} yet. It writes ${docView.kind === 'spec' ? 'the spec before changing code' : 'the plan and keeps it current'}.`}</p>`;
389
390
  else if (d?.markdown) {
390
391
  if (body.dataset.rendered !== d.at) {
391
- body.innerHTML = d.data ? humanAiHtml(d) : docHtml(d.markdown);
392
+ body.innerHTML = docView.kind === 'scorecard' ? scorecardHtml(d, esc) : d.data ? humanAiHtml(d) : docHtml(d.markdown);
392
393
  body.dataset.rendered = d.at;
393
394
  renderDiagrams(body);
394
395
  }
@@ -399,7 +400,7 @@ function renderDocDialog() {
399
400
  for (const id of ['doc-copy', 'doc-download', 'doc-print']) $(`#${id}`).disabled = !d?.markdown;
400
401
  // Diagrams: the whole document as one HTML file with every diagram (each also has Copy / PNG / SVG).
401
402
  $('#doc-html').hidden = !/```mermaid/.test(d?.markdown ?? '');
402
- $('#doc-listen').innerHTML = d?.markdown && docView.kind !== 'diagrams' ? listenHtml(`doc:${docView.agentId}:${docView.kind}`, { title: 'Read the document aloud' }) : '';
403
+ $('#doc-listen').innerHTML = d?.markdown && docView.kind !== 'diagrams' ? listenHtml(docView.kind === 'scorecard' ? `score:${docView.agentId}` : `doc:${docView.agentId}:${docView.kind}`, { title: 'Read the document aloud' }) : '';
403
404
  $('#doc-print').hidden = plan || specKind;
404
405
  $('#doc-regen').hidden = plan || specKind;
405
406
  $('#doc-regen').disabled = Boolean(d?.pending);
@@ -538,6 +539,7 @@ bindListen((id) => {
538
539
  if (what === 'explain') parts = [speakable(a?.explanation?.markdown)];
539
540
  else if (what === 'doc') parts = [speakable(kind === 'explain' ? a?.explanation?.markdown : docOf(a, kind)?.markdown)];
540
541
  else if (what === 'walk') parts = reviewUi.listenParts();
542
+ else if (what === 'score') parts = a?.docs?.scorecard?.data ? scorecardSpeech(a.docs.scorecard) : null;
541
543
  else if (what === 'sidewalk') parts = sidePane?.listenParts();
542
544
  if (!parts?.length || !play(id, parts)) toast('Nothing to read aloud here (or this browser has no speech)');
543
545
  });
package/public/land.js CHANGED
@@ -25,7 +25,7 @@ export function prChips(pr, esc) {
25
25
  return out.join(' ');
26
26
  }
27
27
 
28
- export function initLand({ api, esc, toast, agents, $, select, openReview, isOpen }) {
28
+ export function initLand({ api, esc, toast, agents, $, scoreBadge = () => '', select, openReview, isOpen }) {
29
29
  const view = $('#land-view');
30
30
  let settings = null;
31
31
 
@@ -46,7 +46,7 @@ export function initLand({ api, esc, toast, agents, $, select, openReview, isOpe
46
46
  <div class="line1"><span class="prio p${a.priority ?? 2}">P${a.priority ?? 2}</span>
47
47
  <a class="name" href="${esc(pr.url)}" target="_blank" rel="noopener" title="${esc(pr.title ?? pr.url)}">#${esc(pr.number ?? '')} ${esc(pr.title ?? pr.url)}</a></div>
48
48
  <div class="line2"><button class="link" data-land="goto">${esc(a.name)}</button>${a.runtime?.busy ? ' <span class="hint">working</span>' : ''}</div>
49
- <div class="land-chips">${prChips(pr, esc)}</div>
49
+ <div class="land-chips">${scoreBadge(a.docs?.scorecard, esc)} ${prChips(pr, esc)}</div>
50
50
  <div class="bl-actions">
51
51
  <button class="small" data-land="review" title="Its diff, line comments for the agent, commit and push">Review</button>
52
52
  ${land === 'behind' || pr.mergeable === 'CONFLICTING' ? '<button class="small" data-land="update" title="Merge the base branch in here (when the agent is idle and its work is committed) and push; otherwise, or on conflicts, ask the agent">Update</button>' : ''}
package/public/review.js CHANGED
@@ -3,6 +3,7 @@
3
3
  // With a walkthrough, each hunk shows its explanation and ◀ / ▶ step through them in reading order.
4
4
  import { cardHtml, explainButton, listenParts as walkParts, stepOf, walkthroughOf } from './walkthrough.js';
5
5
  import { listenHtml, now as speaking, stop as stopSpeaking } from './speak.js';
6
+ import { scorecardHtml, scorecardOf } from './scorecard.js';
6
7
 
7
8
  /** Unified diff -> [{ path, status, binary, hunks: [{ head, lines: [{ t, old, new, text }] }] }] */
8
9
  export function parseDiff(text) {
@@ -138,7 +139,7 @@ export function initReview({ api, esc, toast, agents, $ }) {
138
139
  for (const h of f.hunks) {
139
140
  const i = w.doc ? stepOf(w.doc, f.path, h.head) : -1;
140
141
  if (i >= 0) rows.push(`<tr class="rv-explain ${st.step === i ? 'current' : ''}" id="rv-step-${i}"><td colspan="3">${cardHtml(w.doc.data.changes[i], i, w.doc.data.changes.length, esc)}</td></tr>`);
141
- rows.push(`<tr class="rv-hunk"><td colspan="3">${esc(h.head)}</td></tr>`);
142
+ rows.push(`<tr class="rv-hunk" data-rv-hunk="${esc(`${f.path}\t${h.head}`)}"><td colspan="3">${esc(h.head)}</td></tr>`);
142
143
  for (const l of h.lines) {
143
144
  const side = l.t === 'del' ? 'old' : 'new';
144
145
  const line = side === 'old' ? l.old : l.new;
@@ -157,6 +158,32 @@ export function initReview({ api, esc, toast, agents, $ }) {
157
158
  return `<div class="rv-file" id="rv-${encodeURIComponent(f.path)}">${head}<table class="rv-table">${rows.join('')}</table></div>`;
158
159
  }
159
160
 
161
+ /** The review scorecard at the top: scores, verdict and where to look, or the button to score. */
162
+ function scorePanel() {
163
+ if (!st.data?.files.length) return '';
164
+ const sc = scorecardOf(agents.get(st.id), st.data.diffHash);
165
+ const openPr = st.data.prs.find((p) => p.state === 'OPEN');
166
+ const run = `<button type="button" class="small ${sc.doc && !sc.stale ? 'ghost' : 'primary'}" data-sc-run ${sc.pending ? 'disabled' : ''} title="Score the changes against the intent, spec, plan and tests, with where to look (a separate one-off call)">${sc.pending ? 'Scoring…' : sc.doc ? (sc.stale ? 'Score again (changed)' : 'Score again') : '📊 Score changes'}</button>`;
167
+ if (!sc.doc) {
168
+ return `<div class="rv-score-empty">${run}<span class="hint">${sc.error && !sc.pending ? `<span class="error">Couldn't score: ${esc(sc.error)}</span>` : 'Scores against the intent, spec, scope, plan, tests and safety, with an overall readiness and the few places to read.'}</span></div>`;
169
+ }
170
+ const post = openPr ? `<button type="button" class="small ghost" data-sc-post title="Post the scorecard on the pull request as a comment (posting again updates it)">${sc.doc.postedTo === openPr.url ? 'Update on PR' : 'Post to PR'}</button>` : '';
171
+ return `<div class="rv-score">${scorecardHtml(sc.doc, esc, { stale: sc.stale, actions: `${listenHtml(`score:${st.id}`, { title: 'Hear the scorecard' })}${post}${run}` })}</div>`;
172
+ }
173
+
174
+ /** Open the file of a hunk (from "where to look") and scroll to it. */
175
+ function gotoHunk(key) {
176
+ const [file] = key.split('\t');
177
+ st.collapsed.delete(file);
178
+ keepInputs();
179
+ render();
180
+ const row = [...body.querySelectorAll('[data-rv-hunk]')].find((r) => r.dataset.rvHunk === key);
181
+ if (!row) return toast('That part of the change has moved since it was scored');
182
+ row.scrollIntoView({ block: 'center', behavior: 'smooth' });
183
+ row.classList.add('flash');
184
+ setTimeout(() => row.classList.remove('flash'), 1600);
185
+ }
186
+
160
187
  /** Explain changes, then ◀ n of N ▶ once there is a walkthrough. */
161
188
  function walkNav() {
162
189
  const w = walk();
@@ -221,6 +248,7 @@ export function initReview({ api, esc, toast, agents, $ }) {
221
248
  ${d.files.length ? walkNav() : ''}
222
249
  <button type="button" class="small ghost" data-rv-reload>${st.loading ? 'Loading…' : 'Reload'}</button>
223
250
  </div>
251
+ ${scorePanel()}
224
252
  ${walkNote()}
225
253
  ${d.busy ? '<p class="hint rv-warn">The agent is working, so the changes may still move. Reload when it is idle.</p>' : ''}
226
254
  ${d.truncated ? '<p class="hint rv-warn">The diff is large and was cut short; open a shell for the rest.</p>' : ''}
@@ -289,6 +317,29 @@ export function initReview({ api, esc, toast, agents, $ }) {
289
317
  if (t.closest('[data-rv-reload]')) return keepInputs(), reload();
290
318
  const stepBtn = t.closest('[data-rv-step]');
291
319
  if (stepBtn) return step(Number(stepBtn.dataset.rvStep));
320
+ const look = t.closest('[data-sc-look]');
321
+ if (look) return gotoHunk(look.dataset.scLook);
322
+ if (t.closest('[data-sc-run]')) {
323
+ keepInputs();
324
+ try {
325
+ agents.set(st.id, await api('POST', `/agents/${st.id}/docs/scorecard?background=1`));
326
+ render();
327
+ } catch (err) {
328
+ toast(`Error: ${err.message}`, 5000);
329
+ }
330
+ return;
331
+ }
332
+ if (t.closest('[data-sc-post]')) {
333
+ t.disabled = true;
334
+ try {
335
+ const r = await api('POST', `/agents/${st.id}/scorecard/post`);
336
+ toast(r.edited ? 'Scorecard updated on the pull request' : 'Scorecard posted on the pull request', 4000, { label: 'Open', run: () => window.open(r.url, '_blank', 'noopener') });
337
+ } catch (err) {
338
+ toast(`Error: ${err.message}`, 6000);
339
+ }
340
+ t.disabled = false;
341
+ return;
342
+ }
292
343
  if (t.closest('[data-rv-explain]')) {
293
344
  keepInputs();
294
345
  try {
@@ -442,7 +493,8 @@ export function initReview({ api, esc, toast, agents, $ }) {
442
493
  function agentChanged(a) {
443
494
  if (!dlg.open || a.id !== st.id || !st.data) return;
444
495
  const w = a.docs?.walkthrough;
445
- const sig = `${w?.at}|${w?.pending}|${w?.error}`;
496
+ const sc = a.docs?.scorecard;
497
+ const sig = `${w?.at}|${w?.pending}|${w?.error}|${sc?.at}|${sc?.pending}|${sc?.error}|${sc?.postedTo}`;
446
498
  if (sig === st.walkSig) return;
447
499
  st.walkSig = sig;
448
500
  keepInputs();
@@ -0,0 +1,68 @@
1
+ // The review scorecard on the page: an overall readiness, a bar per score with why, the acceptance
2
+ // criteria, and where to look. Shared by Review changes, the Scorecard document and Land's badges.
3
+
4
+ const DIMENSIONS = [
5
+ ['intent', 'Intent'],
6
+ ['spec', 'Spec'],
7
+ ['tests', 'Tests'],
8
+ ['safety', 'Safety'],
9
+ ['scope', 'Scope'],
10
+ ['plan', 'Plan'],
11
+ ];
12
+ export const RECOMMENDATION = { ready: 'Ready to approve', look: 'Look at these first', not_ready: 'Not ready' };
13
+ const tone = (n) => (n == null ? '' : n >= 80 ? 'good' : n >= 50 ? 'mid' : 'bad');
14
+ const recTone = { ready: 'good', look: 'mid', not_ready: 'bad' };
15
+
16
+ /** The scorecard of an agent and whether it still matches the changes (diffHash from /changes). */
17
+ export function scorecardOf(agent, diffHash) {
18
+ const d = agent?.docs?.scorecard;
19
+ return { doc: d?.data ? d : null, pending: Boolean(d?.pending), error: d?.error ?? null, stale: Boolean(d?.data && diffHash && d.diffHash !== diffHash) };
20
+ }
21
+
22
+ /** A small badge: "82 · Ready to approve". */
23
+ export function badgeHtml(doc, esc) {
24
+ if (!doc?.data) return '';
25
+ const sc = doc.data;
26
+ return `<span class="sc-badge ${recTone[sc.recommendation]}" title="Review scorecard: ${esc(RECOMMENDATION[sc.recommendation])}${sc.reasons.length ? ` (${esc(sc.reasons.join('; '))})` : ''}">${sc.overall ?? '—'} · ${esc(RECOMMENDATION[sc.recommendation])}</span>`;
27
+ }
28
+
29
+ /**
30
+ * The full card. `actions` is extra HTML for the head (buttons); look items carry
31
+ * data-sc-look="<file>\t<head>" for the page to jump to.
32
+ */
33
+ export function scorecardHtml(doc, esc, { stale = false, actions = '' } = {}) {
34
+ const sc = doc.data;
35
+ const rows = DIMENSIONS.map(([k, label]) => {
36
+ const d = sc.scores[k];
37
+ if (!d) return `<div class="sc-row na"><span class="sc-label">${label}</span><span class="sc-bar"></span><span class="sc-n">—</span><span class="sc-why hint">${k === 'spec' ? 'No spec written' : 'Not scored'}</span></div>`;
38
+ return `<div class="sc-row"><span class="sc-label">${label}</span><span class="sc-bar"><span class="${tone(d.score)}" style="width:${d.score}%"></span></span><span class="sc-n">${d.score}</span><span class="sc-why">${esc(d.why)}${d.notes.length ? ` <span class="sc-note">${esc(d.notes.join('; '))}</span>` : ''}</span></div>`;
39
+ }).join('');
40
+ const crit = sc.criteria.length
41
+ ? `<details class="sc-crit" ${sc.criteria.some((c) => c.status !== 'met') ? 'open' : ''}><summary>Acceptance criteria: ${sc.criteria.filter((c) => c.status === 'met').length} of ${sc.criteria.length} met</summary><ul>${sc.criteria.map((c) => `<li class="${c.status}"><span class="sc-mark">${c.status === 'met' ? '✓' : c.status === 'not_met' ? '✗' : '?'}</span>${esc(c.criterion)}${c.evidence ? ` <span class="hint">${esc(c.evidence)}</span>` : ''}</li>`).join('')}</ul></details>`
42
+ : '';
43
+ const look = sc.look.length
44
+ ? `<div class="sc-look"><b>Where to look</b><ol>${sc.look.map((l) => `<li><button type="button" class="link" data-sc-look="${esc(`${l.file}\t${l.head}`)}"><code>${esc(l.file)}</code></button> ${esc(l.reason)}</li>`).join('')}</ol></div>`
45
+ : '';
46
+ const f = sc.facts;
47
+ return `<div class="scorecard">
48
+ <div class="sc-head">
49
+ <span class="sc-overall ${recTone[sc.recommendation]}"><b>${sc.overall ?? '—'}</b><small>/100</small></span>
50
+ <div class="sc-verdict"><b>${esc(RECOMMENDATION[sc.recommendation])}</b>${sc.reasons.length ? `<div class="hint">${esc(sc.reasons.join(' · '))}</div>` : ''}<div class="hint">${esc(sc.summary)}</div></div>
51
+ <span class="spacer"></span>${actions}
52
+ </div>
53
+ ${stale ? '<p class="hint sc-stale"><b>The changes moved since this was scored.</b> Score again for the latest.</p>' : ''}
54
+ <div class="sc-rows">${rows}</div>
55
+ ${crit}${look}
56
+ <div class="hint sc-facts">Measured: ${f.files} files (+${f.added} −${f.removed}) · ${f.testFiles} test files · CI ${esc(f.ci ?? 'n/a')} · checks ${f.checks.passed}✓ ${f.checks.failed}✗${f.plan ? ` · plan ${f.plan.done}/${f.plan.steps}` : ''}${doc.writer ? ` · scored by ${esc(doc.writer)}` : ''}</div>
57
+ </div>`;
58
+ }
59
+
60
+ /** What Listen reads: the verdict, then each score with why. */
61
+ export function scorecardSpeech(doc) {
62
+ const sc = doc.data;
63
+ return [
64
+ `${RECOMMENDATION[sc.recommendation]}. Overall ${sc.overall ?? 'not scored'} out of 100. ${sc.reasons.join('. ')} ${sc.summary}`,
65
+ ...DIMENSIONS.filter(([k]) => sc.scores[k]).map(([k, label]) => `${label}: ${sc.scores[k].score}. ${sc.scores[k].why} ${sc.scores[k].notes.join('. ')}`),
66
+ ...(sc.look.length ? [`Where to look: ${sc.look.map((l) => `${l.file.split('/').pop()}: ${l.reason}`).join('. ')}`] : []),
67
+ ];
68
+ }
package/public/style.css CHANGED
@@ -906,3 +906,31 @@ form.dropping { outline: 2px dashed var(--accent, #6b8cff); outline-offset: 4px;
906
906
  .say.on button:first-child { min-width: 30px; }
907
907
  .explain-foot { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
908
908
  .sd-step.current { box-shadow: 0 0 0 2px var(--accent, #6b8cff); }
909
+
910
+ /* Review scorecard. */
911
+ .scorecard { display: flex; flex-direction: column; gap: 8px; padding: 10px 12px; border: 1px solid var(--border); border-radius: 8px; background: var(--panel); }
912
+ .sc-head { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; }
913
+ .sc-overall { display: flex; align-items: baseline; gap: 2px; padding: 4px 10px; border-radius: 8px; border: 2px solid currentColor; }
914
+ .sc-overall b { font-size: 26px; line-height: 1; }
915
+ .sc-overall small { opacity: .7; }
916
+ .sc-verdict { min-width: 0; flex: 1 1 260px; }
917
+ .good { color: var(--green); } .mid { color: var(--yellow); } .bad { color: var(--red); }
918
+ .sc-rows { display: flex; flex-direction: column; gap: 4px; }
919
+ .sc-row { display: grid; grid-template-columns: 64px 140px 32px 1fr; gap: 8px; align-items: center; font-size: 13px; }
920
+ .sc-label { font-weight: 600; }
921
+ .sc-bar { height: 8px; border-radius: 4px; background: var(--panel-2); overflow: hidden; }
922
+ .sc-bar > span { display: block; height: 100%; background: currentColor; }
923
+ .sc-n { text-align: right; font-variant-numeric: tabular-nums; }
924
+ .sc-note { color: var(--yellow); }
925
+ .sc-row.na { opacity: .6; }
926
+ .sc-crit ul, .sc-look ol { margin: 4px 0 0; padding-left: 20px; }
927
+ .sc-crit li { list-style: none; margin-left: -16px; }
928
+ .sc-mark { display: inline-block; width: 16px; font-weight: 700; }
929
+ .sc-crit li.met .sc-mark { color: var(--green); } .sc-crit li.not_met .sc-mark { color: var(--red); } .sc-crit li.unverified .sc-mark { color: var(--yellow); }
930
+ .sc-facts { font-size: 11.5px; }
931
+ .sc-badge { display: inline-block; padding: 1px 8px; border-radius: 999px; border: 1px solid currentColor; font-size: 12px; white-space: nowrap; }
932
+ .rv-score { margin: 0 12px 8px; }
933
+ .rv-score-empty { display: flex; align-items: center; gap: 10px; margin: 0 12px 8px; }
934
+ .rv-hunk.flash td { animation: rvflash 1.6s; }
935
+ @keyframes rvflash { 0%, 60% { background: var(--accent); color: var(--accent-text); } }
936
+ @media (max-width: 640px) { .sc-row { grid-template-columns: 56px 1fr 28px; } .sc-why { grid-column: 1 / -1; } }
package/src/cli.js CHANGED
@@ -430,9 +430,9 @@ export async function main(argv) {
430
430
 
431
431
  program
432
432
  .command('doc')
433
- .description('Write a stakeholder document from the agent work: analysis, design, flow, proposal, humanai (Human vs AI), diagrams (diagrams only) or walkthrough (its changes explained one by one)')
433
+ .description('Write a stakeholder document from the agent work: analysis, design, flow, proposal, humanai (Human vs AI), diagrams (diagrams only), walkthrough (its changes explained one by one) or scorecard (its changes scored for review)')
434
434
  .argument('<agent>')
435
- .argument('<kind>', 'analysis, design, flow, proposal, humanai, diagrams, walkthrough, or all (background; diagrams and walkthrough not included)')
435
+ .argument('<kind>', 'analysis, design, flow, proposal, humanai, diagrams, walkthrough, scorecard, or all (background; diagrams, walkthrough and scorecard not included)')
436
436
  .option('-o, --output <file>', 'also save the Markdown to this file')
437
437
  .option('--cached', 'print the last version instead of writing a new one')
438
438
  .option('-b, --background', 'start writing and return at once (kind "all" writes all five)')
package/src/manager.js CHANGED
@@ -47,6 +47,7 @@ const SUMMARIES = {
47
47
  humanai: { doc: 'humanai', run: 'runHumanAi' },
48
48
  diagrams: { doc: 'diagrams', run: 'runDiagrams' },
49
49
  walkthrough: { doc: 'walkthrough', run: 'runWalkthrough' },
50
+ scorecard: { doc: 'scorecard', run: 'runScorecard' },
50
51
  };
51
52
  /** Read / write where a summary of `kind` is kept on the agent record. */
52
53
  const getSummary = (agent, kind) => (SUMMARIES[kind].doc ? agent.docs?.[kind] : agent[SUMMARIES[kind].field]);
@@ -100,6 +101,7 @@ import { spawn } from 'node:child_process';
100
101
  import * as git from './git.js';
101
102
  import { parseDiff } from '../public/review.js';
102
103
  import { issueSummary, leadingIssue, wordsAfterLink } from './jira.js';
104
+ import { combine, facts as scoreFacts, scorecardMarkdown, scorecardPrompt } from './scorecard.js';
103
105
  import { conflictMessage, findConflicts, related, repoKey, summarize as summarizeConflicts } from './conflicts.js';
104
106
  import { answerMessage, loadQuestions, questionsPath, questionsPrompt } from './inbox.js';
105
107
  import { STUCK_DEFAULTS, detectStuck } from './stuck.js';
@@ -1887,6 +1889,8 @@ export class AgentManager extends EventEmitter {
1887
1889
  this.patch(id, { brief });
1888
1890
  // The agent says it is done: have the judge check the work before the person looks.
1889
1891
  if (brief.status === 'done' && agent.brief?.status !== 'done' && this.judge.enabled) this.requestReview(id, { trigger: 'brief' }).catch(() => {});
1892
+ // ...and score it for the reviewer ("scorecard": { "auto": false } in config.json turns this off).
1893
+ if (brief.status === 'done' && agent.brief?.status !== 'done' && this.config.scorecard?.auto !== false && !NO_REPO.has(agent.kind) && this.changesSource) this.summarize(id, 'scorecard').catch(() => {});
1890
1894
  }
1891
1895
  if (agent.planReview) {
1892
1896
  const file = loadPlanFile(id, agent.planReview);
@@ -2136,6 +2140,42 @@ export class AgentManager extends EventEmitter {
2136
2140
  );
2137
2141
  }
2138
2142
 
2143
+ /**
2144
+ * The review scorecard (see scorecard.js): the same diff the review screen shows, scored against
2145
+ * the intent, spec and plan, with the facts the app measures capping what the model says.
2146
+ */
2147
+ async runScorecard(agent, ctx) {
2148
+ if (!this.changesSource) throw new Error('Changes are not available here');
2149
+ const data = await this.changesSource(agent.id);
2150
+ const { hunks, cut } = walkthroughHunks(parseDiff(data.diff), { maxChars: 50_000 });
2151
+ if (!hunks.length) throw Object.assign(new Error('No changes to score yet'), { status: 400 });
2152
+ const cur0 = this.store.get(agent.id) ?? agent;
2153
+ const doc = (n) => readSpecDoc(agent.id, n)?.markdown ?? '';
2154
+ const f = scoreFacts({ files: data.files, prs: Object.values(cur0.prs ?? {}), brief: cur0.brief, plan: doc('plan'), spec: doc('spec') });
2155
+ const writer = this.writerFor(agent);
2156
+ const prompt = scorecardPrompt({ ...ctx, briefMarkdown: this.briefFile(agent.id).markdown ?? '', intentDoc: doc('intent'), specDoc: doc('spec'), planDoc: doc('plan'), hunks, facts: f });
2157
+ const sc = combine(parseJsonReply(await this.oneShot(agent, prompt, writer)), f, hunks);
2158
+ const cur = this.store.get(agent.id);
2159
+ if (!cur) return null;
2160
+ return this.view(
2161
+ this.patch(
2162
+ agent.id,
2163
+ summaryPatch(cur, 'scorecard', {
2164
+ data: sc,
2165
+ cut,
2166
+ diffHash: hash(data.diff),
2167
+ markdown: scorecardMarkdown(sc, cur.name),
2168
+ writer: `${writer.engine.label}${writer.model ? ` (${writer.model})` : ''}`,
2169
+ postedTo: cur.docs?.scorecard?.postedTo ?? null,
2170
+ at: now(),
2171
+ briefUpdatedAt: cur.brief?.updatedAt ?? null,
2172
+ pending: false,
2173
+ error: null,
2174
+ }),
2175
+ ),
2176
+ );
2177
+ }
2178
+
2139
2179
  /** Diagrams only: the problem and its solution drawn from the documents already written and the live context. */
2140
2180
  async runDiagrams(agent, ctx) {
2141
2181
  const cur0 = this.store.get(agent.id) ?? agent;
package/src/review.js CHANGED
@@ -143,6 +143,32 @@ export class Review {
143
143
  };
144
144
  }
145
145
 
146
+ /**
147
+ * Post the review scorecard on the agent's open pull request as a comment, so approvers see it in
148
+ * GitHub. Posting again edits the last comment instead of adding another.
149
+ */
150
+ async postScorecard(ref) {
151
+ const agent = this.agentWithRepo(ref);
152
+ const sc = agent.docs?.scorecard;
153
+ if (!sc?.markdown) throw err(409, 'Score the changes first');
154
+ const pr = Object.values(agent.prs ?? {}).find((p) => p.state === 'OPEN');
155
+ if (!pr) throw err(409, 'No open pull request for this agent');
156
+ this.manager.plugins.check('agent.pr', { agent: this.manager.auditView(agent), title: 'scorecard comment', draft: false });
157
+ const opts = { cwd: agent.workdir, timeout: 60_000 };
158
+ const args = ['pr', 'comment', pr.url, '--body', sc.markdown];
159
+ const again = sc.postedTo === pr.url;
160
+ try {
161
+ await git.run(ghBin(), again ? [...args, '--edit-last'] : args, opts);
162
+ } catch (e) {
163
+ // The comment we edited may be gone; add a new one.
164
+ if (!again) throw err(502, `Posting the scorecard failed: ${e.message.replace(/^gh .*? failed: /, '')}`);
165
+ await git.run(ghBin(), args, opts);
166
+ }
167
+ const cur = this.manager.store.get(agent.id);
168
+ this.manager.patch(agent.id, { docs: { ...cur.docs, scorecard: { ...cur.docs.scorecard, postedTo: pr.url, postedAt: new Date().toISOString() } } });
169
+ return { url: pr.url, edited: again };
170
+ }
171
+
146
172
  /** Send the comments to the agent as one message (typed in when it is idle). */
147
173
  comment(ref, { comments = [], note = '' } = {}) {
148
174
  const agent = this.agentWithRepo(ref);
@@ -194,8 +220,14 @@ export class Review {
194
220
  }
195
221
  const args = ['pr', 'create', '--head', branch];
196
222
  if (target) args.push('--base', target);
197
- // The description: your text (else the commits), then the intent, spec and plan.
198
- const sections = prSections(agent.id);
223
+ // The description: your text (else the commits), the review scorecard, then the intent, spec and plan.
224
+ const sc = agent.docs?.scorecard;
225
+ let card = '';
226
+ if (sc?.markdown) {
227
+ const now = base ? hash(await git.fullDiff(dir, base.sha)) : null;
228
+ card = sc.diffHash && now && sc.diffHash !== now ? `${sc.markdown}\n\n<sub>Scored before the latest changes.</sub>` : sc.markdown;
229
+ }
230
+ const sections = [card, prSections(agent.id)].filter(Boolean).join('\n\n');
199
231
  if (String(title).trim() || sections) {
200
232
  const log = base ? await git.run('git', ['log', '--reverse', '--format=%s', `${base.sha}..HEAD`], opts).catch(() => '') : '';
201
233
  const subjects = log.split('\n').filter(Boolean);
@@ -0,0 +1,189 @@
1
+ // The review scorecard: an agent's changes scored against what was asked (intent), what done means
2
+ // (spec), staying on task (scope), its plan, the tests and safety, with an overall readiness and the
3
+ // few places a person should actually read. A model judges; facts the app measures (CI, test files,
4
+ // plan ticks, the spec's criteria) cap and correct it, and the recommendation follows fixed rules,
5
+ // so a confident-sounding model can't talk a change into "ready".
6
+
7
+ export const DIMENSIONS = [
8
+ ['intent', 'Intent', 'Does it do what was asked?'],
9
+ ['spec', 'Spec', 'Are the acceptance criteria met?'],
10
+ ['tests', 'Tests', 'Is it verified: CI, tests for the changed code, checks run?'],
11
+ ['safety', 'Safety', 'Risky areas handled: security, data, migrations, dependencies, public APIs (higher is safer)'],
12
+ ['scope', 'Scope', 'Only what the task needs, in proportion?'],
13
+ ['plan', 'Plan', 'Done as planned, departures explained?'],
14
+ ];
15
+ const WEIGHTS = { intent: 25, spec: 20, tests: 20, safety: 15, scope: 10, plan: 10 };
16
+ const TEST_FILE = /(^|\/)(tests?|__tests__|spec|specs)\/|[._-](test|spec)\.[a-z]+$|_test\.go$|Test\.java$/i;
17
+
18
+ /** Facts the app knows without a model: CI, test files touched, the agent's checks, plan and spec progress. */
19
+ export function facts({ files = [], prs = [], brief = null, plan = '', spec = '' }) {
20
+ const paths = files.map((f) => f.path);
21
+ const tests = paths.filter((p) => TEST_FILE.test(p));
22
+ const code = paths.filter((p) => !TEST_FILE.test(p) && !/\.(md|txt|rst|adoc)$/i.test(p) && !/(^|\/)docs?\//i.test(p));
23
+ const open = prs.find((p) => p.state === 'OPEN') ?? prs[0];
24
+ const checks = brief?.checks ?? [];
25
+ const count = (st) => checks.filter((c) => c.state === st).length;
26
+ const steps = [...String(plan).matchAll(/^\s*[-*]\s+\[([ xX~!])\]/gm)].map((m) => m[1].toLowerCase());
27
+ const criteriaBlock = /^##\s*Acceptance criteria\s*$([\s\S]*?)(?=^##\s|(?![\s\S]))/im.exec(String(spec))?.[1] ?? '';
28
+ return {
29
+ files: paths.length,
30
+ added: files.reduce((s, f) => s + (f.added ?? 0), 0),
31
+ removed: files.reduce((s, f) => s + (f.removed ?? 0), 0),
32
+ testFiles: tests.length,
33
+ codeFiles: code.length,
34
+ ci: open?.checks?.state ?? null, // passed | failed | running | null (no PR or no checks)
35
+ ciFailed: open?.checks?.failed ?? [],
36
+ checks: { passed: count('passed'), failed: count('failed'), pending: count('pending') + count('running') },
37
+ plan: steps.length ? { steps: steps.length, done: steps.filter((s) => s === 'x').length } : null,
38
+ criteria: criteriaBlock.split('\n').filter((l) => /^\s*[-*\d]/.test(l) && /\w/.test(l)).length,
39
+ hasSpec: Boolean(String(spec).trim()),
40
+ };
41
+ }
42
+
43
+ const factLines = (f) =>
44
+ [
45
+ `files changed: ${f.files} (+${f.added} -${f.removed}); code files: ${f.codeFiles}; test files: ${f.testFiles}`,
46
+ `CI on the pull request: ${f.ci ?? 'no pull request or no checks yet'}${f.ciFailed.length ? ` (failing: ${f.ciFailed.join(', ')})` : ''}`,
47
+ `the agent's own checks (from its brief): ${f.checks.passed} passed, ${f.checks.failed} failed, ${f.checks.pending} pending`,
48
+ `plan: ${f.plan ? `${f.plan.done} of ${f.plan.steps} steps ticked` : 'no plan written'}`,
49
+ `spec: ${f.hasSpec ? `${f.criteria} acceptance criteria` : 'no spec written'}`,
50
+ ].join('\n');
51
+
52
+ export function scorecardPrompt({ intent, status, digest, briefMarkdown, intentDoc, specDoc, planDoc, hunks, facts: f }) {
53
+ return [
54
+ "You score an AI coding agent's changes for a reviewer who wants to approve with confidence and read as little as possible.",
55
+ 'Be skeptical: a claim without evidence in the diff, the tests or the checks counts as unverified. Base everything on the material below.',
56
+ '',
57
+ 'Reply with one JSON object only:',
58
+ '{"summary": "<two sentences: what the change does and how sure you are>",',
59
+ ' "scores": {"intent": {"score": 0-100, "why": "<one sentence>"}, "scope": {...}, "plan": {...}, "tests": {...}, "safety": {...}},',
60
+ ' "criteria": [{"criterion": "<an acceptance criterion from the spec>", "status": "met|not_met|unverified", "evidence": "<where: file, test, check>"}],',
61
+ ' "look": [{"id": "H3", "reason": "<why a person should read this hunk>"}]}',
62
+ '',
63
+ 'Scoring (100 is best; be calibrated, not generous):',
64
+ '- intent: how fully the change does what INTENT asks. Missing parts of the ask pull it down hard.',
65
+ '- scope: 100 when every change serves the task; lower for unrelated edits, drive-by refactors, or a change far larger than the task needs.',
66
+ '- plan: how closely the change follows PLAN (steps done, departures explained). Without a plan, judge whether the work is coherent.',
67
+ '- tests: whether the changed behavior is verified: tests added or updated for it, CI, checks the agent ran. Code changes with no tests and no passing checks score low.',
68
+ '- safety: 100 when nothing risky is touched or risks are handled; lower for security, auth, data loss, migrations, concurrency, money, new dependencies or public API changes without care.',
69
+ '- criteria: one entry per acceptance criterion in SPEC, in its order; leave the list empty if there is no spec.',
70
+ '- look: at most 5 hunks a person must read (the risky, subtle or unverified ones), most important first. Not the obvious ones.',
71
+ '',
72
+ '<facts_measured_by_the_app>',
73
+ factLines(f),
74
+ '</facts_measured_by_the_app>',
75
+ '<intent>',
76
+ intentDoc || intent || '(none)',
77
+ '</intent>',
78
+ '<spec>',
79
+ specDoc || '(no spec written)',
80
+ '</spec>',
81
+ '<plan>',
82
+ planDoc || '(no plan written)',
83
+ '</plan>',
84
+ '<agent_status>',
85
+ status,
86
+ '</agent_status>',
87
+ '<conversation>',
88
+ (digest || '(none)').slice(-12_000),
89
+ '</conversation>',
90
+ '<brief>',
91
+ briefMarkdown || '(none)',
92
+ '</brief>',
93
+ '<hunks>',
94
+ ...hunks.map((h) => `<hunk id="${h.id}" file="${h.file}">\n${h.head}\n${h.text}\n</hunk>`),
95
+ '</hunks>',
96
+ ].join('\n');
97
+ }
98
+
99
+ const clamp = (n) => Math.max(0, Math.min(100, Math.round(Number(n) || 0)));
100
+
101
+ /**
102
+ * The scorecard from the model's reply and the facts: scores capped by what was measured, the spec
103
+ * score computed from the criteria, the overall weighted, and a recommendation by rule.
104
+ */
105
+ export function combine(reply, f, hunks) {
106
+ const s = reply?.scores ?? {};
107
+ const score = (k) => (s[k] && Number.isFinite(Number(s[k].score)) ? { score: clamp(s[k].score), why: String(s[k].why ?? '').slice(0, 300), notes: [] } : null);
108
+ const out = { intent: score('intent'), scope: score('scope'), plan: score('plan'), tests: score('tests'), safety: score('safety'), spec: null };
109
+ const cap = (k, max, note) => {
110
+ if (out[k] && out[k].score > max) {
111
+ out[k].score = max;
112
+ out[k].notes.push(note);
113
+ }
114
+ };
115
+ // Tests: failing CI or failing checks, or code changed with nothing verifying it.
116
+ if (f.ci === 'failed') cap('tests', 30, `CI is failing${f.ciFailed.length ? ` (${f.ciFailed.join(', ')})` : ''}`);
117
+ if (f.checks.failed) cap('tests', 40, `${f.checks.failed} of the agent's checks failed`);
118
+ if (f.codeFiles && !f.testFiles && !f.checks.passed && f.ci !== 'passed') cap('tests', 45, 'code changed with no tests and no passing checks');
119
+ // Plan: bounded by the steps ticked.
120
+ if (f.plan && out.plan) cap('plan', Math.round(40 + (60 * f.plan.done) / f.plan.steps), `${f.plan.done} of ${f.plan.steps} plan steps ticked`);
121
+ if (!f.plan && out.plan) out.plan.notes.push('no plan written');
122
+ // Spec: computed from the criteria (met 1, unverified ½, not met 0); none without a spec.
123
+ const criteria = (Array.isArray(reply?.criteria) ? reply.criteria : [])
124
+ .filter((c) => c?.criterion)
125
+ .slice(0, 20)
126
+ .map((c) => ({ criterion: String(c.criterion).slice(0, 300), status: ['met', 'not_met', 'unverified'].includes(c.status) ? c.status : 'unverified', evidence: String(c.evidence ?? '').slice(0, 300) }));
127
+ // A spec with criteria the model didn't assess: they count as unverified, not as passed.
128
+ if (f.hasSpec && f.criteria && !criteria.length) out.spec = { score: 50, why: `${f.criteria} acceptance criteri${f.criteria === 1 ? 'on' : 'a'} not assessed`, notes: ['treated as unverified'] };
129
+ if (f.hasSpec && criteria.length) {
130
+ const pts = criteria.reduce((n, c) => n + (c.status === 'met' ? 1 : c.status === 'unverified' ? 0.5 : 0), 0);
131
+ const met = criteria.filter((c) => c.status === 'met').length;
132
+ out.spec = { score: clamp((100 * pts) / criteria.length), why: `${met} of ${criteria.length} acceptance criteri${criteria.length === 1 ? 'on' : 'a'} met${criteria.some((c) => c.status === 'not_met') ? ', some not met' : ''}`, notes: [] };
133
+ }
134
+ // Overall: weighted over the scores there are.
135
+ const have = Object.entries(WEIGHTS).filter(([k]) => out[k]);
136
+ const total = have.reduce((n, [, w]) => n + w, 0);
137
+ const overall = total ? Math.round(have.reduce((n, [k, w]) => n + out[k].score * w, 0) / total) : null;
138
+ const notMet = criteria.filter((c) => c.status === 'not_met');
139
+ const reasons = [];
140
+ if (f.ci === 'failed') reasons.push('CI is failing');
141
+ if (notMet.length) reasons.push(`${notMet.length} acceptance criteri${notMet.length === 1 ? 'on' : 'a'} not met`);
142
+ if (out.intent && out.intent.score < 50) reasons.push('it does not do enough of what was asked');
143
+ if (out.tests && out.tests.score < 40 && f.ci !== 'failed') reasons.push('it is barely verified');
144
+ let recommendation;
145
+ if (reasons.length) recommendation = 'not_ready';
146
+ else if (overall != null && overall >= 80 && (out.tests?.score ?? 0) >= 70 && (out.safety?.score ?? 100) >= 70 && (out.spec?.score ?? 100) >= 70 && f.ci !== 'running') recommendation = 'ready';
147
+ else recommendation = 'look';
148
+ // Where to look, mapped back to hunks.
149
+ const byId = new Map(hunks.map((h) => [h.id, h]));
150
+ const look = (Array.isArray(reply?.look) ? reply.look : [])
151
+ .map((l) => ({ h: byId.get(String(l?.id ?? '').trim()), reason: String(l?.reason ?? '').slice(0, 300) }))
152
+ .filter((l) => l.h)
153
+ .slice(0, 5)
154
+ .map(({ h, reason }) => ({ file: h.file, head: h.head, reason }));
155
+ return { summary: String(reply?.summary ?? '').slice(0, 600), overall, recommendation, reasons, scores: out, criteria, look, facts: f };
156
+ }
157
+
158
+ export const RECOMMENDATION = { ready: 'Ready to approve', look: 'Look at these first', not_ready: 'Not ready' };
159
+
160
+ /** Markdown: for Copy / Download, the pull request's description and a PR comment. */
161
+ export function scorecardMarkdown(sc, title = '') {
162
+ const bar = (n) => (n == null ? '—' : `${n}`);
163
+ const rows = DIMENSIONS.map(([k, label]) => {
164
+ const d = sc.scores[k];
165
+ return `| ${label} | ${bar(d?.score)} | ${d ? [d.why, ...d.notes].filter(Boolean).join('; ').replace(/\|/g, '/') : k === 'spec' ? 'no spec written' : '—'} |`;
166
+ });
167
+ const out = [
168
+ `### Review scorecard${title ? `: ${title}` : ''}`,
169
+ '',
170
+ `**${sc.overall ?? '—'}/100 · ${RECOMMENDATION[sc.recommendation]}**${sc.reasons.length ? ` (${sc.reasons.join('; ')})` : ''}`,
171
+ '',
172
+ sc.summary,
173
+ '',
174
+ '| Score | /100 | Why |',
175
+ '| --- | --- | --- |',
176
+ ...rows,
177
+ ];
178
+ if (sc.criteria.length) {
179
+ out.push('', '**Acceptance criteria**', '');
180
+ for (const c of sc.criteria) out.push(`- ${c.status === 'met' ? '✅' : c.status === 'not_met' ? '❌' : '❔'} ${c.criterion}${c.evidence ? ` (${c.evidence})` : ''}`);
181
+ }
182
+ if (sc.look.length) {
183
+ out.push('', '**Where to look**', '');
184
+ for (const l of sc.look) out.push(`- \`${l.file}\` \`${l.head.replace(/^(@@[^@]*@@).*/, '$1')}\`: ${l.reason}`);
185
+ }
186
+ const f = sc.facts;
187
+ out.push('', `<sub>Measured: ${f.files} files (+${f.added} −${f.removed}), ${f.testFiles} test files, CI ${f.ci ?? 'n/a'}, checks ${f.checks.passed}✓ ${f.checks.failed}✗${f.plan ? `, plan ${f.plan.done}/${f.plan.steps}` : ''}. Scored by AI Agent Manager.</sub>`);
188
+ return out.join('\n');
189
+ }
package/src/server.js CHANGED
@@ -241,6 +241,7 @@ export async function startServer({ port = PORT, host = HOST } = {}) {
241
241
  }),
242
242
  );
243
243
  api.get('/agents/:id/specs/:name', wrap((req) => manager.specDoc(req.params.id, req.params.name)));
244
+ api.post('/agents/:id/scorecard/post', wrap((req) => review.postScorecard(req.params.id)));
244
245
  api.post('/agents/:id/plan', wrap((req) => manager.startPlanning(req.params.id)));
245
246
  api.post('/agents/:id/plan/approve', wrap((req) => manager.approvePlan(req.params.id, req.body?.note)));
246
247
  api.post('/agents/:id/plan/changes', wrap((req) => manager.requestPlanChanges(req.params.id, req.body?.note)));