@xsft/panels 0.7.2 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +9 -9
- package/bin/panels.js +1 -3
- package/lib/args.js +3 -9
- package/lib/judge.js +17 -24
- package/lib/panel.js +6 -2
- package/lib/report.js +6 -2
- package/lib/table.js +279 -135
- package/package.json +2 -3
- package/prompts/judge-system-prompt.txt +21 -23
package/README.md
CHANGED
|
@@ -5,12 +5,10 @@
|
|
|
5
5
|
```text
|
|
6
6
|
panels run <panel> [target...] [flags]
|
|
7
7
|
|
|
8
|
-
<panel> owner/name, or owner/name
|
|
8
|
+
<panel> owner/name for the latest version, or owner/name@N to pin version N
|
|
9
9
|
[target...] free words, joined with single spaces into one target
|
|
10
10
|
|
|
11
11
|
--harness <name> claude | codex default: claude if installed, else codex
|
|
12
|
-
--brief <file> the brief default: none
|
|
13
|
-
--sources <path> a doc folder, a repository, or an app URL; repeat the flag for each
|
|
14
12
|
--yolo the judges skip every permission question (claude --dangerously-skip-permissions, codex --yolo).
|
|
15
13
|
Without it, a judge asks its own questions in its tab. Without a terminal, the flag is required.
|
|
16
14
|
-- end of the flags; each later word joins the target
|
|
@@ -20,20 +18,22 @@ exit 1 fail: every judge answered, and one or more criteria scored below 4
|
|
|
20
18
|
exit 2 error: bad flags, no target, panel not found, harness missing, or a judge failed
|
|
21
19
|
```
|
|
22
20
|
|
|
23
|
-
CAUTION: A judge gets no tool block and no sandbox from the CLI. Without `--yolo`, the harness asks its own permission questions in the judge tab. With `--yolo`, nothing asks: the judge can read, write, run commands, and reach the network as you, and the system prompt (do not write or change files
|
|
21
|
+
CAUTION: A judge gets no tool block and no sandbox from the CLI. Without `--yolo`, the harness asks its own permission questions in the judge tab. With `--yolo`, nothing asks: the judge can read, write, run commands, and reach the network as you, and the system prompt (do not write or change files of the target, no forms, no posts) is the only guard. Do not run a panel on a repository that you do not trust.
|
|
24
22
|
|
|
25
23
|
```sh
|
|
26
|
-
npx @xsft/panels run xsft/marketing-site https://example.com --
|
|
27
|
-
npx @xsft/panels run xsft/marketing-site
|
|
24
|
+
npx @xsft/panels run xsft/marketing-site https://example.com --harness claude --yolo
|
|
25
|
+
npx @xsft/panels run xsft/marketing-site@8 . at http://localhost:4321 --harness codex
|
|
28
26
|
```
|
|
29
27
|
|
|
30
|
-
Run the command in the folder of the product. Each judge starts in that folder, so `.`, a branch, and relative paths resolve there. The CLI does not test, fetch, or change the target. Four judges run at a time. A judge gets 20 minutes.
|
|
28
|
+
Run the command in the folder of the product. Each judge starts in that folder, so `.`, a branch, and relative paths resolve there. The CLI does not test, fetch, or change the target. Four judges run at a time. A judge gets 20 minutes. The criteria of a panel say what to assess; the system prompt of the CLI (`prompts/judge-system-prompt.txt`) carries how to judge and the four levels: 4 Met, 3 Almost met, 2 Falls short, 1 Fails. Only 4 passes.
|
|
31
29
|
|
|
32
30
|
## Terminal mode
|
|
33
31
|
|
|
34
32
|
On a terminal, `panels run` shows the overview, and each judge runs in its interactive harness in a pseudo-terminal inside the command. Each judge writes `judges/N/answer.json` and stays open. The overview writes the report when each judge has an answer or an error, and the command exits from `report.json` when you quit. Without a terminal, the judges run in the background (`claude -p`, `codex exec`), and stdout prints `N/4 name` and the verdict of each criterion, then the report path. Plain mode needs `--yolo`, because a background judge cannot answer a question; without it the command exits 2.
|
|
35
33
|
|
|
36
|
-
The overview fills the terminal: a header line with the run (panel and target at the left, the version, the harness, and the elapsed time at the right edge) and two columns from 100 columns up (stacked below that). The left column lists the criteria: rank, name, score (4 green, 3 ink, 2 yellow, 1 red, a grey `·/4` until the score arrives), the time the judge has run, and the tokens it used, with a total row at the bottom under a hairline: the scores added as quarters over the number of criteria, the longest judge time, and the tokens added up. The right column shows the criterion under the cursor: its name, score, and level word,
|
|
34
|
+
The overview fills the terminal: a header line with the run (panel and target at the left, the version, the harness, and the elapsed time at the right edge) and two columns from 100 columns up (stacked below that). The left column lists the criteria: rank, name, score (4 green, 3 ink, 2 yellow, 1 red, a grey `·/4` until the score arrives), the time the judge has run, and the tokens it used, with a total row at the bottom under a hairline: the scores added as quarters over the number of criteria, the longest judge time, and the tokens added up. The right column shows the criterion under the cursor: its name, score, and level word, then a sub tab bar, `Verdict │ Criterion`, and on the last row the key line. `tab` switches the sub tab: Verdict shows the verdict paragraph, Criterion the description and the body of the criterion as plain text. `enter` moves the focus into the pane (an orange bar marks it): `↑` and `↓` scroll one line, `PgUp` and `PgDn` a page, `esc` or `enter` return to the list. While a judge runs, the Verdict sub tab shows a piece of halftone art in a square frame, generated from a seed per criterion for the run (a half disc, a disc, a ring, a quarter disc, a band, or two discs), breathing, centred in the pane, with the time under it. One soft wave sweeps across the running rows and the art. On the overview, `↑` and `↓` move the cursor. The page never says pass or fail; the total is the result.
|
|
35
|
+
|
|
36
|
+
`r` opens a small modal that asks before it restarts the judge of the cursor row (the current answer is discarded; `enter` on restart runs it, `esc` cancels). `h` opens a modal that picks the harness of that judge, `claude` or `codex`, for its next restart. While a modal is open, every other key is ignored.
|
|
37
37
|
|
|
38
38
|
A tab strip on the last three lines shows a hairline, the tabs (`Overview`, then the name of each criterion, split by a grey `│`, cut to the first word when they do not fit), and a status line with the progress, the elapsed time, and the keys. The strip is the navigation area. With the focus on the strip, `←` and `→` move the active tab and switch the view, `1` to `9` jump to a started judge by list order, `0` to the overview, `enter` gives the focus to the judge, and `q` quits; while a judge still runs, the first `q` asks and a second `q` quits. With the focus on a judge, each key goes to the judge, and `ctrl-q` gives the focus back to the strip. The overview takes no text, so the strip keeps the focus there. A harness that has not seen the folder asks whether to trust it. The first judge starts alone; press `1`, answer, and the other judges start.
|
|
39
39
|
|
|
@@ -50,7 +50,7 @@ The last line on stdout is the path of `report.json`.
|
|
|
50
50
|
## Skipped
|
|
51
51
|
|
|
52
52
|
- Run upload to cross.software, and `--no-upload`.
|
|
53
|
-
- `--version`: `owner/name
|
|
53
|
+
- `--version`: `owner/name@N` pins a version. The CLI fetches the panel from the api on every run and never caches it.
|
|
54
54
|
- `--out`: the run folder is fixed.
|
|
55
55
|
- `--json`: stdout ends with the path of `report.json`.
|
|
56
56
|
- `--concurrency`: fixed at 4.
|
package/bin/panels.js
CHANGED
|
@@ -29,7 +29,7 @@ const EXIT = { pass: 0, fail: 1, error: 2 };
|
|
|
29
29
|
|
|
30
30
|
/** Runs the judges in the overview, and exits from report.json when the engineer quits. Prints nothing. */
|
|
31
31
|
async function runOnTerminal({ panel, runDir, harness, cwd, args }) {
|
|
32
|
-
await runTable({ panel, runDir, harness, cwd, target: args.target,
|
|
32
|
+
await runTable({ panel, runDir, harness, cwd, target: args.target, yolo: args.yolo });
|
|
33
33
|
const reportPath = join(runDir, 'report.json');
|
|
34
34
|
if (!existsSync(reportPath)) return 2;
|
|
35
35
|
return EXIT[JSON.parse(readFileSync(reportPath, 'utf8')).status];
|
|
@@ -77,8 +77,6 @@ async function main() {
|
|
|
77
77
|
harness,
|
|
78
78
|
cwd,
|
|
79
79
|
target: args.target,
|
|
80
|
-
brief: args.brief,
|
|
81
|
-
sources: args.sources,
|
|
82
80
|
yolo: true,
|
|
83
81
|
}));
|
|
84
82
|
const report = buildReport({ panel, target: args.target, outcomes });
|
package/lib/args.js
CHANGED
|
@@ -2,12 +2,10 @@ import { parseArgs } from 'node:util';
|
|
|
2
2
|
|
|
3
3
|
export const SYNOPSIS = `panels run <panel> [target...] [flags]
|
|
4
4
|
|
|
5
|
-
<panel> owner/name, or owner/name
|
|
5
|
+
<panel> owner/name for the latest version, or owner/name@N to pin version N
|
|
6
6
|
[target...] free words, joined with single spaces into one target
|
|
7
7
|
|
|
8
8
|
--harness <name> claude | codex default: claude if installed, else codex
|
|
9
|
-
--brief <file> the brief default: none
|
|
10
|
-
--sources <path> a doc folder, a repository, or an app URL; repeat the flag for each
|
|
11
9
|
--yolo the judges skip every permission question (claude --dangerously-skip-permissions, codex --yolo).
|
|
12
10
|
Without it, a judge asks its own questions in its tab. Without a terminal, the flag is required.
|
|
13
11
|
-- end of the flags; each later word joins the target
|
|
@@ -18,13 +16,11 @@ exit 2 error: bad flags, no target, panel not found, harness missing, or a judg
|
|
|
18
16
|
|
|
19
17
|
const options = {
|
|
20
18
|
harness: { type: 'string' },
|
|
21
|
-
brief: { type: 'string' },
|
|
22
|
-
sources: { type: 'string', multiple: true },
|
|
23
19
|
yolo: { type: 'boolean' },
|
|
24
20
|
help: { type: 'boolean' },
|
|
25
21
|
};
|
|
26
22
|
|
|
27
|
-
const PANEL = /^([a-z0-9-]+)\/([a-z0-9-]+)(
|
|
23
|
+
const PANEL = /^([a-z0-9-]+)\/([a-z0-9-]+)(?:@([1-9]\d*))?$/;
|
|
28
24
|
|
|
29
25
|
/** Returns the run options, { help: true }, { usage: true }, or { error } with a one-line message. */
|
|
30
26
|
export function parse(argv) {
|
|
@@ -38,15 +34,13 @@ export function parse(argv) {
|
|
|
38
34
|
const [command, panel, ...words] = positionals;
|
|
39
35
|
if (command !== 'run' || panel === undefined) return { usage: true };
|
|
40
36
|
const match = PANEL.exec(panel);
|
|
41
|
-
if (!match) return { error: `bad panel ${panel}: use owner/name or owner/name
|
|
37
|
+
if (!match) return { error: `bad panel ${panel}: use owner/name or owner/name@N` };
|
|
42
38
|
if (words.length === 0) return { error: 'no target: type what the panel reviews after the panel name, such as a URL, a path, or a branch' };
|
|
43
39
|
if (values.harness !== undefined && !['claude', 'codex'].includes(values.harness)) return { error: `--harness needs claude or codex, not ${values.harness}` };
|
|
44
40
|
return {
|
|
45
41
|
panel: { owner: match[1], name: match[2], version: match[3] && Number(match[3]) },
|
|
46
42
|
target: words.join(' '),
|
|
47
43
|
harness: values.harness,
|
|
48
|
-
brief: values.brief,
|
|
49
|
-
sources: values.sources ?? [],
|
|
50
44
|
yolo: Boolean(values.yolo),
|
|
51
45
|
};
|
|
52
46
|
}
|
package/lib/judge.js
CHANGED
|
@@ -1,26 +1,24 @@
|
|
|
1
1
|
import { spawn } from 'node:child_process';
|
|
2
2
|
import { closeSync, mkdirSync, openSync, readFileSync, writeFileSync } from 'node:fs';
|
|
3
|
-
import {
|
|
3
|
+
import { join } from 'node:path';
|
|
4
4
|
|
|
5
5
|
const { criterion: CRITERION } = JSON.parse(readFileSync(new URL('../schema/report.json', import.meta.url), 'utf8')).$defs;
|
|
6
6
|
const SYSTEM_PROMPT = readFileSync(new URL('../prompts/judge-system-prompt.txt', import.meta.url), 'utf8');
|
|
7
7
|
export const MINUTES_PER_CRITERION = 20;
|
|
8
8
|
// A judge gets no tool list, no permission mode, and no sandbox flag: the harness defaults, and its own questions
|
|
9
9
|
// in its tab. With --yolo, the judge skips every question: no sandbox, nothing is denied, nothing asks. The system
|
|
10
|
-
// prompt holds the rules on writing files,
|
|
10
|
+
// prompt holds the rules on writing files, forms, and messages. exec resume takes no sandbox flag,
|
|
11
11
|
// so it gets the same setting as config values.
|
|
12
12
|
// codex --yolo is the alias of --dangerously-bypass-approvals-and-sandbox.
|
|
13
13
|
const yoloFlags = (yolo, harness) => (!yolo ? [] : harness === 'claude' ? ['--dangerously-skip-permissions'] : ['--yolo']);
|
|
14
14
|
const CODEX_RESUME_YOLO = ['-c', 'sandbox_mode="danger-full-access"', '-c', 'approval_policy="never"'];
|
|
15
15
|
|
|
16
16
|
/**
|
|
17
|
-
* The user message of a judge. criterion is { rank, name, description, body }.
|
|
18
|
-
* because a body is markdown with headings of its own.
|
|
17
|
+
* The user message of a judge: the criterion and the target. criterion is { rank, name, description, body }.
|
|
18
|
+
* Each part sits in a tag, because a body is markdown with headings of its own.
|
|
19
19
|
*/
|
|
20
|
-
export function userMessage(criterion, { target
|
|
21
|
-
|
|
22
|
-
parts.push(`<target>\n${target}\n</target>`, `<brief>\n${brief ?? 'none supplied'}\n</brief>`, `<sources>\n${sources.length ? sources.join('\n') : 'none'}\n</sources>`);
|
|
23
|
-
return `${parts.join('\n\n')}\n`;
|
|
20
|
+
export function userMessage(criterion, { target }) {
|
|
21
|
+
return `<criterion rank="${criterion.rank}" name="${criterion.name}">\nDescription: ${criterion.description}\n\n${criterion.body}\n</criterion>\n\n<target>\n${target}\n</target>\n`;
|
|
24
22
|
}
|
|
25
23
|
|
|
26
24
|
/** The output schema of a judge: one criterion object { name, score, verdict }, with the name fixed. */
|
|
@@ -41,12 +39,6 @@ export function check(answer, criterion) {
|
|
|
41
39
|
return errors;
|
|
42
40
|
}
|
|
43
41
|
|
|
44
|
-
/** The folders of the brief and the path sources that are outside cwd. A Claude judge cannot read them without --add-dir. */
|
|
45
|
-
export function addDirs(cwd, brief, sources) {
|
|
46
|
-
const paths = [brief && dirname(resolve(cwd, brief)), ...sources.filter((s) => !/^https?:\/\//.test(s)).map((s) => resolve(cwd, s))];
|
|
47
|
-
return [...new Set(paths.filter(Boolean))].filter((p) => relative(cwd, p).split(sep)[0] === '..');
|
|
48
|
-
}
|
|
49
|
-
|
|
50
42
|
function exec(command, args, { cwd, stdin, timeout }) {
|
|
51
43
|
return new Promise((done) => {
|
|
52
44
|
const child = spawn(command, args, { cwd, timeout, stdio: [stdin, 'pipe', 'pipe'] });
|
|
@@ -65,7 +57,6 @@ function exec(command, args, { cwd, stdin, timeout }) {
|
|
|
65
57
|
async function askClaude(j, prompt, session) {
|
|
66
58
|
const args = [
|
|
67
59
|
'-p', ...yoloFlags(j.yolo, 'claude'),
|
|
68
|
-
...j.addDirs.flatMap((d) => ['--add-dir', d]),
|
|
69
60
|
'--system-prompt', SYSTEM_PROMPT,
|
|
70
61
|
'--json-schema', JSON.stringify(j.schema),
|
|
71
62
|
'--output-format', 'json',
|
|
@@ -109,13 +100,13 @@ export function parseAnswer(text) {
|
|
|
109
100
|
* Runs one judge to the end: the first answer, and one re-ask when check() rejects it.
|
|
110
101
|
* Returns { answer } or { error }. command is the harness binary; the tests pass a stub.
|
|
111
102
|
*/
|
|
112
|
-
export async function runJudge({ dir, criterion, harness, cwd, target,
|
|
103
|
+
export async function runJudge({ dir, criterion, harness, cwd, target, yolo = false }, command = harness) {
|
|
113
104
|
mkdirSync(dir, { recursive: true });
|
|
114
|
-
const prompt = userMessage(criterion, { target
|
|
105
|
+
const prompt = userMessage(criterion, { target });
|
|
115
106
|
const schema = judgeSchema(criterion);
|
|
116
107
|
writeFileSync(join(dir, 'prompt.md'), prompt);
|
|
117
108
|
writeFileSync(join(dir, 'schema.json'), JSON.stringify(schema, null, 2));
|
|
118
|
-
const j = { dir, cwd, schema, command, yolo,
|
|
109
|
+
const j = { dir, cwd, schema, command, yolo, timeout: MINUTES_PER_CRITERION * 60_000 };
|
|
119
110
|
const ask = harness === 'claude' ? askClaude : askCodex;
|
|
120
111
|
let reply = await ask(j, prompt);
|
|
121
112
|
if (reply.error) return reply;
|
|
@@ -137,17 +128,19 @@ export const tomlString = (s) => `"${s.replace(/["\\\x00-\x1f\x7f]/g, (c) => TOM
|
|
|
137
128
|
* Writes prompt.md and schema.json of one judge of the terminal mode, and returns the command and the
|
|
138
129
|
* arguments that start the judge in its interactive harness.
|
|
139
130
|
*/
|
|
140
|
-
export function writeJudge({ dir, criterion, harness, cwd, target,
|
|
131
|
+
export function writeJudge({ dir, criterion, harness, cwd, target, yolo = false }) {
|
|
141
132
|
mkdirSync(dir, { recursive: true });
|
|
142
|
-
|
|
143
|
-
|
|
133
|
+
// The answer file rule sits at the end of the user prompt and in the system prompt, so a judge that hands the
|
|
134
|
+
// work to a sub-agent still leaves the file at the exact path.
|
|
135
|
+
const closing = `The engineer ran the command in ${cwd}. Your answer counts only as the file ${join(dir, 'answer.json')}: write it with that exact absolute path, as one JSON object { name, score, verdict } that matches ${join(dir, 'schema.json')}. JSON in your reply does not count. Whoever does the work, that file must exist at that path when you stop. These two files are the only exceptions to the rules on writing files and on the .panels folder.\n`;
|
|
136
|
+
const prompt = `${userMessage(criterion, { target })}\n${closing}`;
|
|
137
|
+
const system = `${SYSTEM_PROMPT}\n${closing}`;
|
|
144
138
|
writeFileSync(join(dir, 'prompt.md'), prompt);
|
|
145
139
|
writeFileSync(join(dir, 'schema.json'), JSON.stringify(judgeSchema(criterion), null, 2));
|
|
146
140
|
if (harness === 'codex') {
|
|
147
141
|
// Codex ranks the developer instructions above the prompt, and the system prompt says to return the JSON and to
|
|
148
142
|
// write no files. Without the closing in the developer instructions, the judge prints the JSON and writes no answer.json.
|
|
149
|
-
return ['codex', '-C', dir, ...yoloFlags(yolo, 'codex'), '-c', `developer_instructions=${tomlString(
|
|
143
|
+
return ['codex', '-C', dir, ...yoloFlags(yolo, 'codex'), '-c', `developer_instructions=${tomlString(system)}`, prompt];
|
|
150
144
|
}
|
|
151
|
-
|
|
152
|
-
return ['claude', ...yoloFlags(yolo, 'claude'), '--system-prompt', SYSTEM_PROMPT, ...add, prompt];
|
|
145
|
+
return ['claude', ...yoloFlags(yolo, 'claude'), '--system-prompt', system, prompt];
|
|
153
146
|
}
|
package/lib/panel.js
CHANGED
|
@@ -1,8 +1,12 @@
|
|
|
1
1
|
const API = 'https://cross.software/panels/api/panels';
|
|
2
2
|
|
|
3
|
-
/**
|
|
3
|
+
/**
|
|
4
|
+
* Reads a panel from the public api on every run, the latest version unless one is pinned, and never from a
|
|
5
|
+
* cache: the api answers no-store, and the fetch asks for no store too, so a proxy cannot keep a copy.
|
|
6
|
+
* An unpublished panel also answers 404.
|
|
7
|
+
*/
|
|
4
8
|
export async function fetchPanel({ owner, name, version }) {
|
|
5
|
-
const res = await fetch(`${API}/${owner}/${name}${version ? `/v/${version}` : ''}
|
|
9
|
+
const res = await fetch(`${API}/${owner}/${name}${version ? `/v/${version}` : ''}`, { cache: 'no-store' });
|
|
6
10
|
if (res.status === 404) return { error: 'panel not found' };
|
|
7
11
|
if (!res.ok) return { error: `panel fetch failed: ${res.status}` };
|
|
8
12
|
return { panel: await res.json() };
|
package/lib/report.js
CHANGED
|
@@ -18,7 +18,11 @@ export function buildReport({ panel, target, outcomes }) {
|
|
|
18
18
|
return { panel: `${panel.owner}/${panel.name}@${panel.version}`, target, status, criteria };
|
|
19
19
|
}
|
|
20
20
|
|
|
21
|
-
/**
|
|
21
|
+
/**
|
|
22
|
+
* The lines that the plain mode prints: the run (panel and its version, the target, the CLI version), then
|
|
23
|
+
* "N/4 name" and the verdict under it, for each criterion.
|
|
24
|
+
*/
|
|
22
25
|
export function renderPlain(report) {
|
|
23
|
-
|
|
26
|
+
const [name, version] = report.panel.split('@');
|
|
27
|
+
return [`panels run ${name} v${version} ${report.target} ${VERSION}`, '', ...report.criteria.flatMap((c) => [`${c.score === null ? 'err' : `${c.score}/4`} ${c.name}`, ` ${c.verdict}`, ''])];
|
|
24
28
|
}
|
package/lib/table.js
CHANGED
|
@@ -5,7 +5,6 @@ import { join } from 'node:path';
|
|
|
5
5
|
import pty from '@homebridge/node-pty-prebuilt-multiarch';
|
|
6
6
|
import { SerializeAddon } from '@xterm/addon-serialize';
|
|
7
7
|
import xterm from '@xterm/headless';
|
|
8
|
-
import figlet from 'figlet';
|
|
9
8
|
import { check, MINUTES_PER_CRITERION, parseAnswer, writeJudge } from './judge.js';
|
|
10
9
|
import { buildReport, VERSION } from './report.js';
|
|
11
10
|
|
|
@@ -76,18 +75,9 @@ export function tokensOnScreen(text) {
|
|
|
76
75
|
return best;
|
|
77
76
|
}
|
|
78
77
|
|
|
79
|
-
let artLines;
|
|
80
|
-
/** The word "judging" as big letters, in the ANSI Shadow font (Standard when that font is missing). Rendered once. */
|
|
81
|
-
export function art() {
|
|
82
|
-
if (!artLines) {
|
|
83
|
-
const font = figlet.fontsSync().includes('ANSI Shadow') ? 'ANSI Shadow' : 'Standard';
|
|
84
|
-
artLines = figlet.textSync('judging', { font }).split('\n').filter((l) => l.trim());
|
|
85
|
-
}
|
|
86
|
-
return artLines;
|
|
87
|
-
}
|
|
88
|
-
|
|
89
78
|
const plainLength = (s) => s.replace(/\x1b\[[0-9;]*m/g, '').length;
|
|
90
79
|
const padTo = (s, width) => s + ' '.repeat(Math.max(0, width - plainLength(s)));
|
|
80
|
+
const centre = (s, width) => ' '.repeat(Math.max(0, Math.floor((width - plainLength(s)) / 2))) + s;
|
|
91
81
|
|
|
92
82
|
/** The first width visible characters of a styled line, with every escape sequence kept, so the line never wraps. */
|
|
93
83
|
export function clip(line, width) {
|
|
@@ -117,8 +107,77 @@ export function wrap(text, width) {
|
|
|
117
107
|
/** The scores added as quarters: 1/4 + 1/4 + 2/4 is "1". No trailing zeros. */
|
|
118
108
|
const quarters = (sum) => String(Math.round((sum / 4) * 100) / 100);
|
|
119
109
|
|
|
110
|
+
// The art of a running judge: a halftone piece inside a square frame, generated from a seed. Two sizes, rows by columns.
|
|
111
|
+
const MARK_SIZES = [[11, 22], [7, 14]];
|
|
112
|
+
const HALFTONE = [' ', '░', '▒', '▓'];
|
|
113
|
+
const SHAPES = ['half-left', 'half-right', 'half-top', 'half-bottom', 'disc', 'ring', 'quarter-tl', 'quarter-tr', 'quarter-bl', 'quarter-br', 'band', 'twin'];
|
|
114
|
+
|
|
115
|
+
/** A small seeded random generator (mulberry32): the same seed gives the same sequence. */
|
|
116
|
+
export function mulberry32(seed) {
|
|
117
|
+
let a = seed | 0;
|
|
118
|
+
return () => {
|
|
119
|
+
a = (a + 0x6D2B79F5) | 0;
|
|
120
|
+
let t = Math.imul(a ^ (a >>> 15), 1 | a);
|
|
121
|
+
t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
|
|
122
|
+
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
|
|
123
|
+
};
|
|
124
|
+
}
|
|
125
|
+
|
|
120
126
|
/**
|
|
121
|
-
* The
|
|
127
|
+
* The lines of the art of a judge at frame: a grey square frame of box-drawing lines, and inside it a shape in a
|
|
128
|
+
* halftone screen, two cells per row unit so it looks round. The seed picks the shape (a half disc on one side,
|
|
129
|
+
* a disc, a ring, a quarter disc in a corner, a diagonal band, or two discs), its size and place, the grain of
|
|
130
|
+
* the screen, and the phase and the speed of its pulse. The screen breathes, and the glimmer wave runs across the
|
|
131
|
+
* cells by screen column, where start is the screen column of the frame. size is [rows, columns].
|
|
132
|
+
*/
|
|
133
|
+
export function renderArt({ seed = 0, frame = 0, start = 0, width = 80, size = MARK_SIZES[0] }) {
|
|
134
|
+
const rng = mulberry32(seed);
|
|
135
|
+
const shape = SHAPES[Math.floor(rng() * SHAPES.length)];
|
|
136
|
+
const scale = 0.65 + rng() * 0.35;
|
|
137
|
+
const dx = (rng() - 0.5) * 0.3;
|
|
138
|
+
const dy = (rng() - 0.5) * 0.3;
|
|
139
|
+
const phase = rng() * Math.PI * 2;
|
|
140
|
+
const period = 14 + rng() * 12;
|
|
141
|
+
const grain = 0.4 + rng() * 0.6;
|
|
142
|
+
const [rows, cols] = size;
|
|
143
|
+
const inner = { rows: rows - 2, cols: cols - 2 };
|
|
144
|
+
const w = inner.cols / 2;
|
|
145
|
+
const h = inner.rows;
|
|
146
|
+
const r = (scale * h) / 2;
|
|
147
|
+
const centres = {
|
|
148
|
+
'half-left': [0, h / 2], 'half-right': [w, h / 2], 'half-top': [w / 2, 0], 'half-bottom': [w / 2, h],
|
|
149
|
+
'quarter-tl': [0, 0], 'quarter-tr': [w, 0], 'quarter-bl': [0, h], 'quarter-br': [w, h],
|
|
150
|
+
};
|
|
151
|
+
const [cx, cy] = centres[shape] ?? [w / 2 + dx * w, h / 2 + dy * h];
|
|
152
|
+
// The distance of a cell from the shape, 0 at the heart and 1 at the edge; past 1 the cell is empty.
|
|
153
|
+
const distance = (u, v) => {
|
|
154
|
+
const d = Math.hypot(u - cx, v - cy);
|
|
155
|
+
if (shape === 'ring') { const t = Math.abs(d - r * 0.75) / (r * 0.3); return t; }
|
|
156
|
+
if (shape === 'band') return Math.abs((u - cx) - (v - cy)) / (r * 0.6);
|
|
157
|
+
if (shape === 'twin') return Math.min(Math.hypot(u - (cx - r * 0.5), v - cy), Math.hypot(u - (cx + r * 0.5), v - cy)) / (r * 0.75);
|
|
158
|
+
if (shape.startsWith('quarter')) return d / (r * 1.5);
|
|
159
|
+
return d / r;
|
|
160
|
+
};
|
|
161
|
+
const pulse = Math.sin((2 * Math.PI * frame) / period + phase);
|
|
162
|
+
const lines = [`${GREY}┌${'─'.repeat(inner.cols)}┐${RESET}`];
|
|
163
|
+
for (let y = 0; y < inner.rows; y++) {
|
|
164
|
+
let cells = '';
|
|
165
|
+
for (let x = 0; x < inner.cols; x++) {
|
|
166
|
+
const d = distance((x + 0.5) / 2, y + 0.5);
|
|
167
|
+
const noise = mulberry32(seed * 7919 + y * 1021 + x * 31)() - 0.5;
|
|
168
|
+
cells += d > 1 ? ' ' : HALFTONE[Math.max(1, Math.min(3, Math.round(1.5 + (1 - d) * 1.5 + pulse + noise * grain)))];
|
|
169
|
+
}
|
|
170
|
+
lines.push(`${GREY}│${RESET}${glimmer(cells, frame, start + 1, width)}${GREY}│${RESET}`);
|
|
171
|
+
}
|
|
172
|
+
lines.push(`${GREY}└${'─'.repeat(inner.cols)}┘${RESET}`);
|
|
173
|
+
return lines;
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
/** The size of the art that fits a pane of columnWidth by lines, or null when only the one-line form fits. */
|
|
177
|
+
export const markSize = (columnWidth, lines) => MARK_SIZES.find(([r, c]) => r + 2 <= lines && c <= columnWidth) ?? null;
|
|
178
|
+
|
|
179
|
+
/**
|
|
180
|
+
* The body of a criterion as plain lines for the text pane: a heading becomes a grey upper-case label, a list
|
|
122
181
|
* item keeps its dash, a fenced block is indented four spaces as it is, a table line stays as it is, and the
|
|
123
182
|
* other text is wrapped to width with the emphasis, code, and link marks removed.
|
|
124
183
|
*/
|
|
@@ -149,23 +208,10 @@ export function plainMarkdown(markdown, width) {
|
|
|
149
208
|
return lines;
|
|
150
209
|
}
|
|
151
210
|
|
|
152
|
-
/** The lines of the
|
|
153
|
-
function
|
|
154
|
-
const lines = [
|
|
155
|
-
if (row.
|
|
156
|
-
// The judge works: the word judging in big letters with the wave over it, or in one line in a narrow pane.
|
|
157
|
-
const letters = art();
|
|
158
|
-
lines.push(...(letters[0].length <= columnWidth ? letters : ['judging']).map((l) => glimmer(l, frame, start, width)));
|
|
159
|
-
const since = row.restarted !== null ? `restarted with ${row.harness} · ` : '';
|
|
160
|
-
lines.push(`${GREY}${since}${clock(row.time ?? 0)} · ${row.rank} to watch${RESET}`);
|
|
161
|
-
} else {
|
|
162
|
-
const text = row.state === 'done' ? row.result.verdict
|
|
163
|
-
: row.state === 'error' ? row.error
|
|
164
|
-
: row.state === 'queued' ? 'The judge has not started.'
|
|
165
|
-
: `Press ${row.rank} and answer the trust prompt.`;
|
|
166
|
-
lines.push(...wrap(text, columnWidth));
|
|
167
|
-
}
|
|
168
|
-
if (row.description) lines.push('', ...wrap(row.description, columnWidth));
|
|
211
|
+
/** The lines of the body of a criterion in the Panel tab. */
|
|
212
|
+
function criterionText(row, columnWidth) {
|
|
213
|
+
const lines = [];
|
|
214
|
+
if (row.description) lines.push(...wrap(row.description, columnWidth));
|
|
169
215
|
if (row.body) lines.push('', ...plainMarkdown(row.body, columnWidth));
|
|
170
216
|
return lines;
|
|
171
217
|
}
|
|
@@ -176,35 +222,81 @@ const layout = (rows, width) => {
|
|
|
176
222
|
return width >= TWO_COLUMNS ? { leftWidth, columnWidth: width - leftWidth - 3, start: leftWidth + 3 } : { leftWidth, columnWidth: width - 2, start: 2 };
|
|
177
223
|
};
|
|
178
224
|
|
|
179
|
-
|
|
180
|
-
|
|
225
|
+
const headerLine = (title, target, harness, elapsed, width, version) => {
|
|
226
|
+
const run = ` panels run ${title} ${target}`;
|
|
227
|
+
const meta = `${version} · ${harness} · ${clock(elapsed)} `;
|
|
228
|
+
return ` ${BOLD}panels run${RESET} ${title}${GREY} ${target}${' '.repeat(Math.max(2, width - run.length - meta.length))}${meta}${RESET}`;
|
|
229
|
+
};
|
|
230
|
+
|
|
231
|
+
const paneHeight = (width, body) => (width >= TWO_COLUMNS ? body : body - Math.floor(body / 2) - 1);
|
|
232
|
+
|
|
233
|
+
// The detail pane holds the header, the sub tab bar, its hairline, the content, then the key line on its last row.
|
|
234
|
+
const PANE_FRAME = 4;
|
|
235
|
+
|
|
236
|
+
/** The lines of the detail pane under its sub tab bar: the verdict (or the mark while running), or the criterion text. */
|
|
237
|
+
function detailContent(row, sub, columnWidth, start, width, frame, window) {
|
|
238
|
+
if (sub === 'criterion') return criterionText(row, columnWidth);
|
|
239
|
+
if (row.state === 'running' && !row.trust) {
|
|
240
|
+
const since = row.restarted !== null ? `restarted with ${row.harness} · ` : '';
|
|
241
|
+
const line = `${GREY}${since}${clock(row.time ?? 0)} · ${row.rank} to watch${RESET}`;
|
|
242
|
+
const size = markSize(columnWidth, window ?? MARK_SIZES[0][0] + 2);
|
|
243
|
+
const art = size ? renderArt({ seed: row.seed ?? 0, frame, start: start + Math.floor((columnWidth - size[1]) / 2), width, size }) : [glimmer('judging', frame, start + Math.floor((columnWidth - 7) / 2), width)];
|
|
244
|
+
const block = [...art, '', line].map((l) => centre(l, columnWidth));
|
|
245
|
+
const above = window === undefined ? 0 : Math.max(0, Math.floor((window - block.length) / 2));
|
|
246
|
+
return [...Array(above).fill(''), ...block];
|
|
247
|
+
}
|
|
248
|
+
const text = row.state === 'done' ? row.result.verdict : row.state === 'error' ? row.error : row.state === 'queued' ? 'The judge has not started.' : `Press ${row.rank} and answer the trust prompt.`;
|
|
249
|
+
return wrap(text, columnWidth);
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
/** The number of lines of the detail pane of the overview that the scroll can hide, for the sub tab sub. */
|
|
253
|
+
export function overviewScrollMax({ rows, cursor, sub = 'verdict', width, height }) {
|
|
181
254
|
const { columnWidth, start } = layout(rows, width);
|
|
182
|
-
const
|
|
183
|
-
|
|
184
|
-
const paneHeight = width >= TWO_COLUMNS ? body : body - Math.floor(body / 2) - 1;
|
|
185
|
-
// The pane holds the header, the content window, a blank line, and the three bottom lines.
|
|
186
|
-
const window = paneHeight - 5;
|
|
187
|
-
return window < 1 ? 0 : Math.max(0, content.length - window);
|
|
255
|
+
const window = paneHeight(width, height - 2) - PANE_FRAME;
|
|
256
|
+
return window < 1 ? 0 : Math.max(0, detailContent(rows[cursor], sub, columnWidth, start, width, 0, window).length - window);
|
|
188
257
|
}
|
|
189
258
|
|
|
190
259
|
/**
|
|
191
|
-
*
|
|
192
|
-
*
|
|
193
|
-
*
|
|
194
|
-
* a restart (null for none). cursor is the index of the row whose detail shows at the right; focus is 'list'
|
|
195
|
-
* or 'detail' (the detail pane shows an orange bar and its header turns orange); scroll hides that many lines
|
|
196
|
-
* of the detail content under its header. The header line holds the run: panel and target at the left, the
|
|
197
|
-
* version, the harness, and the elapsed time at the right edge. Two columns from TWO_COLUMNS wide, the list
|
|
198
|
-
* above the detail below that. height is the number of lines the overview may fill: the column divider runs
|
|
199
|
-
* the whole height, the rows and the detail header stay at the top, and the total row and the key and fact
|
|
200
|
-
* lines sit at the bottom. Without a height, or when the content is taller than the height, the lines follow
|
|
201
|
-
* each other and nothing scrolls.
|
|
260
|
+
* Lays a page out: the header and the hairline, then the list at the left and a pane at the right from
|
|
261
|
+
* TWO_COLUMNS wide, or the list above the pane below that. pane(lines) gives the lines of the pane for a height,
|
|
262
|
+
* or for undefined when there is no height. edge is the divider character between the columns.
|
|
202
263
|
*/
|
|
203
|
-
|
|
204
|
-
const
|
|
205
|
-
const
|
|
206
|
-
|
|
207
|
-
|
|
264
|
+
function page({ header, width, height, left, leftBottom, pane, edge }) {
|
|
265
|
+
const { leftWidth } = layout(left.rows, width);
|
|
266
|
+
const fill = (top, bottom, lines) => {
|
|
267
|
+
const n = lines === undefined || lines < top.length + bottom.length ? 0 : lines - top.length - bottom.length;
|
|
268
|
+
return [...top, ...Array(n).fill(''), ...bottom];
|
|
269
|
+
};
|
|
270
|
+
const lines = [header, `${GREY}${'─'.repeat(width)}${RESET}`];
|
|
271
|
+
const body = height === undefined ? undefined : height - 2;
|
|
272
|
+
if (width >= TWO_COLUMNS) {
|
|
273
|
+
const leftLines = fill(left.lines, leftBottom, body);
|
|
274
|
+
const rightLines = pane(body);
|
|
275
|
+
for (let i = 0; i < Math.max(leftLines.length, rightLines.length); i++) lines.push(`${padTo(leftLines[i] ?? '', leftWidth)} ${edge} ${rightLines[i] ?? ''}`);
|
|
276
|
+
} else {
|
|
277
|
+
const half = body === undefined ? undefined : Math.floor(body / 2);
|
|
278
|
+
const list = fill(left.lines, leftBottom, half);
|
|
279
|
+
lines.push(...list, '', ...pane(body === undefined ? undefined : body - list.length - 1).map((l) => `${edge === `${GREY}│${RESET}` ? ' ' : edge} ${l}`));
|
|
280
|
+
}
|
|
281
|
+
return lines.map((l) => clip(l, width));
|
|
282
|
+
}
|
|
283
|
+
|
|
284
|
+
/**
|
|
285
|
+
* The lines of the overview. rows hold { rank, name, seed, state, result, error, trust, time, tokens, harness, restarted },
|
|
286
|
+
* and state is queued, running, done, or error. time is the seconds the judge has run, tokens the count it used
|
|
287
|
+
* (null for none), harness the one its next restart uses, restarted the seconds since a restart (null for none),
|
|
288
|
+
* seed the seed of the art of the criterion.
|
|
289
|
+
* cursor is the index of the row whose detail shows at the right. The header line holds the run: panel and target
|
|
290
|
+
* at the left, the version, the harness, and the elapsed time at the right edge. Two columns from TWO_COLUMNS wide,
|
|
291
|
+
* the list above the detail below that. height is the number of lines the overview may fill: the column divider
|
|
292
|
+
* runs the whole height, the rows and the detail header stay at the top, and the total row and the key
|
|
293
|
+
* line sits at the bottom. Under the detail header a sub tab bar picks the content: sub 'verdict' shows the
|
|
294
|
+
* verdict, or the Cross mark centred in the pane with the time under it while the judge runs; sub 'criterion'
|
|
295
|
+
* shows the description and the body of the criterion as plain text. focus 'text' marks the pane with an orange
|
|
296
|
+
* edge and an orange header, and scroll hides that many lines of the content. Without a height, or when the
|
|
297
|
+
* content is taller than the height, the lines follow each other and nothing scrolls.
|
|
298
|
+
*/
|
|
299
|
+
export function renderOverview({ title, target, harness, elapsed, rows, cursor, width, height, frame = 0, version = VERSION, sub = 'verdict', focus = 'list', scroll = 0 }) {
|
|
208
300
|
const { leftWidth, columnWidth, start } = layout(rows, width);
|
|
209
301
|
const pad = leftWidth - 27;
|
|
210
302
|
const cells = (score, time, tokens) => ` ${score.padStart(3)} ${time.padStart(5)} ${tokens.padStart(6)}`;
|
|
@@ -229,65 +321,94 @@ export function renderOverview({ title, target, harness, elapsed, rows, cursor,
|
|
|
229
321
|
const sum = done.length ? `${quarters(done.reduce((n, r) => n + r.result.score, 0))}/${rows.length}` : `·/${rows.length}`;
|
|
230
322
|
const total = cells(sum, clock(Math.max(0, ...rows.map((r) => r.time ?? 0))), tokensText(rows.some((r) => r.tokens !== null) ? rows.reduce((n, r) => n + (r.tokens ?? 0), 0) : null));
|
|
231
323
|
const totalLine = `${padTo(' total', leftWidth - total.length)}${done.length ? total : total.replace(sum, `${GREY}${sum}${RESET}`)}`;
|
|
232
|
-
const leftBottom = [`${GREY}${'─'.repeat(leftWidth)}${RESET}`, totalLine];
|
|
233
324
|
|
|
234
325
|
const row = rows[cursor];
|
|
235
326
|
const level = row.state === 'done' ? `${row.result.score}/4 ${LEVELS[row.result.score]}` : row.state === 'error' ? 'error' : row.state;
|
|
236
327
|
const name = `${row.rank} ${row.name.toUpperCase()}`;
|
|
237
|
-
const detailHeader = `${focus === '
|
|
238
|
-
const
|
|
239
|
-
const
|
|
240
|
-
const bottom = [`${ORANGE}r${RESET} restart ${ORANGE}h${RESET} harness ${row.harness} ${ORANGE}${row.rank}${RESET} watch
|
|
241
|
-
|
|
242
|
-
//
|
|
243
|
-
const pane = (
|
|
244
|
-
const
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
const detailPane = (lines) => {
|
|
249
|
-
const window = lines === undefined ? 0 : lines - 5;
|
|
250
|
-
if (window < 1) return pane([detailHeader, ...content], bottom, lines, 1);
|
|
328
|
+
const detailHeader = `${focus === 'text' ? `${ORANGE}${BOLD}` : BOLD}${name}${RESET}${' '.repeat(Math.max(1, columnWidth - name.length - level.length))}${GREY}${level}${RESET}`;
|
|
329
|
+
const subTab = (label, key) => `${sub === key ? BOLD : GREY}${label}${RESET}`;
|
|
330
|
+
const subBar = [`${subTab('Verdict', 'verdict')}${GREY} │ ${RESET}${subTab('Criterion', 'criterion')}`, `${GREY}${'─'.repeat(columnWidth)}${RESET}`];
|
|
331
|
+
const bottom = [`${ORANGE}r${RESET} restart ${ORANGE}h${RESET} harness ${row.harness} ${ORANGE}${row.rank}${RESET} watch`];
|
|
332
|
+
// The pane: the header, the sub tab bar and its hairline, a window of the content that the scroll moves, then
|
|
333
|
+
// the key line on the last row.
|
|
334
|
+
const pane = (lines) => {
|
|
335
|
+
const window = lines === undefined ? undefined : lines - PANE_FRAME;
|
|
336
|
+
const content = detailContent(row, sub, columnWidth, start, width, frame, window);
|
|
337
|
+
const top = [detailHeader, ...subBar];
|
|
338
|
+
if (window === undefined || window < 1) return [...top, ...content, ...bottom];
|
|
251
339
|
const from = Math.min(scroll, Math.max(0, content.length - window));
|
|
252
|
-
return [
|
|
340
|
+
return [...top, ...content.slice(from, from + window), ...Array(Math.max(0, window - content.length)).fill(''), ...bottom];
|
|
253
341
|
};
|
|
254
|
-
|
|
342
|
+
return page({
|
|
343
|
+
header: headerLine(title, target, harness, elapsed, width, version),
|
|
344
|
+
width,
|
|
345
|
+
height,
|
|
346
|
+
left: { rows, lines: left },
|
|
347
|
+
leftBottom: [`${GREY}${'─'.repeat(leftWidth)}${RESET}`, totalLine],
|
|
348
|
+
pane,
|
|
349
|
+
edge: focus === 'text' ? `${ORANGE}▌${RESET}` : `${GREY}│${RESET}`,
|
|
350
|
+
});
|
|
351
|
+
}
|
|
255
352
|
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
353
|
+
// The overview is view 0. The other views are judges.
|
|
354
|
+
export const isPage = (view) => view === 0;
|
|
355
|
+
|
|
356
|
+
const HARNESSES = ['claude', 'codex'];
|
|
357
|
+
|
|
358
|
+
/**
|
|
359
|
+
* The modal states: { kind: 'harness', rank, options, cursor } picks the harness of judge rank; { kind: 'restart',
|
|
360
|
+
* rank, options, cursor } asks before a restart. options are the rows, cursor the index of the orange row.
|
|
361
|
+
*/
|
|
362
|
+
export const harnessModal = (rank, current) => ({ kind: 'harness', rank, options: HARNESSES, cursor: Math.max(0, HARNESSES.indexOf(current)) });
|
|
363
|
+
export const restartModal = (rank) => ({ kind: 'restart', rank, options: ['restart', 'cancel'], cursor: 1 });
|
|
364
|
+
|
|
365
|
+
/**
|
|
366
|
+
* The lines of a modal and where they go: { top, left, lines }, centred on a screen width by height. A grey
|
|
367
|
+
* box-drawing frame, a bold title, a grey note when there is one, then one row per option, the cursor row orange
|
|
368
|
+
* with ▸. current is the harness the judge uses now, marked in the harness picker.
|
|
369
|
+
*/
|
|
370
|
+
export function renderModal({ modal, current, width, height }) {
|
|
371
|
+
const title = modal.kind === 'harness' ? `Harness for judge ${modal.rank}` : `Restart judge ${modal.rank}?`;
|
|
372
|
+
const note = modal.kind === 'restart' ? `with ${current} · the current answer is discarded` : null;
|
|
373
|
+
const rows = modal.options.map((o) => (modal.kind === 'harness' && o === current ? `${o} (current)` : o));
|
|
374
|
+
const inner = Math.max(title.length, note?.length ?? 0, ...rows.map((r) => r.length + 2)) + 2;
|
|
375
|
+
const line = (text) => `${GREY}│${RESET} ${padTo(text, inner - 2)} ${GREY}│${RESET}`;
|
|
376
|
+
const lines = [`${GREY}┌${'─'.repeat(inner)}┐${RESET}`, line(`${BOLD}${title}${RESET}`)];
|
|
377
|
+
if (note) lines.push(line(`${GREY}${note}${RESET}`));
|
|
378
|
+
lines.push(line(''));
|
|
379
|
+
rows.forEach((r, i) => lines.push(line(i === modal.cursor ? `${ORANGE}▸ ${r}${RESET}` : ` ${r}`)));
|
|
380
|
+
lines.push(`${GREY}└${'─'.repeat(inner)}┘${RESET}`);
|
|
381
|
+
return { top: Math.max(0, Math.floor((height - lines.length) / 2)), left: Math.max(0, Math.floor((width - inner - 2) / 2)), lines };
|
|
268
382
|
}
|
|
269
383
|
|
|
270
384
|
/**
|
|
271
385
|
* The three lines of the tab strip: a grey hairline, the tabs, then a grey status line with the progress and
|
|
272
|
-
* the elapsed time at the left and the keys of the focus at the right. The
|
|
273
|
-
*
|
|
274
|
-
*
|
|
275
|
-
*
|
|
276
|
-
*
|
|
386
|
+
* the elapsed time at the left and the keys of the focus at the right. The tabs are Overview, then the name of
|
|
387
|
+
* each criterion, with a grey │ between the tabs. A running tab glimmers. The names step down to the first word
|
|
388
|
+
* when they do not fit the width. focus is 'strip' (the active tab is bold orange, the arrows move it), 'text'
|
|
389
|
+
* (the detail pane of the overview scrolls), or 'judge' (the active tab is bold ink, the keys go to the judge).
|
|
390
|
+
* confirm replaces the progress with the quit question; focus 'modal' shows the keys of the open modal. The keys
|
|
391
|
+
* of the overview drop their last words when they do not fit beside the progress.
|
|
277
392
|
*/
|
|
278
393
|
export function renderStrip({ rows, view, frame = 0, width, focus = 'strip', elapsed = 0, confirm = false }) {
|
|
279
394
|
const finished = rows.filter((r) => r.state === 'done' || r.state === 'error').length;
|
|
280
|
-
const
|
|
281
|
-
|
|
282
|
-
|
|
395
|
+
const progress = confirm ? 'judges still run. q again quits, another key stays' : `${finished} of ${rows.length} done · ${clock(elapsed)}`;
|
|
396
|
+
const full = focus === 'judge' ? 'ctrl-q tabs'
|
|
397
|
+
: focus === 'modal' ? '↑ ↓ · enter pick · esc close'
|
|
398
|
+
: focus === 'text' ? '↑ ↓ scroll · enter list · ← → tabs · q q quit'
|
|
399
|
+
: view === 0 ? '↑ ↓ criterion · tab verdict/criterion · enter scroll · r restart · h harness · ← → tabs · q q quit' : '← → tabs · enter back · q quit';
|
|
400
|
+
const parts = full.split(' · ');
|
|
401
|
+
while (parts.length > 1 && progress.length + 2 + parts.join(' · ').length + 2 > width) parts.pop();
|
|
402
|
+
const keys = parts.join(' · ');
|
|
283
403
|
const names = (level) => ['Overview', ...rows.map((r) => (level === 0 ? r.name : r.name.split(' ')[0]))];
|
|
284
404
|
const used = (list) => 1 + list.reduce((n, t) => n + t.length, 0) + 3 * (list.length - 1);
|
|
285
405
|
const list = [0, 1].map(names).find((l) => used(l) <= width) ?? names(1);
|
|
406
|
+
const activeIndex = view;
|
|
286
407
|
let line = ' ';
|
|
287
408
|
let col = 1;
|
|
288
409
|
list.forEach((text, i) => {
|
|
289
410
|
if (i) { line += `${GREY} │ ${RESET}`; col += 3; }
|
|
290
|
-
const active = i ===
|
|
411
|
+
const active = i === activeIndex;
|
|
291
412
|
const row = rows[i - 1];
|
|
292
413
|
const start = col;
|
|
293
414
|
col += text.length;
|
|
@@ -295,9 +416,8 @@ export function renderStrip({ rows, view, frame = 0, width, focus = 'strip', ela
|
|
|
295
416
|
line += glimmer(text, frame, start, width, active && focus === 'strip' ? GLIMMER_ORANGE : GLIMMER, active ? BOLD : '');
|
|
296
417
|
return;
|
|
297
418
|
}
|
|
298
|
-
line += `${active ? (focus === '
|
|
419
|
+
line += `${active ? (focus === 'judge' ? BOLD : `${ORANGE}${BOLD}`) : GREY}${text}${RESET}`;
|
|
299
420
|
});
|
|
300
|
-
const progress = confirm ? 'judges still run. q again quits, another key stays' : `${finished} of ${rows.length} done · ${clock(elapsed)}`;
|
|
301
421
|
// A narrow terminal drops the keys before the progress, and then the end of the progress.
|
|
302
422
|
const gap = width - progress.length - keys.length - 2;
|
|
303
423
|
const status = gap >= 2 ? ` ${progress}${' '.repeat(gap)}${keys}` : ` ${progress}`;
|
|
@@ -305,60 +425,79 @@ export function renderStrip({ rows, view, frame = 0, width, focus = 'strip', ela
|
|
|
305
425
|
}
|
|
306
426
|
|
|
307
427
|
/**
|
|
308
|
-
* The next { view, confirm, focus, cursor, scroll } after one key, plus send: true when the key goes to the
|
|
309
|
-
* quit: true, restart: N to restart judge N, or flip: N to flip the harness of judge N. view is 0 for the
|
|
428
|
+
* The next { view, confirm, focus, cursor, sub, scroll } after one key, plus send: true when the key goes to the
|
|
429
|
+
* judge, quit: true, restart: N to restart judge N, or flip: N to flip the harness of judge N. view is 0 for the
|
|
310
430
|
* overview, or N for judge N. open holds the ranks of the judges that have started, count the number of
|
|
311
431
|
* criteria, maxScroll the lines the detail pane can hide, page the lines one PgUp or PgDn moves. With the
|
|
312
432
|
* focus on a judge, every key but ctrl-q goes to the judge, and ctrl-q moves the focus to the strip. With the
|
|
313
433
|
* focus on the strip, the arrows left and right move the active tab and switch the view, 1-9 jump to a started
|
|
314
|
-
* judge, 0 to the overview,
|
|
315
|
-
*
|
|
316
|
-
* the judge of the cursor row, h
|
|
317
|
-
* and down scroll one line, PgUp and PgDn a page, and Esc or Enter return the focus
|
|
434
|
+
* judge, 0 to the overview, and q quits (a second q confirms while a judge runs). On the overview, which takes
|
|
435
|
+
* no text, up and down move the cursor, Tab switches the sub tab of the detail pane (verdict or criterion),
|
|
436
|
+
* r opens the restart modal of the judge of the cursor row, h the harness picker, and Enter moves the focus into
|
|
437
|
+
* the pane: there up and down scroll one line, PgUp and PgDn a page, and Esc or Enter return the focus. A modal
|
|
438
|
+
* (state.modal) takes every key until Enter or Esc. On a judge tab, Enter, Esc, or ctrl-q give the focus to the
|
|
439
|
+
* judge. The result carries harness: { rank, value } when the picker picked one.
|
|
318
440
|
*/
|
|
319
|
-
export function press({ view, confirm, focus, cursor, scroll = 0 }, key, { open, running, count, maxScroll: limit = 0, page = 10 }) {
|
|
320
|
-
const strip = (v) => ({ view: v, confirm: false, focus: 'strip', cursor, scroll });
|
|
321
|
-
if (
|
|
441
|
+
export function press({ view, confirm, focus, cursor, sub = 'verdict', scroll = 0, modal = null }, key, { open, running, count, maxScroll: limit = 0, page: step = 10, harnesses = [] }) {
|
|
442
|
+
const strip = (v) => ({ view: v, confirm: false, focus: 'strip', cursor, sub, scroll, modal: null });
|
|
443
|
+
if (modal) {
|
|
444
|
+
// The modal takes every key: up and down move its cursor, Enter picks the row, Esc closes it, the rest is ignored.
|
|
445
|
+
const same = { view, confirm, focus, cursor, sub, scroll, modal };
|
|
446
|
+
if (UP.includes(key)) return { ...same, modal: { ...modal, cursor: Math.max(0, modal.cursor - 1) } };
|
|
447
|
+
if (DOWN.includes(key)) return { ...same, modal: { ...modal, cursor: Math.min(modal.options.length - 1, modal.cursor + 1) } };
|
|
448
|
+
if (key === '\x1b') return strip(0);
|
|
449
|
+
if (key === '\r') {
|
|
450
|
+
const pick = modal.options[modal.cursor];
|
|
451
|
+
if (modal.kind === 'harness') return { ...strip(0), harness: { rank: modal.rank, value: pick } };
|
|
452
|
+
return pick === 'restart' ? { ...strip(0), restart: modal.rank } : strip(0);
|
|
453
|
+
}
|
|
454
|
+
return same;
|
|
455
|
+
}
|
|
456
|
+
if (focus === 'judge') return TOGGLE.includes(key) ? strip(view) : { view, confirm, focus, cursor, sub, scroll, modal: null, send: true };
|
|
322
457
|
if (confirm) return { ...strip(0), quit: key === 'q' };
|
|
323
|
-
if (key === 'q') return running ? { view: 0, confirm: true, focus: 'strip', cursor, scroll } : { ...strip(0), quit: true };
|
|
458
|
+
if (key === 'q') return running ? { view: 0, confirm: true, focus: 'strip', cursor, sub, scroll, modal: null } : { ...strip(0), quit: true };
|
|
324
459
|
if (RIGHT.includes(key) || LEFT.includes(key)) {
|
|
325
460
|
const cycle = [0, ...open];
|
|
326
|
-
const
|
|
327
|
-
return strip(cycle[(cycle.indexOf(view) +
|
|
461
|
+
const dir = RIGHT.includes(key) ? 1 : -1;
|
|
462
|
+
return strip(cycle[(cycle.indexOf(view) + dir + cycle.length) % cycle.length]);
|
|
328
463
|
}
|
|
329
464
|
if (/^[1-9]$/.test(key)) return open.includes(Number(key)) ? strip(Number(key)) : { ...strip(view), focus };
|
|
330
465
|
if (key === '0') return strip(0);
|
|
331
466
|
if (view === 0) {
|
|
332
|
-
const
|
|
333
|
-
if (key === 'r') return open.includes(cursor + 1) ? { ...
|
|
334
|
-
if (key === 'h') return { ...
|
|
335
|
-
if (
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
if (key
|
|
339
|
-
if (key
|
|
467
|
+
const keep = { ...strip(0), focus };
|
|
468
|
+
if (key === 'r') return open.includes(cursor + 1) ? { ...keep, focus: 'modal', modal: restartModal(cursor + 1) } : keep;
|
|
469
|
+
if (key === 'h') return { ...keep, focus: 'modal', modal: harnessModal(cursor + 1, harnesses[cursor]) };
|
|
470
|
+
if (key === '\t') return { ...keep, sub: sub === 'verdict' ? 'criterion' : 'verdict', scroll: 0 };
|
|
471
|
+
if (focus === 'text') {
|
|
472
|
+
const text = (s) => ({ ...keep, scroll: Math.max(0, Math.min(limit, s)) });
|
|
473
|
+
if (UP.includes(key)) return text(scroll - 1);
|
|
474
|
+
if (DOWN.includes(key)) return text(scroll + 1);
|
|
475
|
+
if (key === '\x1b[5~') return text(scroll - step);
|
|
476
|
+
if (key === '\x1b[6~') return text(scroll + step);
|
|
340
477
|
if (key === '\r' || key === '\x1b') return strip(0);
|
|
341
|
-
return
|
|
478
|
+
return keep;
|
|
342
479
|
}
|
|
343
480
|
if (UP.includes(key)) return { ...strip(0), cursor: Math.max(0, cursor - 1), scroll: 0 };
|
|
344
481
|
if (DOWN.includes(key)) return { ...strip(0), cursor: Math.min(count - 1, cursor + 1), scroll: 0 };
|
|
345
|
-
if (key === '\r') return
|
|
482
|
+
if (key === '\r') return { ...strip(0), focus: 'text', scroll: 0 };
|
|
346
483
|
return strip(0);
|
|
347
484
|
}
|
|
348
|
-
if (BACK.includes(key)) return { view, confirm: false, focus: 'judge', cursor, scroll };
|
|
485
|
+
if (BACK.includes(key)) return { view, confirm: false, focus: 'judge', cursor, sub, scroll, modal: null };
|
|
349
486
|
return strip(view);
|
|
350
487
|
}
|
|
351
488
|
|
|
352
489
|
/** Runs the judges, shows the overview, and writes the report. Resolves when the engineer quits. */
|
|
353
|
-
export function runTable({ panel, runDir, harness, cwd, target,
|
|
490
|
+
export function runTable({ panel, runDir, harness, cwd, target, yolo = false }) {
|
|
354
491
|
const out = process.stdout;
|
|
355
|
-
|
|
492
|
+
// One random seed per run; each criterion keeps its own art for the whole run, through restarts.
|
|
493
|
+
const runSeed = Math.floor(Math.random() * 2 ** 31);
|
|
494
|
+
const criteria = panel.criteria.map((c, i) => ({ rank: i + 1, seed: runSeed + i + 1, ...c }));
|
|
356
495
|
const judges = [];
|
|
357
496
|
// The harness of each judge: the run harness, until h flips it.
|
|
358
497
|
const harnesses = criteria.map(() => harness);
|
|
359
498
|
let rows = [];
|
|
360
499
|
let final = null;
|
|
361
|
-
let ui = { view: 0, confirm: false, focus: 'strip', cursor: 0, scroll: 0 };
|
|
500
|
+
let ui = { view: 0, confirm: false, focus: 'strip', cursor: 0, sub: 'verdict', scroll: 0, modal: null };
|
|
362
501
|
let stripTimer;
|
|
363
502
|
let lastData = 0;
|
|
364
503
|
const started = Date.now();
|
|
@@ -395,7 +534,7 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
395
534
|
}
|
|
396
535
|
|
|
397
536
|
function start(rank, restartedAt = null) {
|
|
398
|
-
const [command, ...argv] = writeJudge({ dir: judgeDir(rank), criterion: criteria[rank - 1], harness: harnesses[rank - 1], cwd, target,
|
|
537
|
+
const [command, ...argv] = writeJudge({ dir: judgeDir(rank), criterion: criteria[rank - 1], harness: harnesses[rank - 1], cwd, target, yolo });
|
|
399
538
|
const { cols, rows: lines } = size();
|
|
400
539
|
const term = new xterm.Terminal({ cols, rows: lines, allowProposedApi: true });
|
|
401
540
|
const serializer = new SerializeAddon();
|
|
@@ -441,7 +580,7 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
441
580
|
const lines = renderStrip({ rows, view: ui.view, frame, width: out.columns, focus: ui.focus, elapsed: seconds(started), confirm: ui.confirm });
|
|
442
581
|
const top = out.rows - STRIP_ROWS + 1;
|
|
443
582
|
const strip = lines.map((l, i) => `\x1b[${top + i};1H${l}\x1b[K`).join('');
|
|
444
|
-
if (
|
|
583
|
+
if (isPage(ui.view)) return void out.write(`${RESET}${strip}`);
|
|
445
584
|
const j = judges[ui.view - 1];
|
|
446
585
|
const { cursorX, cursorY } = j.term.buffer.active;
|
|
447
586
|
if (cursorX >= out.columns) return;
|
|
@@ -475,9 +614,14 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
475
614
|
|
|
476
615
|
let frame = 0;
|
|
477
616
|
function draw() {
|
|
478
|
-
const lines = renderOverview({ title: `${panel.owner}/${panel.name} v${panel.version}`, target, harness, elapsed: seconds(started), rows, cursor: ui.cursor, width: out.columns, height: out.rows - STRIP_ROWS, frame, focus: ui.focus === '
|
|
617
|
+
const lines = renderOverview({ title: `${panel.owner}/${panel.name} v${panel.version}`, target, harness, elapsed: seconds(started), rows, cursor: ui.cursor, width: out.columns, height: out.rows - STRIP_ROWS, frame, sub: ui.sub, focus: ui.focus === 'text' ? 'text' : 'list', scroll: ui.scroll });
|
|
479
618
|
while (lines.length < out.rows - STRIP_ROWS) lines.push('');
|
|
480
619
|
out.write(lines.slice(0, out.rows - STRIP_ROWS).map((l, i) => `\x1b[${i + 1};1H${l}\x1b[K`).join(''));
|
|
620
|
+
// The modal sits over the overview; the next draw without it shows the overview again.
|
|
621
|
+
if (ui.modal) {
|
|
622
|
+
const { top, left, lines: box } = renderModal({ modal: ui.modal, current: harnesses[ui.modal.rank - 1], width: out.columns, height: out.rows - STRIP_ROWS });
|
|
623
|
+
out.write(box.map((l, i) => `\x1b[${top + i + 1};${left + 1}H${l}`).join(''));
|
|
624
|
+
}
|
|
481
625
|
drawStrip();
|
|
482
626
|
}
|
|
483
627
|
|
|
@@ -494,14 +638,14 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
494
638
|
}
|
|
495
639
|
if (rows.every((r) => r.state === 'done' || r.state === 'error')) finish();
|
|
496
640
|
}
|
|
497
|
-
if (
|
|
641
|
+
if (isPage(ui.view)) draw();
|
|
498
642
|
}
|
|
499
643
|
|
|
500
644
|
return new Promise((done) => {
|
|
501
645
|
const timer = setInterval(tick, 1000);
|
|
502
646
|
const spinner = setInterval(() => {
|
|
503
647
|
frame += 1;
|
|
504
|
-
if (
|
|
648
|
+
if (isPage(ui.view)) draw();
|
|
505
649
|
else if (Date.now() - lastData > 60) drawStrip();
|
|
506
650
|
}, 100);
|
|
507
651
|
out.write('\x1b[?1049h\x1b[?25l');
|
|
@@ -510,9 +654,9 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
510
654
|
const key = data.toString();
|
|
511
655
|
const from = ui.view;
|
|
512
656
|
const height = out.rows - STRIP_ROWS;
|
|
513
|
-
const limit = rows.length ?
|
|
514
|
-
const next = press(ui, key, { open: judges.flatMap((j, i) => (j ? [i + 1] : [])), running: rows.some((r) => r.state === 'running'), count: criteria.length, maxScroll: limit, page: Math.max(1, height -
|
|
515
|
-
ui = { view: next.view, confirm: next.confirm, focus: next.focus, cursor: next.cursor, scroll: next.scroll };
|
|
657
|
+
const limit = rows.length ? overviewScrollMax({ rows, cursor: ui.cursor, sub: ui.sub, width: out.columns, height }) : 0;
|
|
658
|
+
const next = press(ui, key, { open: judges.flatMap((j, i) => (j ? [i + 1] : [])), running: rows.some((r) => r.state === 'running'), count: criteria.length, maxScroll: limit, page: Math.max(1, height - PANE_FRAME), harnesses });
|
|
659
|
+
ui = { view: next.view, confirm: next.confirm, focus: next.focus, cursor: next.cursor, sub: next.sub, scroll: next.scroll, modal: next.modal };
|
|
516
660
|
if (next.send) {
|
|
517
661
|
judges[from - 1].proc.write(key);
|
|
518
662
|
} else if (next.quit) {
|
|
@@ -526,16 +670,16 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
526
670
|
} else if (next.restart) {
|
|
527
671
|
restart(next.restart);
|
|
528
672
|
tick();
|
|
529
|
-
} else if (next.
|
|
530
|
-
harnesses[next.
|
|
673
|
+
} else if (next.harness) {
|
|
674
|
+
harnesses[next.harness.rank - 1] = next.harness.value;
|
|
531
675
|
tick();
|
|
532
|
-
} else if (ui.view && ui.view === from) {
|
|
676
|
+
} else if (!isPage(ui.view) && ui.view === from) {
|
|
533
677
|
drawStrip();
|
|
534
678
|
} else {
|
|
535
679
|
// A judge can leave its modes on (mouse, bracketed paste, the kitty keyboard). The next view starts clean.
|
|
536
|
-
if (from) out.write(TABLE_MODES);
|
|
537
|
-
if (ui.view)
|
|
538
|
-
else
|
|
680
|
+
if (!isPage(from)) out.write(TABLE_MODES);
|
|
681
|
+
if (isPage(ui.view)) draw();
|
|
682
|
+
else showJudge(ui.view);
|
|
539
683
|
}
|
|
540
684
|
});
|
|
541
685
|
out.on('resize', () => {
|
|
@@ -544,8 +688,8 @@ export function runTable({ panel, runDir, harness, cwd, target, brief, sources,
|
|
|
544
688
|
if (j && !j.exited) j.proc.resize(cols, lines);
|
|
545
689
|
j?.term.resize(cols, lines);
|
|
546
690
|
}
|
|
547
|
-
if (ui.view)
|
|
548
|
-
else
|
|
691
|
+
if (isPage(ui.view)) draw();
|
|
692
|
+
else showJudge(ui.view);
|
|
549
693
|
});
|
|
550
694
|
tick();
|
|
551
695
|
});
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@xsft/panels",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.8.1",
|
|
4
4
|
"private": false,
|
|
5
5
|
"description": "Run a panel of AI judges from cross.software/panels against a target.",
|
|
6
6
|
"type": "module",
|
|
@@ -24,7 +24,6 @@
|
|
|
24
24
|
"dependencies": {
|
|
25
25
|
"@homebridge/node-pty-prebuilt-multiarch": "0.14.1",
|
|
26
26
|
"@xterm/addon-serialize": "0.14.0",
|
|
27
|
-
"@xterm/headless": "6.0.0"
|
|
28
|
-
"figlet": "1.12.0"
|
|
27
|
+
"@xterm/headless": "6.0.0"
|
|
29
28
|
}
|
|
30
29
|
}
|
|
@@ -3,44 +3,42 @@ You are one judge in a run of a panel. A panel is a named list of criteria. You
|
|
|
3
3
|
YOUR INPUTS
|
|
4
4
|
|
|
5
5
|
The user message gives you:
|
|
6
|
-
- Your criterion: its rank, its name, its one-line description, and its body.
|
|
6
|
+
- Your criterion: its rank, its name, its one-line description, and its body. They define what to assess, not how. This prompt holds the method.
|
|
7
7
|
- The target: the text that the engineer typed after the panel name, unchanged. It can be a URL, a path, a branch, a pull request, or a sentence.
|
|
8
|
-
- The brief: a file path, or "none supplied".
|
|
9
|
-
- The sources: paths or URLs, or "none".
|
|
10
8
|
|
|
11
9
|
Your working directory is the folder where the engineer ran the command.
|
|
12
10
|
|
|
13
|
-
|
|
11
|
+
HOW TO JUDGE
|
|
14
12
|
|
|
15
|
-
|
|
13
|
+
- Look at the target itself: open it, read it, and run it when it is code. Reach it with your tools: read files, run commands, open URLs.
|
|
14
|
+
- Collect evidence for every claim you make: a URL, a file and line, a quote, or a command and its output. The engineer must be able to check each one.
|
|
15
|
+
- Give no credit for intent. A comment or a doc is not done work.
|
|
16
|
+
- When you cannot reach the target, or cannot check a part of the criterion with your tools, say so at the end of your verdict after "Not checked:" with the reason. Never score what you did not see.
|
|
17
|
+
- Keep any notes or tables for yourself; they do not go in the answer.
|
|
16
18
|
|
|
17
|
-
|
|
19
|
+
SCORE
|
|
18
20
|
|
|
19
|
-
|
|
21
|
+
The score is a whole number from 1 to 4. Only 4 passes.
|
|
20
22
|
|
|
21
|
-
|
|
23
|
+
4 Met. The criterion holds across the whole target, every claim has evidence, and a real look found no material flaw.
|
|
24
|
+
3 Almost met. The criterion holds in the main. Material flaws remain, and each one is named with its evidence.
|
|
25
|
+
2 Falls short. Parts hold, but the flaws outweigh them, or a core part is missing.
|
|
26
|
+
1 Fails. The criterion does not hold, or the target cannot be assessed for it, or a claim is invented.
|
|
22
27
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
3. When the evidence fits two levels, use the lower level.
|
|
26
|
-
4. A criterion passes only at score 4.
|
|
27
|
-
|
|
28
|
-
EVIDENCE
|
|
29
|
-
|
|
30
|
-
- Each claim needs evidence that the engineer can check: a URL, a file and line, a quote, or a command and its output. Put the place in brackets inside the verdict, such as (index.html:90).
|
|
31
|
-
- Give no credit for intent. A promise in a comment, a doc, or the brief is not done work.
|
|
28
|
+
Rules:
|
|
29
|
+
- When the evidence fits two levels, use the lower level.
|
|
32
30
|
- No evidence for a level means the lower level.
|
|
31
|
+
- Anything you did not check cannot support a 4. List it at the end of the verdict after "Not checked:".
|
|
33
32
|
|
|
34
33
|
THE VERDICT
|
|
35
34
|
|
|
36
|
-
One paragraph: what holds, then what is wrong and where, each place in brackets, then "Not checked:" and the
|
|
35
|
+
One paragraph: what holds, then what is wrong and where, each place in brackets such as (index.html:90), then "Not checked:" and the parts you could not check, when there are any. Judge; do not propose fixes. The fixes are the job of the engineer.
|
|
37
36
|
|
|
38
37
|
PROHIBITIONS
|
|
39
38
|
|
|
40
|
-
- Do not change the
|
|
41
|
-
- Do not score a fault that
|
|
42
|
-
- Do not write or change files
|
|
43
|
-
- Do not start sub-agents.
|
|
39
|
+
- Do not change the criterion or the levels.
|
|
40
|
+
- Do not score a fault that another criterion of the panel covers.
|
|
41
|
+
- Do not write or change files of the target. The answer file, when the user message names one, is the only file you write.
|
|
44
42
|
- Do not submit forms, send messages, or post data.
|
|
45
43
|
- Do not read the .panels folder. It holds earlier reports.
|
|
46
44
|
|
|
@@ -59,5 +57,5 @@ Example:
|
|
|
59
57
|
{
|
|
60
58
|
"name": "Original design assets",
|
|
61
59
|
"score": 2,
|
|
62
|
-
"verdict": "The hero shows the real app, and the cutaway hotel and the team scenes are original and share one idea (public/assets/hotel-world). Every content icon is a generic Lucide symbol (src/components/Features.tsx:3),
|
|
60
|
+
"verdict": "The hero shows the real app, and the cutaway hotel and the team scenes are original and share one idea (public/assets/hotel-world). Every content icon is a generic Lucide symbol (src/components/Features.tsx:3), so the icon set is not original. Not checked: a reverse image search of hero.webp, the target is on localhost."
|
|
63
61
|
}
|