kodelyth-ecc 2.8.0 → 2.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +103 -0
- package/CLAUDE.md +7 -6
- package/README.md +4 -2
- package/VERSION +1 -1
- package/bin/kodelyth-ecc.js +87 -1
- package/commands/arena.md +35 -0
- package/package.json +1 -1
- package/scripts/arena/arena.js +9 -1
- package/scripts/arena/evil.js +7 -4
- package/scripts/arena/learn.js +273 -0
- package/scripts/dashboard/data.js +111 -0
- package/scripts/dashboard/server.js +4 -0
- package/scripts/dashboard/static/index.html +126 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,109 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to Kodelyth ECC are documented here.
|
|
4
4
|
|
|
5
|
+
## v2.10.0 — Arena dashboard tab + docs (phases 5 & 6) (August 2026)
|
|
6
|
+
|
|
7
|
+
### Added — Arena tab in the dashboard
|
|
8
|
+
|
|
9
|
+
The tab answers one question: **did the attacker give up?**
|
|
10
|
+
|
|
11
|
+
- **Convergence trend** per run, drawn as a block-character sparkline — no chart
|
|
12
|
+
library, no CDN, legible at one round or twenty. Rounds that surface nothing
|
|
13
|
+
render green; a round worse than the last renders red.
|
|
14
|
+
- **Still open** — confirmed findings GOD has *not* answered for, ranked by real
|
|
15
|
+
risk (`severity × confidence × exploitability`).
|
|
16
|
+
- **Recurring bug classes** — the same class twice is flagged `recurring`, because
|
|
17
|
+
one is an incident and several is a process gap.
|
|
18
|
+
- Runs that began with recalled memories are marked `recalled`.
|
|
19
|
+
|
|
20
|
+
Read-only. The dashboard never writes to your memory store.
|
|
21
|
+
|
|
22
|
+
`GET /api/arena[?limit=N]` → `{ available, runs, open, classes, totals }`. Raw
|
|
23
|
+
findings are stripped from the wire payload — the page needs counts and the open
|
|
24
|
+
list, not every finding on every run.
|
|
25
|
+
|
|
26
|
+
### Fixed — a confirmed finding is not the same as an open one
|
|
27
|
+
|
|
28
|
+
`closeRound` recorded *how many* findings were left outstanding but not *which
|
|
29
|
+
ones*, so nothing downstream could tell a confirmed-and-fixed bug from a
|
|
30
|
+
confirmed-and-ignored one. The dashboard's first draft reported all ten fixed
|
|
31
|
+
findings as open risk — a healthy run reading as alarming, which is exactly
|
|
32
|
+
backwards. Rounds now persist `addressedIds`.
|
|
33
|
+
|
|
34
|
+
### Added — `docs/arena.md`
|
|
35
|
+
|
|
36
|
+
A full feature doc: why a loop beats a review pass, what makes a finding count,
|
|
37
|
+
how convergence is decided, the compound-learning return path, guard proposals,
|
|
38
|
+
and cost control. Wired into the sitemap, the docs index, and `dashboard.md`.
|
|
39
|
+
|
|
40
|
+
### Fixed — stale counts
|
|
41
|
+
|
|
42
|
+
The README and `CLAUDE.md` advertised 194 skills and 97 commands; the real
|
|
43
|
+
figures are 196 and 102. Both now match what is on disk.
|
|
44
|
+
|
|
45
|
+
**525 tests passing**, up from 516.
|
|
46
|
+
|
|
47
|
+
## v2.9.0 — Compound learning: the arena now remembers (phase 4) (August 2026)
|
|
48
|
+
|
|
49
|
+
A finished arena run used to be knowledge thrown away. Every new run started from
|
|
50
|
+
zero and EVIL re-derived the same bug classes forever. Phase 4 closes the loop.
|
|
51
|
+
|
|
52
|
+
```
|
|
53
|
+
arena run ──▶ confirmed + refuted findings ──▶ memories
|
|
54
|
+
▲ │
|
|
55
|
+
└──────── prior-knowledge brief ◀──────────────┘
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Added — `scripts/arena/learn.js`
|
|
59
|
+
|
|
60
|
+
Pure functions, no I/O — the callers own the disk and the confirmation prompt, so
|
|
61
|
+
the learning logic is testable without writing to a real memory store.
|
|
62
|
+
|
|
63
|
+
- **Confirmed findings become memories** carrying the fix and the repro that proved
|
|
64
|
+
it, tagged by bug class, file, language, and scope.
|
|
65
|
+
- **Refuted findings become memories too** — the more valuable half. Without them
|
|
66
|
+
the next run re-investigates the same non-bug and burns a real verification pass
|
|
67
|
+
proving the same negative.
|
|
68
|
+
- **Unverified findings are deliberately skipped.** Storing a question as knowledge
|
|
69
|
+
would launder a guess into a fact, and future runs would recall it as settled.
|
|
70
|
+
- **Bug classification** — a small, reviewable keyword map (not a classifier) that
|
|
71
|
+
groups findings well enough to see a class recurring across runs.
|
|
72
|
+
|
|
73
|
+
### Added — the return path
|
|
74
|
+
|
|
75
|
+
`arena start` now recalls arena-sourced memories for the scope and injects them
|
|
76
|
+
into every round-1 EVIL brief, split into *"confirmed here before, verify these
|
|
77
|
+
stayed fixed"* and *"already refuted, do not re-report without new evidence."*
|
|
78
|
+
The run reports how many memories it recalled, because silent magic is
|
|
79
|
+
untrustworthy. `--fresh` skips it.
|
|
80
|
+
|
|
81
|
+
### Added — `arena learn` and guard proposals
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
kodelythecc arena learn <run-id> # show what would be remembered
|
|
85
|
+
kodelythecc arena learn <run-id> --commit # store it
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Nothing is written without `--commit`. Memory that writes itself silently is
|
|
89
|
+
memory you cannot trust.
|
|
90
|
+
|
|
91
|
+
When one bug **class** is confirmed repeatedly, `--commit` also files an
|
|
92
|
+
`arena-guard` proposal into evolve. One symlink bug is an incident; two in the
|
|
93
|
+
same file is a process gap — and the proposal says so, recommending a shared
|
|
94
|
+
path-safety helper rather than patching the third one later. Proposal ids are
|
|
95
|
+
deterministic, so re-running analysis never spawns duplicates.
|
|
96
|
+
|
|
97
|
+
### Fixed — three bugs in the classifier, found by running it on real data
|
|
98
|
+
|
|
99
|
+
- A bare `permission` matched *"widens permission thresholds"*, filing a semantics
|
|
100
|
+
bug under file modes.
|
|
101
|
+
- `/\bvalidat/` could not match inside *"unvalidated"* — there is no word boundary
|
|
102
|
+
after `un`.
|
|
103
|
+
- The injection class keyed on the word *"injection"*, so a finding phrased as a
|
|
104
|
+
missing trust boundary fell through to uncategorized.
|
|
105
|
+
|
|
106
|
+
**516 tests passing**, up from 493.
|
|
107
|
+
|
|
5
108
|
## v2.8.0 — The Arena's first real run: 12 bugs found and fixed in `terse` (August 2026)
|
|
6
109
|
|
|
7
110
|
The arena was pointed at `scripts/terse` — the markdown compressor that ships with
|
package/CLAUDE.md
CHANGED
|
@@ -7,8 +7,8 @@ Guidance for Claude Code when working with this repository.
|
|
|
7
7
|
**Kodelyth ECC** — a production-grade AI coding toolkit:
|
|
8
8
|
|
|
9
9
|
- **70 specialist agents** — debug-detective, incident-commander, load-tester, image-architect, kodelyth-memory, security-reviewer, plus 8 adversarial devil-mode agents
|
|
10
|
-
- **
|
|
11
|
-
- **
|
|
10
|
+
- **196 skills** — domain knowledge, patterns, testing, security, intent routing, local memory, swarm orchestration, MCP integration
|
|
11
|
+
- **102 commands** — slash workflows (`/tdd`, `/plan`, `/code-review`, `/team-review`, `/devil-mode`, `/debug-blitz`, `/security-audit`, ...)
|
|
12
12
|
- **22+ hooks** — quality gates, memory inject + capture, correction encoding, prompt-injection guard, token-budget enforcer
|
|
13
13
|
- **14 rules** — always-on coding standards + semantic intent routing + memory protocol + self-improvement
|
|
14
14
|
|
|
@@ -18,15 +18,15 @@ Works with Claude Code, Windsurf, Cursor, Codex CLI, Antigravity, OpenCode, Clin
|
|
|
18
18
|
|
|
19
19
|
```
|
|
20
20
|
agents/ → 70 specialist subagents (planner, code-reviewer, debug-detective, devil-mode crew, ...)
|
|
21
|
-
commands/ →
|
|
22
|
-
skills/ →
|
|
21
|
+
commands/ → 102 slash commands (8 parallel multi-agent, 1 adversarial loop, rest single-agent)
|
|
22
|
+
skills/ → 196 workflow + domain knowledge files (loadable via slash commands)
|
|
23
23
|
hooks/ → 22+ automations (pre-commit, session memory, prompt-injection guard, token-budget)
|
|
24
24
|
rules/ → 14 always-on guidelines (agent-intent-routing, self-improvement, memory-protocol, ...)
|
|
25
25
|
scripts/ → Node.js utilities: MCP server, dashboard, swarm, replay, router, memory, supply-chain
|
|
26
26
|
bundles/ → 3 power bundles (indie-hacker, red-team, enterprise)
|
|
27
27
|
actions/ → GitHub Action (CI/CD integration for PR review)
|
|
28
|
-
docs/ → Feature docs (mcp.md, dashboard.md, swarm.md, replay.md, evolve.md, supply-chain.md)
|
|
29
|
-
tests/ →
|
|
28
|
+
docs/ → Feature docs (arena.md, mcp.md, dashboard.md, swarm.md, replay.md, evolve.md, supply-chain.md)
|
|
29
|
+
tests/ → 525 passing tests across 29 test files
|
|
30
30
|
```
|
|
31
31
|
|
|
32
32
|
## Running Tests
|
|
@@ -80,6 +80,7 @@ These fire multiple specialist agents simultaneously:
|
|
|
80
80
|
| `/pre-release` | release-captain + security-reviewer + code-reviewer | Go/no-go verdict before shipping |
|
|
81
81
|
| `/onboard` | code-explorer + architect + doc-updater | Understand any codebase in 15 minutes |
|
|
82
82
|
| `/devil-mode` | prompt-injection-hunter + supply-chain-auditor + secret-hunter + backdoor-hunter | Adversarial sweep (use `--all` for all 8) |
|
|
83
|
+
| `/arena` | GOD crew vs EVIL crew, looped | Ship something that must not break — runs until the attacker gives up |
|
|
83
84
|
|
|
84
85
|
## Key Commands
|
|
85
86
|
|
package/README.md
CHANGED
|
@@ -33,7 +33,7 @@
|
|
|
33
33
|
|
|
34
34
|
</div>
|
|
35
35
|
|
|
36
|
-
**Kodelyth ECC** is a production-grade AI coding toolkit — **70 specialist agents (incl. an 8-agent devil-mode adversarial crew),
|
|
36
|
+
**Kodelyth ECC** is a production-grade AI coding toolkit — **70 specialist agents (incl. an 8-agent devil-mode adversarial crew), 196 skills, 102 commands**, a god-tier **semantic intent-routing system**, local self-learning memory, MCP server, swarm orchestrator, and an observability dashboard — all local, zero telemetry.
|
|
37
37
|
|
|
38
38
|
Now bundled with:
|
|
39
39
|
|
|
@@ -73,7 +73,7 @@ You never typed `use debug-detective`. You didn't have to. The toolkit read the
|
|
|
73
73
|
| **Intent routing** | Plain-language → right specialist via 10-tier priority rules | Mostly missing — you memorize names |
|
|
74
74
|
| **70 agents** | Specialists with playbooks, severity calibration, real commands | Often persona-only ("you are a senior engineer...") |
|
|
75
75
|
| **194 skills** | Domain knowledge files agents read on demand | Rarely separated from agents |
|
|
76
|
-
| **
|
|
76
|
+
| **102 commands** | Slash workflows (`/tdd`, `/arena`, `/devil-mode`, `/team-review`) | Limited or none |
|
|
77
77
|
| **8 parallel commands** | Fire 3-8 agents simultaneously, aggregate results | Rare |
|
|
78
78
|
| **Compound memory** | BM25 local recall + auto-inject + project lessons | Cloud-only or absent |
|
|
79
79
|
| **22+ hooks** | Quality gates, secret scan, project-DNA detection | Often missing |
|
|
@@ -92,6 +92,7 @@ You never typed `use debug-detective`. You didn't have to. The toolkit read the
|
|
|
92
92
|
| **Local BM25 self-learning memory** | ✅ | ❌ | ❌ | ❌ |
|
|
93
93
|
| **Compound learning from corrections** | ✅ `tasks/lessons.md` | ❌ | ❌ | ❌ |
|
|
94
94
|
| **Adversarial / red-team agents** | ✅ 8 (devil-mode) | ❌ | ❌ | ❌ |
|
|
95
|
+
| **Adversarial build/attack loop** | ✅ `/arena` — scored, verified, converges | ❌ | ❌ | ❌ |
|
|
95
96
|
| Quality hooks | ✅ 22+ | Some | ❌ | ❌ |
|
|
96
97
|
| IDE platforms | **11** (Claude Code, Windsurf, Cursor, Codex, Antigravity, OpenCode, Cline, Roo Code, Aider, Kimi, Gemini CLI) | 1-2 | 1 | Varies |
|
|
97
98
|
| Telemetry | ❌ none | Varies | ❌ | Varies |
|
|
@@ -372,6 +373,7 @@ Eight commands fire multiple specialist agents simultaneously and aggregate thei
|
|
|
372
373
|
| `/pre-release` | release-captain + security-reviewer + code-reviewer | 30 min → 8 min |
|
|
373
374
|
| `/onboard` | code-explorer + architect + doc-updater | 45 min → 12 min |
|
|
374
375
|
| `/devil-mode` | 8 adversarial agents (see below) | Hours → 20 min |
|
|
376
|
+
| `/arena` | GOD crew vs EVIL crew, looped until the attacker gives up | Days → 1 session |
|
|
375
377
|
|
|
376
378
|
Each command waits for all agents to complete, then returns a single **Team Review Report** with findings bucketed by severity: CRITICAL → HIGH → MEDIUM → LOW.
|
|
377
379
|
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
2.
|
|
1
|
+
2.10.0
|
package/bin/kodelyth-ecc.js
CHANGED
|
@@ -429,6 +429,7 @@ if (args[0] === 'memory') {
|
|
|
429
429
|
// kodelythecc god --task "<task>" [--json] plan a GOD-mode build
|
|
430
430
|
// kodelythecc evil [scope] [--all|--license|...] plan an EVIL-mode sweep
|
|
431
431
|
// kodelythecc arena list list past arena runs
|
|
432
|
+
// kodelythecc arena learn <run-id> [--commit] remember what the run proved
|
|
432
433
|
// kodelythecc arena report <run-id> show a run's report
|
|
433
434
|
if (args[0] === 'god' || args[0] === 'evil' || args[0] === 'arena') {
|
|
434
435
|
const mode = args[0];
|
|
@@ -491,11 +492,27 @@ if (args[0] === 'god' || args[0] === 'evil' || args[0] === 'arena') {
|
|
|
491
492
|
const limits = {};
|
|
492
493
|
if (flag('max-rounds')) limits.maxRounds = Number(flag('max-rounds'));
|
|
493
494
|
if (flag('budget')) limits.tokenBudget = Number(flag('budget'));
|
|
495
|
+
// Compound learning: pull what past runs proved about this scope and hand
|
|
496
|
+
// it to EVIL, so round 1 opens where the last run closed. --fresh skips it.
|
|
497
|
+
const scopeArg = flag('scope', '.');
|
|
498
|
+
let priorKnowledge = '';
|
|
499
|
+
let recalledCount = 0;
|
|
500
|
+
if (!rest.includes('--fresh')) {
|
|
501
|
+
try {
|
|
502
|
+
const learn = require(path.join(ROOT, 'scripts', 'arena', 'learn.js'));
|
|
503
|
+
const memStore = require(path.join(ROOT, 'scripts', 'memory', 'store.js'));
|
|
504
|
+
const hits = memStore.recall(`arena ${scopeArg} ${task}`, { limit: 20 })
|
|
505
|
+
.filter(m => (m.source || '') === 'arena');
|
|
506
|
+
recalledCount = hits.length;
|
|
507
|
+
priorKnowledge = learn.priorKnowledgeBrief(hits);
|
|
508
|
+
} catch { /* memory is optional — a missing store must never block a run */ }
|
|
509
|
+
}
|
|
494
510
|
const run = arena.startArena({
|
|
495
511
|
task,
|
|
496
|
-
scope:
|
|
512
|
+
scope: scopeArg,
|
|
497
513
|
flags: rest.filter(a => ['--all', '--license', '--theft', '--jailbreak', '--chaos', '--pre-public', '--pre-launch'].includes(a)),
|
|
498
514
|
limits,
|
|
515
|
+
priorKnowledge,
|
|
499
516
|
});
|
|
500
517
|
const first = arena.nextAction(run);
|
|
501
518
|
if (wantJson) { w(JSON.stringify({ run: state.summarize(run), next: first }, null, 2)); process.exit(0); }
|
|
@@ -506,6 +523,10 @@ if (args[0] === 'god' || args[0] === 'evil' || args[0] === 'arena') {
|
|
|
506
523
|
w(` limits: ${run.limits.maxRounds} rounds · ${run.limits.tokenBudget.toLocaleString()} tokens · ${Math.round(run.limits.wallClockMs / 60000)} min`);
|
|
507
524
|
w(` next: \x1b[32m${first.action}\x1b[0m (round ${first.round}, ~${((first.estimatedTokens || 0) / 1000).toFixed(0)}k tokens)`);
|
|
508
525
|
w('');
|
|
526
|
+
if (recalledCount) {
|
|
527
|
+
w(` recalled: \x1b[36m${recalledCount}\x1b[0m memor${recalledCount === 1 ? 'y' : 'ies'} from past runs on this scope — EVIL starts informed`);
|
|
528
|
+
}
|
|
529
|
+
w('');
|
|
509
530
|
w(`Drive the loop in your AI tool: \x1b[36m/arena ${run.task}\x1b[0m`);
|
|
510
531
|
w(`Inspect anytime: kodelythecc arena report ${run.runId}`);
|
|
511
532
|
w('');
|
|
@@ -534,6 +555,71 @@ if (args[0] === 'god' || args[0] === 'evil' || args[0] === 'arena') {
|
|
|
534
555
|
w('');
|
|
535
556
|
process.exit(0);
|
|
536
557
|
}
|
|
558
|
+
if (sub === 'learn') {
|
|
559
|
+
const runId = rest[1];
|
|
560
|
+
if (!runId) { process.stderr.write('usage: kodelythecc arena learn <run-id> [--commit]\n'); process.exit(2); }
|
|
561
|
+
const run = state.load(runId);
|
|
562
|
+
if (!run) { process.stderr.write(`no such run: ${runId}\n`); process.exit(1); }
|
|
563
|
+
const learn = require(path.join(ROOT, 'scripts', 'arena', 'learn.js'));
|
|
564
|
+
const memStore = require(path.join(ROOT, 'scripts', 'memory', 'store.js'));
|
|
565
|
+
|
|
566
|
+
const drafts = learn.runToMemories(run, { project: process.cwd() });
|
|
567
|
+
if (wantJson) { w(JSON.stringify({ drafts: drafts.map(d => d.memory) }, null, 2)); process.exit(0); }
|
|
568
|
+
|
|
569
|
+
if (!drafts.length) {
|
|
570
|
+
w('');
|
|
571
|
+
w('Nothing to learn from this run — no confirmed or refuted findings.');
|
|
572
|
+
w('Unverified findings are deliberately skipped: storing a question as');
|
|
573
|
+
w('knowledge would launder a guess into a fact.');
|
|
574
|
+
w('');
|
|
575
|
+
process.exit(0);
|
|
576
|
+
}
|
|
577
|
+
|
|
578
|
+
w('');
|
|
579
|
+
w(`\x1b[1mArena learn\x1b[0m — ${runId}`);
|
|
580
|
+
w('─'.repeat(64));
|
|
581
|
+
for (const d of drafts) {
|
|
582
|
+
const kind = d.finding.verdict === 'confirmed' ? '\x1b[32mconfirmed\x1b[0m' : '\x1b[33mrefuted \x1b[0m';
|
|
583
|
+
w(`${kind} ${d.memory.problem}`);
|
|
584
|
+
}
|
|
585
|
+
w('─'.repeat(64));
|
|
586
|
+
|
|
587
|
+
const clusters = learn.recurringClasses(drafts.map(d => d.memory), { minRuns: 2 });
|
|
588
|
+
if (clusters.length) {
|
|
589
|
+
w('');
|
|
590
|
+
w('\x1b[1mRecurring classes\x1b[0m — one is an incident, several is a process gap:');
|
|
591
|
+
for (const c of clusters) w(` ${String(c.count).padStart(2)}x ${c.class}`);
|
|
592
|
+
}
|
|
593
|
+
|
|
594
|
+
// Capture is explicit. Showing the drafts and requiring --commit is the
|
|
595
|
+
// whole point: memory that writes itself silently is memory you cannot trust.
|
|
596
|
+
if (!rest.includes('--commit')) {
|
|
597
|
+
w('');
|
|
598
|
+
w(`${drafts.length} draft(s). Nothing was written.`);
|
|
599
|
+
w(`Store them with: \x1b[36mkodelythecc arena learn ${runId} --commit\x1b[0m`);
|
|
600
|
+
w('');
|
|
601
|
+
process.exit(0);
|
|
602
|
+
}
|
|
603
|
+
|
|
604
|
+
let stored = 0;
|
|
605
|
+
for (const d of drafts) {
|
|
606
|
+
try { memStore.capture(d.memory); stored++; }
|
|
607
|
+
catch (err) { process.stderr.write(` skipped: ${err.message}\n`); }
|
|
608
|
+
}
|
|
609
|
+
const proposals = learn.analyzeRunForProposals(drafts.map(d => d.memory), { minRuns: 2 });
|
|
610
|
+
if (proposals.length) {
|
|
611
|
+
try {
|
|
612
|
+
const P = require(path.join(ROOT, 'scripts', 'evolve', 'proposals.js'));
|
|
613
|
+
for (const pr of proposals) P.appendProposal(pr);
|
|
614
|
+
} catch { /* evolve is optional */ }
|
|
615
|
+
}
|
|
616
|
+
w('');
|
|
617
|
+
w(`Stored ${stored} memory item(s)${proposals.length ? ` and ${proposals.length} guard proposal(s)` : ''}.`);
|
|
618
|
+
w('Future arena runs on this scope will recall them automatically.');
|
|
619
|
+
w('');
|
|
620
|
+
process.exit(0);
|
|
621
|
+
}
|
|
622
|
+
|
|
537
623
|
if (sub === 'report') {
|
|
538
624
|
const runId = rest[1];
|
|
539
625
|
if (!runId) { process.stderr.write('usage: kodelythecc arena report <run-id>\n'); process.exit(2); }
|
package/commands/arena.md
CHANGED
|
@@ -69,6 +69,41 @@ When the action is `report`, print the full markdown:
|
|
|
69
69
|
kodelythecc arena report <run-id> --md
|
|
70
70
|
```
|
|
71
71
|
|
|
72
|
+
### 6. `learn` — close the compound loop
|
|
73
|
+
|
|
74
|
+
A run that ends is knowledge thrown away. After the report:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
kodelythecc arena learn <run-id> # show what would be remembered
|
|
78
|
+
kodelythecc arena learn <run-id> --commit # store it
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Every **confirmed** finding becomes a memory carrying the fix and the repro that
|
|
82
|
+
proved it. Every **refuted** finding becomes a memory too — the more valuable
|
|
83
|
+
half, because without it the next run re-investigates the same non-bug and burns
|
|
84
|
+
a real verification pass proving the same negative.
|
|
85
|
+
|
|
86
|
+
**Unverified findings are deliberately skipped.** Storing a question as knowledge
|
|
87
|
+
would launder a guess into a fact, and future runs would recall it as settled.
|
|
88
|
+
|
|
89
|
+
The return path is automatic: the next `arena start` on that scope recalls those
|
|
90
|
+
memories and injects them into round 1's briefs, so EVIL opens where the last run
|
|
91
|
+
closed instead of rediscovering it. Pass `--fresh` to skip the recall.
|
|
92
|
+
|
|
93
|
+
```
|
|
94
|
+
arena run ──▶ confirmed + refuted findings ──▶ memories
|
|
95
|
+
▲ │
|
|
96
|
+
└──────── prior-knowledge brief ◀──────────────┘
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
When the same bug **class** is confirmed repeatedly, `learn --commit` also files an
|
|
100
|
+
`arena-guard` proposal into evolve. One symlink bug is an incident; two in the same
|
|
101
|
+
file is a process gap, and the proposal says so — "add a shared path-safety helper
|
|
102
|
+
and route every file write through it" rather than patching the third one later.
|
|
103
|
+
|
|
104
|
+
Nothing is written without `--commit`. Memory that writes itself silently is memory
|
|
105
|
+
you cannot trust.
|
|
106
|
+
|
|
72
107
|
## Rules that keep this honest
|
|
73
108
|
|
|
74
109
|
- **Never skip verification.** Unverified findings waste GOD's next entire round — that's the expensive failure mode this design exists to prevent.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "kodelyth-ecc",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.10.0",
|
|
4
4
|
"description": "Production-grade AI coding toolkit — 70 agents (incl. devil-mode adversarial crew), 194 skills, 97 commands, parallel multi-agent commands, semantic intent routing, self-learning memory, and a built-in MCP server (16 tools / 6 prompts / 377 resources) that bridges to Claude Desktop, LangGraph, AutoGen, CrewAI, and OpenAI Agents SDK. Works with Claude Code, Windsurf, Cursor, Codex, Antigravity, OpenCode, Cline, RooCode, Aider, Kimi, and Gemini CLI.",
|
|
5
5
|
"author": "Kodelyth <github.com/sifxprime>",
|
|
6
6
|
"license": "MIT",
|
package/scripts/arena/arena.js
CHANGED
|
@@ -33,10 +33,13 @@ const PHASE = {
|
|
|
33
33
|
|
|
34
34
|
// ── Starting a run ───────────────────────────────────────────────────────────
|
|
35
35
|
|
|
36
|
-
function startArena({ task, scope = '.', flags = [], limits = {} } = {}) {
|
|
36
|
+
function startArena({ task, scope = '.', flags = [], limits = {}, priorKnowledge = '' } = {}) {
|
|
37
37
|
const run = state.createRun({ task, limits });
|
|
38
38
|
run.scope = scope;
|
|
39
39
|
run.flags = flags;
|
|
40
|
+
// Recalled from past runs against this scope. Injected into every EVIL brief so
|
|
41
|
+
// round 1 opens where the last run closed instead of rediscovering it.
|
|
42
|
+
run.priorKnowledge = String(priorKnowledge || '');
|
|
40
43
|
run.phase = PHASE.GOD_BUILD;
|
|
41
44
|
run.pending = { artifacts: [], findings: [], addressedIds: [] };
|
|
42
45
|
state.save(run);
|
|
@@ -78,6 +81,7 @@ function nextAction(run) {
|
|
|
78
81
|
flags: run.flags,
|
|
79
82
|
round,
|
|
80
83
|
knownFindingIds: run.seenFindingIds,
|
|
84
|
+
priorKnowledge: run.priorKnowledge || '',
|
|
81
85
|
});
|
|
82
86
|
return {
|
|
83
87
|
action: PHASE.EVIL_HUNT,
|
|
@@ -172,6 +176,10 @@ function closeRound(run) {
|
|
|
172
176
|
});
|
|
173
177
|
|
|
174
178
|
// Attach whether GOD actually finished its side of the round.
|
|
179
|
+
// Persist WHICH findings GOD answered for, not just how many are left over.
|
|
180
|
+
// Without the ids, nothing downstream can tell a confirmed-and-fixed finding
|
|
181
|
+
// from confirmed-and-ignored — the dashboard would report every fix as open risk.
|
|
182
|
+
verdict.addressedIds = [...(run.pending.addressedIds || [])];
|
|
175
183
|
verdict.godComplete = completion.complete;
|
|
176
184
|
verdict.unverifiedArtifacts = completion.unverifiedArtifacts;
|
|
177
185
|
verdict.outstandingFindings = completion.outstandingFindings;
|
package/scripts/arena/evil.js
CHANGED
|
@@ -61,7 +61,7 @@ function selectCrew(flags = []) {
|
|
|
61
61
|
// Each agent is told to report in the Finding contract, and — critically — that
|
|
62
62
|
// a finding without a reproduction is worth less than no finding at all.
|
|
63
63
|
|
|
64
|
-
function huntBrief({ agent, scope, round, knownFindingIds = [] }) {
|
|
64
|
+
function huntBrief({ agent, scope, round, knownFindingIds = [], priorKnowledge = '' }) {
|
|
65
65
|
const meta = CREW[agent] || { hunts: 'issues' };
|
|
66
66
|
return [
|
|
67
67
|
`You are ${agent}. Hunt: ${meta.hunts}.`,
|
|
@@ -80,7 +80,10 @@ function huntBrief({ agent, scope, round, knownFindingIds = [] }) {
|
|
|
80
80
|
'- `repro` must be concrete steps or a command. A finding you cannot reproduce is `speculative` at best.',
|
|
81
81
|
'- Do not pad. Five confirmed findings beat forty guesses — unverifiable findings get refuted and count against you.',
|
|
82
82
|
'- Only claim `confirmed` when you have actually reproduced it.',
|
|
83
|
-
|
|
83
|
+
// Prior knowledge lands last so it reads as context for the rules above,
|
|
84
|
+
// not as a competing instruction set.
|
|
85
|
+
priorKnowledge || null,
|
|
86
|
+
].filter(line => line !== null).join('\n');
|
|
84
87
|
}
|
|
85
88
|
|
|
86
89
|
// ── Stage 2: verification briefs ─────────────────────────────────────────────
|
|
@@ -174,12 +177,12 @@ function actionable(findings = [], { minRisk = 2.0 } = {}) {
|
|
|
174
177
|
// ── Sweep plan ───────────────────────────────────────────────────────────────
|
|
175
178
|
// The orchestrator asks for a plan, dispatches the agents, and feeds results back.
|
|
176
179
|
|
|
177
|
-
function planSweep({ scope, flags = [], round = 1, knownFindingIds = [] } = {}) {
|
|
180
|
+
function planSweep({ scope, flags = [], round = 1, knownFindingIds = [], priorKnowledge = '' } = {}) {
|
|
178
181
|
const crew = selectCrew(flags);
|
|
179
182
|
return {
|
|
180
183
|
round,
|
|
181
184
|
crew,
|
|
182
|
-
briefs: crew.map(agent => ({ agent, brief: huntBrief({ agent, scope, round, knownFindingIds }) })),
|
|
185
|
+
briefs: crew.map(agent => ({ agent, brief: huntBrief({ agent, scope, round, knownFindingIds, priorKnowledge }) })),
|
|
183
186
|
estimatedTokens: crew.length * 12000, // rough: one agent sweep ≈ 12k
|
|
184
187
|
};
|
|
185
188
|
}
|
|
@@ -0,0 +1,273 @@
|
|
|
1
|
+
// scripts/arena/learn.js
|
|
2
|
+
//
|
|
3
|
+
// Phase 4 — compound learning. Turns a finished arena run into durable
|
|
4
|
+
// knowledge, and feeds that knowledge back into the next run.
|
|
5
|
+
//
|
|
6
|
+
// The loop only compounds if it closes:
|
|
7
|
+
//
|
|
8
|
+
// arena run ──▶ confirmed + refuted findings ──▶ memories
|
|
9
|
+
// ▲ │
|
|
10
|
+
// └──────── prior-knowledge brief ◀───────────────┘
|
|
11
|
+
//
|
|
12
|
+
// Without the return path, every run starts from zero and EVIL re-derives the
|
|
13
|
+
// same bug classes forever. With it, round 1 of run N begins where run N-1
|
|
14
|
+
// finished, and a refuted false positive is never re-litigated across runs.
|
|
15
|
+
//
|
|
16
|
+
// Everything here is PURE — no I/O, no clock, no randomness. Callers own the
|
|
17
|
+
// disk (scripts/memory/store.js) and the confirmation prompt. That keeps the
|
|
18
|
+
// logic testable without writing to the user's real memory store, and keeps
|
|
19
|
+
// capture an explicit, visible act rather than a hidden side effect of closing
|
|
20
|
+
// a round.
|
|
21
|
+
|
|
22
|
+
'use strict';
|
|
23
|
+
|
|
24
|
+
const { VERDICT } = require('./contract');
|
|
25
|
+
|
|
26
|
+
// ── Bug classes ─────────────────────────────────────────────────────────────
|
|
27
|
+
//
|
|
28
|
+
// A deliberately small, reviewable keyword map — not a classifier. Its only job
|
|
29
|
+
// is to group findings well enough that the same class recurring across runs is
|
|
30
|
+
// visible. Order matters: the most specific pattern must win, so `symlink` is
|
|
31
|
+
// tested before the broader `path` rules.
|
|
32
|
+
const CLASSES = [
|
|
33
|
+
['filesystem-symlink', /\bsymlink|lstat|O_NOFOLLOW|hard ?link|dangling\b/i],
|
|
34
|
+
['file-permissions', /\bchmod|umask|0600|0644|0444|world-readable|file mode|file permission|permissions? (?:not )?preserved\b/i],
|
|
35
|
+
['redos', /\bredos|backtrack|quadratic|catastrophic|O\(n\^?2\)|unanchored\b/i],
|
|
36
|
+
['path-traversal', /\btraversal|confinement|arbitrary (?:write|path)|escape the root\b/i],
|
|
37
|
+
['race-condition', /\btoctou|race condition|check.to.use\b/i],
|
|
38
|
+
['resource-exhaustion', /\bmemory|heap|rss|oom|amplification|exhaust\b/i],
|
|
39
|
+
['missing-limit', /\bno (?:input )?(?:size )?cap|unbounded|no limit|missing limit\b/i],
|
|
40
|
+
['temp-file-handling', /\btemp(?:orary)? file|tmp file|leftover|predictable (?:name|filename)\b/i],
|
|
41
|
+
['prompt-injection', /\binjection|jailbreak|untrusted (?:text|content|input)|system.prompt leak|inert data|trust.boundary\b/i],
|
|
42
|
+
['secret-exposure', /\bsecret|credential|api key|token leak|password\b/i],
|
|
43
|
+
['semantic-corruption', /\bsemantic|meaning|threshold|negation|inverts?|widen/i],
|
|
44
|
+
['idempotency', /\bidempoten|fixed point|second run|re-?run\b/i],
|
|
45
|
+
['input-validation', /validat|wrong shape|silently accept|malformed|type ?error|unsanitiz/i],
|
|
46
|
+
['supply-chain', /\btyposquat|lockfile|install script|dependency confusion\b/i],
|
|
47
|
+
];
|
|
48
|
+
|
|
49
|
+
function classify(finding = {}) {
|
|
50
|
+
const hay = `${finding.title || ''} ${finding.evidence || ''} ${finding.fix || ''}`;
|
|
51
|
+
const hit = CLASSES.find(([, re]) => re.test(hay));
|
|
52
|
+
return hit ? hit[0] : 'uncategorized';
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
// ── Findings → memory drafts ────────────────────────────────────────────────
|
|
56
|
+
|
|
57
|
+
function langFromFile(file) {
|
|
58
|
+
if (!file) return null;
|
|
59
|
+
const ext = String(file).split('.').pop().toLowerCase();
|
|
60
|
+
return {
|
|
61
|
+
js: 'javascript', mjs: 'javascript', cjs: 'javascript',
|
|
62
|
+
ts: 'typescript', tsx: 'typescript', jsx: 'javascript',
|
|
63
|
+
py: 'python', go: 'go', rs: 'rust', rb: 'ruby',
|
|
64
|
+
java: 'java', kt: 'kotlin', swift: 'swift', php: 'php',
|
|
65
|
+
c: 'c', h: 'c', cc: 'cpp', cpp: 'cpp', hpp: 'cpp', cs: 'csharp',
|
|
66
|
+
sh: 'shell', sql: 'sql', md: null,
|
|
67
|
+
}[ext] || null;
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
function scopeTag(scope) {
|
|
71
|
+
if (!scope) return null;
|
|
72
|
+
const parts = String(scope).replace(/\/+$/, '').split('/').filter(Boolean);
|
|
73
|
+
return parts.length ? parts[parts.length - 1] : null;
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
function where(finding) {
|
|
77
|
+
if (!finding.file) return '';
|
|
78
|
+
return finding.line ? ` (${finding.file}:${finding.line})` : ` (${finding.file})`;
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
// A CONFIRMED finding becomes "this was really broken, here is what fixed it".
|
|
82
|
+
function confirmedToMemory(finding, ctx) {
|
|
83
|
+
const cls = classify(finding);
|
|
84
|
+
const approach = [
|
|
85
|
+
finding.fix ? `Fix: ${finding.fix}` : null,
|
|
86
|
+
finding.repro ? `Proved by: ${finding.repro}` : null,
|
|
87
|
+
finding.evidence ? `The offending code was: ${finding.evidence}` : null,
|
|
88
|
+
].filter(Boolean).join('\n\n');
|
|
89
|
+
|
|
90
|
+
return {
|
|
91
|
+
problem: `${cls}: ${finding.title}${where(finding)}`.slice(0, 500),
|
|
92
|
+
approach: (approach || 'Confirmed by the arena; no fix recorded.').slice(0, 2000),
|
|
93
|
+
tags: dedupeTags(['arena', cls, 'confirmed', finding.severity, langFromFile(finding.file), scopeTag(ctx.scope)]),
|
|
94
|
+
project: ctx.project || null,
|
|
95
|
+
language: langFromFile(finding.file),
|
|
96
|
+
files: finding.file ? [finding.file] : [],
|
|
97
|
+
gotchas: finding.repro ? [`Reproduce with: ${finding.repro}`.slice(0, 300)] : [],
|
|
98
|
+
source: 'arena',
|
|
99
|
+
};
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
// A REFUTED finding becomes "this LOOKS wrong and is not" — the more valuable
|
|
103
|
+
// half. Without it, every future run re-investigates the same false positive
|
|
104
|
+
// and spends a real verification pass proving the same negative.
|
|
105
|
+
function refutedToMemory(finding, ctx) {
|
|
106
|
+
const cls = classify(finding);
|
|
107
|
+
return {
|
|
108
|
+
problem: `False positive — ${finding.title}${where(finding)} looks like a bug but is not`.slice(0, 500),
|
|
109
|
+
approach: [
|
|
110
|
+
`An arena verification pass refuted this. Do not re-report it without new evidence.`,
|
|
111
|
+
finding.evidence ? `Code in question: ${finding.evidence}` : null,
|
|
112
|
+
finding.repro ? `What was tried: ${finding.repro}` : null,
|
|
113
|
+
].filter(Boolean).join('\n\n').slice(0, 2000),
|
|
114
|
+
tags: dedupeTags(['arena', cls, 'refuted', 'false-positive', langFromFile(finding.file), scopeTag(ctx.scope)]),
|
|
115
|
+
project: ctx.project || null,
|
|
116
|
+
language: langFromFile(finding.file),
|
|
117
|
+
files: finding.file ? [finding.file] : [],
|
|
118
|
+
gotchas: [],
|
|
119
|
+
source: 'arena',
|
|
120
|
+
};
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
function dedupeTags(tags) {
|
|
124
|
+
return Array.from(new Set(tags.filter(Boolean).map(String))).slice(0, 20);
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
// Only settled findings are worth remembering. An `unverified` finding is a
|
|
128
|
+
// question nobody answered — storing it as knowledge would launder a guess into
|
|
129
|
+
// a fact, and future runs would recall it as though it had been established.
|
|
130
|
+
function runToMemories(run = {}, { project = null } = {}) {
|
|
131
|
+
const ctx = { scope: run.scope, project: project || run.projectRoot || null };
|
|
132
|
+
const seen = new Set();
|
|
133
|
+
const drafts = [];
|
|
134
|
+
|
|
135
|
+
for (const round of run.rounds || []) {
|
|
136
|
+
for (const f of round.findings || []) {
|
|
137
|
+
if (seen.has(f.id)) continue;
|
|
138
|
+
seen.add(f.id);
|
|
139
|
+
if (f.verdict === VERDICT.CONFIRMED) drafts.push({ finding: f, memory: confirmedToMemory(f, ctx) });
|
|
140
|
+
else if (f.verdict === VERDICT.REFUTED) drafts.push({ finding: f, memory: refutedToMemory(f, ctx) });
|
|
141
|
+
}
|
|
142
|
+
}
|
|
143
|
+
return drafts;
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
// ── Memories → the next run's brief ─────────────────────────────────────────
|
|
147
|
+
//
|
|
148
|
+
// This is the return path. Recalled memories are injected into round 1 so EVIL
|
|
149
|
+
// opens where the last run closed instead of rediscovering it.
|
|
150
|
+
function priorKnowledgeBrief(memories = [], { limit = 8 } = {}) {
|
|
151
|
+
if (!memories.length) return '';
|
|
152
|
+
|
|
153
|
+
const confirmed = memories.filter(m => (m.tags || []).includes('confirmed')).slice(0, limit);
|
|
154
|
+
const refuted = memories.filter(m => (m.tags || []).includes('refuted')).slice(0, limit);
|
|
155
|
+
if (!confirmed.length && !refuted.length) return '';
|
|
156
|
+
|
|
157
|
+
const lines = ['', 'PRIOR KNOWLEDGE — this scope has been attacked before.'];
|
|
158
|
+
|
|
159
|
+
if (confirmed.length) {
|
|
160
|
+
lines.push('', 'Bugs confirmed here previously. They should be fixed; verify they stayed fixed,');
|
|
161
|
+
lines.push('then hunt for what those fixes may have broken or missed:');
|
|
162
|
+
confirmed.forEach(m => lines.push(` - ${m.problem}`));
|
|
163
|
+
}
|
|
164
|
+
if (refuted.length) {
|
|
165
|
+
lines.push('', 'Already investigated and REFUTED. Do not re-report these without new');
|
|
166
|
+
lines.push('evidence — a previous verification pass proved them wrong:');
|
|
167
|
+
refuted.forEach(m => lines.push(` - ${m.problem}`));
|
|
168
|
+
}
|
|
169
|
+
return lines.join('\n');
|
|
170
|
+
}
|
|
171
|
+
|
|
172
|
+
// ── Recurring classes → evolve proposals ────────────────────────────────────
|
|
173
|
+
//
|
|
174
|
+
// One bug is an incident. The same class across several runs is a gap in the
|
|
175
|
+
// process, and the fix belongs upstream — a lint rule, a test, or routing that
|
|
176
|
+
// summons the right specialist before the code is written.
|
|
177
|
+
function recurringClasses(memories = [], { minRuns = 2 } = {}) {
|
|
178
|
+
const byClass = new Map();
|
|
179
|
+
|
|
180
|
+
for (const m of memories) {
|
|
181
|
+
if ((m.source || '') !== 'arena') continue;
|
|
182
|
+
if (!(m.tags || []).includes('confirmed')) continue;
|
|
183
|
+
const cls = (m.tags || []).find(t => CLASSES.some(([name]) => name === t));
|
|
184
|
+
if (!cls) continue;
|
|
185
|
+
|
|
186
|
+
if (!byClass.has(cls)) byClass.set(cls, { class: cls, count: 0, files: new Set(), examples: [] });
|
|
187
|
+
const entry = byClass.get(cls);
|
|
188
|
+
entry.count += 1;
|
|
189
|
+
(m.files || []).forEach(f => entry.files.add(f));
|
|
190
|
+
if (entry.examples.length < 5) entry.examples.push(m.problem);
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
return [...byClass.values()]
|
|
194
|
+
.filter(e => e.count >= minRuns)
|
|
195
|
+
.map(e => ({ ...e, files: [...e.files] }))
|
|
196
|
+
.sort((a, b) => b.count - a.count);
|
|
197
|
+
}
|
|
198
|
+
|
|
199
|
+
const GUARD_ADVICE = {
|
|
200
|
+
'filesystem-symlink': 'Add a shared path-safety helper (lstat + O_NOFOLLOW + O_EXCL) and route every file write through it.',
|
|
201
|
+
'file-permissions': 'Capture the source mode and fchmod every replacement file; add a test asserting mode is preserved under a hostile umask.',
|
|
202
|
+
'redos': 'Bound every unanchored quantifier that scans user input, and add a timing assertion to the test suite.',
|
|
203
|
+
'path-traversal': 'Decide the containment boundary explicitly and document who owns it — the library or the caller.',
|
|
204
|
+
'race-condition': 'Operate on a file descriptor opened once, not on a path re-resolved at each step.',
|
|
205
|
+
'resource-exhaustion': 'Cap input size at a realistic bound and assert peak memory in a test.',
|
|
206
|
+
'missing-limit': 'Every public entry point that accepts untrusted input needs a documented, tested limit.',
|
|
207
|
+
'temp-file-handling': 'Randomize temp names, create them O_EXCL at the final mode, and unlink in a finally.',
|
|
208
|
+
'prompt-injection': 'State the trust boundary in the command doc: file contents are data, never instructions.',
|
|
209
|
+
'secret-exposure': 'Add a secret scan to the pre-commit hook for this path.',
|
|
210
|
+
'semantic-corruption': 'Add golden tests asserting that meaning-bearing tokens survive transformation.',
|
|
211
|
+
'idempotency': 'Assert f(f(x)) === f(x) in the test suite for every transform.',
|
|
212
|
+
'input-validation': 'Validate argument shape at every public boundary and throw — never silently coerce to a plausible default.',
|
|
213
|
+
'supply-chain': 'Pin and verify dependencies; add a lockfile-drift check to CI.',
|
|
214
|
+
};
|
|
215
|
+
|
|
216
|
+
function buildGuardProposalMarkdown(cluster) {
|
|
217
|
+
const advice = GUARD_ADVICE[cluster.class] || 'Add a durable guard so this class cannot recur silently.';
|
|
218
|
+
return [
|
|
219
|
+
`## Recurring arena finding: \`${cluster.class}\``,
|
|
220
|
+
'',
|
|
221
|
+
`The arena has confirmed **${cluster.count}** findings of this class. One is an`,
|
|
222
|
+
`incident; ${cluster.count} is a process gap — the guard belongs upstream of the next bug.`,
|
|
223
|
+
'',
|
|
224
|
+
'**Files involved:**',
|
|
225
|
+
...(cluster.files.length ? cluster.files.map(f => `- \`${f}\``) : ['- _(none recorded)_']),
|
|
226
|
+
'',
|
|
227
|
+
'**Examples:**',
|
|
228
|
+
...cluster.examples.map(e => `- ${e}`),
|
|
229
|
+
'',
|
|
230
|
+
'**Proposed guard:**',
|
|
231
|
+
'',
|
|
232
|
+
advice,
|
|
233
|
+
].join('\n');
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
// Deterministic id so re-running analysis does not spawn duplicate proposals.
|
|
237
|
+
function guardProposalId(cluster) {
|
|
238
|
+
let h = 0x811c9dc5;
|
|
239
|
+
for (const ch of `arena-guard::${cluster.class}::${cluster.count}`) {
|
|
240
|
+
h ^= ch.charCodeAt(0);
|
|
241
|
+
h = Math.imul(h, 0x01000193) >>> 0;
|
|
242
|
+
}
|
|
243
|
+
return `guard-${h.toString(16).padStart(8, '0')}`;
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
function analyzeRunForProposals(memories = [], { minRuns = 2 } = {}) {
|
|
247
|
+
return recurringClasses(memories, { minRuns }).map(cluster => ({
|
|
248
|
+
id: guardProposalId(cluster),
|
|
249
|
+
type: 'arena-guard',
|
|
250
|
+
evidence: { class: cluster.class, count: cluster.count, files: cluster.files, examples: cluster.examples },
|
|
251
|
+
proposal: {
|
|
252
|
+
kind: 'arena-guard',
|
|
253
|
+
target_path: 'tasks/lessons.md',
|
|
254
|
+
diff: buildGuardProposalMarkdown(cluster),
|
|
255
|
+
rationale: `${cluster.count} confirmed arena findings share the class "${cluster.class}".`,
|
|
256
|
+
},
|
|
257
|
+
}));
|
|
258
|
+
}
|
|
259
|
+
|
|
260
|
+
module.exports = {
|
|
261
|
+
CLASSES,
|
|
262
|
+
GUARD_ADVICE,
|
|
263
|
+
classify,
|
|
264
|
+
confirmedToMemory,
|
|
265
|
+
refutedToMemory,
|
|
266
|
+
runToMemories,
|
|
267
|
+
priorKnowledgeBrief,
|
|
268
|
+
recurringClasses,
|
|
269
|
+
buildGuardProposalMarkdown,
|
|
270
|
+
guardProposalId,
|
|
271
|
+
analyzeRunForProposals,
|
|
272
|
+
_internals: { langFromFile, scopeTag, dedupeTags },
|
|
273
|
+
};
|
|
@@ -580,9 +580,120 @@ function tokenBudgetSnapshot({ budgetDir = defaultBudgetDir() } = {}) {
|
|
|
580
580
|
return { sessions: sessions.slice(0, 50), total_tokens: total };
|
|
581
581
|
}
|
|
582
582
|
|
|
583
|
+
// ── Arena ────────────────────────────────────────────────────────────────────
|
|
584
|
+
//
|
|
585
|
+
// The dashboard's job here is to make ONE thing legible: did the attacker give
|
|
586
|
+
// up? A run whose new-finding count falls to zero is converging. A flat or
|
|
587
|
+
// rising line means the code has deeper problems, or the scope is too broad for
|
|
588
|
+
// EVIL to ever exhaust — either way, look before shipping.
|
|
589
|
+
|
|
590
|
+
function arenaSnapshot({ runLimit = 20 } = {}) {
|
|
591
|
+
let arenaState = null;
|
|
592
|
+
let learn = null;
|
|
593
|
+
try { arenaState = require('../arena/state.js'); } catch { /* arena is optional */ }
|
|
594
|
+
try { learn = require('../arena/learn.js'); } catch { /* */ }
|
|
595
|
+
if (!arenaState) return { available: false, runs: [], classes: [], open: [], totals: {} };
|
|
596
|
+
|
|
597
|
+
let list = [];
|
|
598
|
+
try { list = arenaState.listRuns() || []; } catch { /* no runs yet */ }
|
|
599
|
+
|
|
600
|
+
const runs = list.slice(0, Math.max(1, Math.min(100, runLimit))).map(meta => {
|
|
601
|
+
let run = null;
|
|
602
|
+
try { run = arenaState.load(meta.runId); } catch { /* skip unreadable */ }
|
|
603
|
+
if (!run) return null;
|
|
604
|
+
|
|
605
|
+
const rounds = run.rounds || [];
|
|
606
|
+
const settled = [];
|
|
607
|
+
const seen = new Set();
|
|
608
|
+
for (const r of rounds) {
|
|
609
|
+
for (const f of r.findings || []) {
|
|
610
|
+
if (seen.has(f.id)) continue;
|
|
611
|
+
seen.add(f.id);
|
|
612
|
+
settled.push(f);
|
|
613
|
+
}
|
|
614
|
+
}
|
|
615
|
+
|
|
616
|
+
return {
|
|
617
|
+
runId: run.runId,
|
|
618
|
+
task: run.task,
|
|
619
|
+
scope: run.scope || '.',
|
|
620
|
+
status: run.status,
|
|
621
|
+
stopReason: run.stopReason || null,
|
|
622
|
+
startedAt: run.startedAt,
|
|
623
|
+
rounds: rounds.length,
|
|
624
|
+
tokens: run.spent?.tokens || 0,
|
|
625
|
+
// The trend IS the story: new findings per round, which should fall to zero.
|
|
626
|
+
trend: rounds.map(r => r.counts?.new || 0),
|
|
627
|
+
confirmed: settled.filter(f => f.verdict === 'confirmed').length,
|
|
628
|
+
refuted: settled.filter(f => f.verdict === 'refuted').length,
|
|
629
|
+
unverified: settled.filter(f => f.verdict === 'unverified').length,
|
|
630
|
+
artifacts: rounds.reduce((n, r) => n + (r.artifacts?.length || 0), 0),
|
|
631
|
+
recalled: run.priorKnowledge ? true : false,
|
|
632
|
+
findings: settled,
|
|
633
|
+
};
|
|
634
|
+
}).filter(Boolean);
|
|
635
|
+
|
|
636
|
+
// Still-open risk across every run, worst first. A confirmed finding nobody
|
|
637
|
+
// fixed is the single most useful thing this page can surface.
|
|
638
|
+
const open = [];
|
|
639
|
+
// Findings GOD answered for are not open risk. Counting a confirmed-and-fixed
|
|
640
|
+
// bug as outstanding would make a healthy run look alarming.
|
|
641
|
+
const addressed = new Set();
|
|
642
|
+
for (const meta of list.slice(0, runLimit)) {
|
|
643
|
+
try {
|
|
644
|
+
const full = arenaState.load(meta.runId);
|
|
645
|
+
for (const rd of full?.rounds || []) for (const id of rd.addressedIds || []) addressed.add(id);
|
|
646
|
+
} catch { /* */ }
|
|
647
|
+
}
|
|
648
|
+
for (const r of runs) {
|
|
649
|
+
for (const f of r.findings) {
|
|
650
|
+
if (f.verdict !== 'confirmed') continue;
|
|
651
|
+
if (addressed.has(f.id)) continue; // GOD answered for this one
|
|
652
|
+
open.push({
|
|
653
|
+
runId: r.runId, scope: r.scope, title: f.title,
|
|
654
|
+
file: f.file, line: f.line, severity: f.severity,
|
|
655
|
+
risk: f.risk || 0,
|
|
656
|
+
class: learn ? learn.classify(f) : null,
|
|
657
|
+
});
|
|
658
|
+
}
|
|
659
|
+
}
|
|
660
|
+
open.sort((a, b) => b.risk - a.risk);
|
|
661
|
+
|
|
662
|
+
// Which bug classes keep coming back — the signal that a guard belongs upstream.
|
|
663
|
+
const classCount = new Map();
|
|
664
|
+
if (learn) {
|
|
665
|
+
for (const r of runs) {
|
|
666
|
+
for (const f of r.findings) {
|
|
667
|
+
if (f.verdict !== 'confirmed') continue;
|
|
668
|
+
const c = learn.classify(f);
|
|
669
|
+
classCount.set(c, (classCount.get(c) || 0) + 1);
|
|
670
|
+
}
|
|
671
|
+
}
|
|
672
|
+
}
|
|
673
|
+
const classes = [...classCount.entries()]
|
|
674
|
+
.map(([name, count]) => ({ name, count }))
|
|
675
|
+
.sort((a, b) => b.count - a.count);
|
|
676
|
+
|
|
677
|
+
const totals = {
|
|
678
|
+
runs: runs.length,
|
|
679
|
+
converged: runs.filter(r => r.status === 'converged').length,
|
|
680
|
+
confirmed: runs.reduce((n, r) => n + r.confirmed, 0),
|
|
681
|
+
refuted: runs.reduce((n, r) => n + r.refuted, 0),
|
|
682
|
+
artifacts: runs.reduce((n, r) => n + r.artifacts, 0),
|
|
683
|
+
tokens: runs.reduce((n, r) => n + r.tokens, 0),
|
|
684
|
+
};
|
|
685
|
+
|
|
686
|
+
// Drop the raw findings from the wire payload — the page needs the counts and
|
|
687
|
+
// the open list, not every finding on every run.
|
|
688
|
+
const wireRuns = runs.map(({ findings, ...rest }) => rest);
|
|
689
|
+
return { available: true, runs: wireRuns, open: open.slice(0, 40), classes, totals };
|
|
690
|
+
}
|
|
691
|
+
|
|
583
692
|
module.exports = {
|
|
584
693
|
// overview
|
|
585
694
|
overview,
|
|
695
|
+
// arena
|
|
696
|
+
arenaSnapshot,
|
|
586
697
|
// memory
|
|
587
698
|
memoryStats,
|
|
588
699
|
recentMemories,
|
|
@@ -304,6 +304,10 @@ function handleRequest(req, res) {
|
|
|
304
304
|
return jsonResponse(res, 200, { ok: true, ...s });
|
|
305
305
|
}
|
|
306
306
|
|
|
307
|
+
if (p === '/api/arena') {
|
|
308
|
+
return jsonResponse(res, 200, data.arenaSnapshot({ runLimit: Number(q.get('limit')) || 20 }));
|
|
309
|
+
}
|
|
310
|
+
|
|
307
311
|
if (p === '/api/codebase') {
|
|
308
312
|
const cb = require('../codebase/index.js');
|
|
309
313
|
return jsonResponse(res, 200, cb.dashboardSnapshot());
|
|
@@ -223,6 +223,7 @@
|
|
|
223
223
|
<button data-tab="rtk">Token Savings</button>
|
|
224
224
|
<button data-tab="memory">Memory</button>
|
|
225
225
|
<button data-tab="codebase">Codebase</button>
|
|
226
|
+
<button data-tab="arena">Arena</button>
|
|
226
227
|
<button data-tab="evolve">Evolve</button>
|
|
227
228
|
<button data-tab="catalog">Catalog</button>
|
|
228
229
|
<button data-tab="sessions">Sessions</button>
|
|
@@ -320,6 +321,50 @@ kodelyth-ecc codebase query detect_changes '{}'</pre>
|
|
|
320
321
|
</section>
|
|
321
322
|
|
|
322
323
|
<!-- ───────── EVOLVE ───────── -->
|
|
324
|
+
<!-- ───────── ARENA ───────── -->
|
|
325
|
+
<section data-panel="arena" hidden>
|
|
326
|
+
<div class="grid" id="arenaCards"></div>
|
|
327
|
+
<hr class="sep">
|
|
328
|
+
<div class="card">
|
|
329
|
+
<h2>Did the attacker give up?</h2>
|
|
330
|
+
<p class="muted" style="font-size:12.5px;margin:-4px 0 14px;">
|
|
331
|
+
New findings per round. Falling to zero means EVIL ran out of ideas.
|
|
332
|
+
A flat or rising line means stop and look — either the code has deeper
|
|
333
|
+
problems, or the scope is too broad to ever exhaust.
|
|
334
|
+
</p>
|
|
335
|
+
<div id="arenaRuns">loading…</div>
|
|
336
|
+
</div>
|
|
337
|
+
<hr class="sep">
|
|
338
|
+
<div class="row">
|
|
339
|
+
<div class="card">
|
|
340
|
+
<h2>Still open</h2>
|
|
341
|
+
<p class="muted" style="font-size:12.5px;margin:-4px 0 12px;">
|
|
342
|
+
Confirmed and not yet answered for, worst risk first.
|
|
343
|
+
</p>
|
|
344
|
+
<div id="arenaOpen">loading…</div>
|
|
345
|
+
</div>
|
|
346
|
+
<div class="card">
|
|
347
|
+
<h2>Recurring bug classes</h2>
|
|
348
|
+
<p class="muted" style="font-size:12.5px;margin:-4px 0 12px;">
|
|
349
|
+
One is an incident. Several is a process gap — the guard belongs upstream.
|
|
350
|
+
</p>
|
|
351
|
+
<div id="arenaClasses">loading…</div>
|
|
352
|
+
</div>
|
|
353
|
+
</div>
|
|
354
|
+
<hr class="sep">
|
|
355
|
+
<div class="card">
|
|
356
|
+
<h2>Compound learning</h2>
|
|
357
|
+
<p class="muted" style="font-size:12.5px;margin:-4px 0 10px;">
|
|
358
|
+
Store what a run proved so the next run on that scope starts informed:
|
|
359
|
+
</p>
|
|
360
|
+
<div><code>kodelyth-ecc arena learn <run-id> --commit</code></div>
|
|
361
|
+
<p class="muted" style="margin-top:12px;font-size:12.5px;">
|
|
362
|
+
Nothing is written without <code>--commit</code>. The dashboard NEVER
|
|
363
|
+
writes to your memory store.
|
|
364
|
+
</p>
|
|
365
|
+
</div>
|
|
366
|
+
</section>
|
|
367
|
+
|
|
323
368
|
<section data-panel="evolve" hidden>
|
|
324
369
|
<div class="grid" id="evolveCards"></div>
|
|
325
370
|
<hr class="sep">
|
|
@@ -527,6 +572,86 @@ kodelyth-ecc codebase query detect_changes '{}'</pre>
|
|
|
527
572
|
}
|
|
528
573
|
|
|
529
574
|
// ───── evolve ─────
|
|
575
|
+
// A compact sparkline made of block characters — no chart library, no CDN,
|
|
576
|
+
// and it stays legible when the run has one round or twenty.
|
|
577
|
+
function trendBar(trend) {
|
|
578
|
+
if (!trend || !trend.length) return '<span class="dim">no rounds yet</span>';
|
|
579
|
+
const max = Math.max(...trend, 1);
|
|
580
|
+
const blocks = '▁▂▃▄▅▆▇█';
|
|
581
|
+
return trend.map((n, i) => {
|
|
582
|
+
const idx = n === 0 ? 0 : Math.min(blocks.length - 1, Math.ceil((n / max) * (blocks.length - 1)));
|
|
583
|
+
const colour = n === 0 ? '#22c55e' : (i > 0 && n < trend[i - 1] ? '#eab308' : '#ef4444');
|
|
584
|
+
return `<span title="round ${i + 1}: ${n} new" style="color:${colour};font-size:18px;line-height:1;">${blocks[idx]}</span>`;
|
|
585
|
+
}).join('');
|
|
586
|
+
}
|
|
587
|
+
|
|
588
|
+
function arenaStatusPill(status) {
|
|
589
|
+
const map = {
|
|
590
|
+
converged: ['ok', 'converged'],
|
|
591
|
+
running: ['', 'running'],
|
|
592
|
+
exhausted: ['pending', 'max rounds'],
|
|
593
|
+
aborted: ['pending', 'stopped'],
|
|
594
|
+
};
|
|
595
|
+
const [cls, label] = map[status] || ['', status || 'unknown'];
|
|
596
|
+
return `<span class="pill ${cls}">${escapeHtml(label)}</span>`;
|
|
597
|
+
}
|
|
598
|
+
|
|
599
|
+
async function loadArena() {
|
|
600
|
+
try {
|
|
601
|
+
const a = await api('/api/arena');
|
|
602
|
+
if (!a.available) {
|
|
603
|
+
$('#arenaCards').innerHTML = '';
|
|
604
|
+
$('#arenaRuns').innerHTML = '<div class="empty">Arena is not installed.</div>';
|
|
605
|
+
$('#arenaOpen').innerHTML = '';
|
|
606
|
+
$('#arenaClasses').innerHTML = '';
|
|
607
|
+
return;
|
|
608
|
+
}
|
|
609
|
+
|
|
610
|
+
$('#arenaCards').innerHTML = [
|
|
611
|
+
statCard(a.totals.runs, 'Runs'),
|
|
612
|
+
statCard(a.totals.confirmed, 'Bugs confirmed'),
|
|
613
|
+
statCard(a.totals.refuted, 'False positives caught'),
|
|
614
|
+
statCard(a.open.length, 'Still open'),
|
|
615
|
+
statCard(a.totals.artifacts, 'Verified fixes'),
|
|
616
|
+
].join('');
|
|
617
|
+
|
|
618
|
+
$('#arenaRuns').innerHTML = a.runs.length
|
|
619
|
+
? `<div class="table-wrap"><table>
|
|
620
|
+
<thead><tr><th>Run</th><th>Scope</th><th>Trend</th><th>Rounds</th><th>Confirmed</th><th>Refuted</th><th>Tokens</th><th>Status</th></tr></thead>
|
|
621
|
+
<tbody>${a.runs.map(r => `<tr>
|
|
622
|
+
<td><span class="strong">${escapeHtml(r.task || r.runId)}</span>${r.recalled ? ' <span class="pill" title="started with recalled memories from past runs">recalled</span>' : ''}</td>
|
|
623
|
+
<td><code>${escapeHtml(r.scope)}</code></td>
|
|
624
|
+
<td>${trendBar(r.trend)}</td>
|
|
625
|
+
<td>${r.rounds}</td>
|
|
626
|
+
<td>${r.confirmed}</td>
|
|
627
|
+
<td>${r.refuted}</td>
|
|
628
|
+
<td>${(r.tokens || 0).toLocaleString()}</td>
|
|
629
|
+
<td>${arenaStatusPill(r.status)}</td>
|
|
630
|
+
</tr>`).join('')}</tbody></table></div>`
|
|
631
|
+
: '<div class="empty">No arena runs yet. Start one with <code>kodelyth-ecc arena start --task "..."</code></div>';
|
|
632
|
+
|
|
633
|
+
$('#arenaOpen').innerHTML = a.open.length
|
|
634
|
+
? a.open.map(f => `<div style="margin-bottom:12px;">
|
|
635
|
+
<div class="strong">${escapeHtml(f.title)}</div>
|
|
636
|
+
<div class="muted" style="font-size:12px;margin-top:3px;">
|
|
637
|
+
<span class="pill">${escapeHtml(f.severity)}</span>
|
|
638
|
+
risk ${f.risk}${f.class ? ` · ${escapeHtml(f.class)}` : ''}
|
|
639
|
+
${f.file ? ` · <code>${escapeHtml(f.file)}${f.line ? ':' + f.line : ''}</code>` : ''}
|
|
640
|
+
</div>
|
|
641
|
+
</div>`).join('')
|
|
642
|
+
: '<div class="empty">Nothing open. Every confirmed finding was either fixed or refuted.</div>';
|
|
643
|
+
|
|
644
|
+
$('#arenaClasses').innerHTML = a.classes.length
|
|
645
|
+
? a.classes.map(c => `<div style="display:flex;justify-content:space-between;margin-bottom:8px;">
|
|
646
|
+
<span>${escapeHtml(c.name)}${c.count > 1 ? ' <span class="pill pending">recurring</span>' : ''}</span>
|
|
647
|
+
<span class="strong">${c.count}</span>
|
|
648
|
+
</div>`).join('')
|
|
649
|
+
: '<div class="empty">No confirmed findings yet.</div>';
|
|
650
|
+
} catch (err) {
|
|
651
|
+
$('#arenaRuns').innerHTML = `<div class="empty">Could not load arena data: ${escapeHtml(String(err.message || err))}</div>`;
|
|
652
|
+
}
|
|
653
|
+
}
|
|
654
|
+
|
|
530
655
|
async function loadEvolve() {
|
|
531
656
|
try {
|
|
532
657
|
const e = await api('/api/evolve');
|
|
@@ -838,6 +963,7 @@ kodelyth-ecc codebase query detect_changes '{}'</pre>
|
|
|
838
963
|
if (tab === 'rtk') loadRtk();
|
|
839
964
|
if (tab === 'memory') loadMemory();
|
|
840
965
|
if (tab === 'codebase') loadCodebase();
|
|
966
|
+
if (tab === 'arena') loadArena();
|
|
841
967
|
if (tab === 'evolve') loadEvolve();
|
|
842
968
|
if (tab === 'catalog') loadCatalog();
|
|
843
969
|
if (tab === 'sessions') { loadIdeSessions(); loadSessions(); }
|