karajan-code 2.11.0 → 2.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/package.json +1 -1
- package/src/audit/plan-adherence.js +222 -0
- package/src/golden/asserter.js +90 -0
- package/src/golden/loader.js +52 -0
- package/src/golden/runner.js +92 -0
- package/src/golden/schema.js +41 -0
- package/src/orchestrator/drivers/post-loop.js +34 -2
- package/src/session/journal/summary-writer.js +39 -0
- package/src/utils/git.js +27 -0
- package/templates/roles/architect/anthropic.md +9 -0
- package/templates/roles/architect/default.md +9 -0
- package/templates/roles/architect/google.md +9 -0
- package/templates/roles/architect/openai.md +9 -0
- package/templates/roles/architect/opencode.md +9 -0
- package/templates/roles/architect.md +9 -0
- package/templates/roles/coder/anthropic.md +9 -0
- package/templates/roles/coder/default.md +9 -0
- package/templates/roles/coder/google.md +9 -0
- package/templates/roles/coder/openai.md +9 -0
- package/templates/roles/coder/opencode.md +9 -0
- package/templates/roles/coder.md +9 -0
- package/templates/roles/planner/anthropic.md +9 -0
- package/templates/roles/planner/default.md +9 -0
- package/templates/roles/planner/google.md +9 -0
- package/templates/roles/planner/openai.md +9 -0
- package/templates/roles/planner/opencode.md +9 -0
- package/templates/roles/planner.md +9 -0
- package/templates/roles/refactorer.md +9 -0
- package/templates/roles/repairer.md +9 -0
- package/templates/roles/tester.md +9 -0
package/README.md
CHANGED
|
@@ -23,7 +23,9 @@
|
|
|
23
23
|
|
|
24
24
|
---
|
|
25
25
|
|
|
26
|
-
> **v2.
|
|
26
|
+
> **v2.12.0 released** — Quality-measurement release. Two new features land together: every `kj run` against a known plan now scores how faithfully the coder followed it (deterministic 0–100 **plan adherence** metric, four weighted components, rendered in `summary.md`), and a small **golden-tasks** regression suite (`todo-rest-api`, `npm-package-cli`, `react-counter-component`) catches output-quality drops between Karajan versions before npm publish. Plus the shrink-budget CI gate now exempts human-facing docs from its 200-LOC ceiling while keeping AI-rule files (CLAUDE.md, AGENTS.md, role prompts) capped. 3 PRs for plan adherence (#645–#647), 4 for golden tasks (#648, #650–#652), 1 for the CI policy (#649). 4522/4522 tests passing. Safe upgrade from 2.11.0.
|
|
27
|
+
>
|
|
28
|
+
> **v2.11.0** — Dogfooding pass release. Two-day pass through a 10-level test plan surfaced and fixed a long tail of UX papercuts and three latent bugs that only show up on fresh `/tmp` repos: the `SonarStage` no longer loops on remoteless projects (was burning iterations until `max_iterations`-fallback-approval), the post-loop `commitAll` now tolerates the locale-specific "nothing to commit" race, the HU sub-pipeline branches off `master`/`HEAD` when the configured `main` doesn't exist, and `runFlow` now seals `session.status` at the boundary so `kj status` never shows zombi `running` runs again. Plus `hu-board` gains automatic ephemeral-project cleanup and an in-UI help modal for the five views. 14 PRs (#624–#637), 4452/4452 tests passing. Safe upgrade from 2.10.2.
|
|
27
29
|
>
|
|
28
30
|
> **v2.10.2** — Patch release. `kj init` wizard expanded from 9 prompts to a full setup: per-role provider selection (10 roles, "inherit / pick CLI / disable"), automatic SonarQube token generation via REST API (no more web UI walkthrough), git automation flags (`auto_commit/push/pr`) and HU Board security (bind host + port). +16 new tests. Safe upgrade from 2.10.1.
|
|
29
31
|
>
|
package/package.json
CHANGED
|
@@ -0,0 +1,222 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Plan adherence metric — KJC-TSK-0376 (deepeval-inspired).
|
|
3
|
+
* Pure offline 0-100 score: how faithfully did the coder follow the plan?
|
|
4
|
+
* Components (weighted): commit_attribution 40%, acceptance_tests 30%,
|
|
5
|
+
* scope_discipline 20%, dependency_order 10%. Components return null
|
|
6
|
+
* when no data; aggregator redistributes the weight.
|
|
7
|
+
* See docs/plan-adherence.md (PR-C) for the full spec.
|
|
8
|
+
*/
|
|
9
|
+
|
|
10
|
+
const WEIGHTS = {
|
|
11
|
+
commit_attribution: 40,
|
|
12
|
+
acceptance_tests: 30,
|
|
13
|
+
scope_discipline: 20,
|
|
14
|
+
dependency_order: 10,
|
|
15
|
+
};
|
|
16
|
+
|
|
17
|
+
/**
|
|
18
|
+
* Build a regex that matches the HU id anywhere in a commit message
|
|
19
|
+
* (case-insensitive, word-boundary friendly). The `_` and `-` characters
|
|
20
|
+
* appear inside HU ids so we anchor on whole-token rather than \b.
|
|
21
|
+
*/
|
|
22
|
+
function huIdMatcher(huId) {
|
|
23
|
+
const escaped = String(huId).replace(/[-/\\^$*+?.()|[\]{}]/g, "\\$&");
|
|
24
|
+
return new RegExp(`(^|[^a-z0-9])${escaped}([^a-z0-9]|$)`, "i");
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* Slugify an HU title for branch-name / scope matching. Mirrors the
|
|
29
|
+
* `buildHuBranchName` slug rule (lowercase, non-alnum → "-").
|
|
30
|
+
*/
|
|
31
|
+
function slugify(text) {
|
|
32
|
+
return String(text || "")
|
|
33
|
+
.toLowerCase()
|
|
34
|
+
.replaceAll(/[^a-z0-9]+/g, "-")
|
|
35
|
+
.replaceAll(/(^-+)|(-+$)/g, "");
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/**
|
|
39
|
+
* Try to attribute a single commit to one of the plan's HUs.
|
|
40
|
+
*
|
|
41
|
+
* Order of evidence, strongest first:
|
|
42
|
+
* 1. The HU id appears in the commit message.
|
|
43
|
+
* 2. The HU id appears in the branch name where the commit lives
|
|
44
|
+
* (commit.branch is supplied by listCommitsBetween's caller when
|
|
45
|
+
* it knows the per-HU branches).
|
|
46
|
+
* 3. The HU title slug matches the commit message.
|
|
47
|
+
*
|
|
48
|
+
* Returns the matched huId, or null when none of the above fires.
|
|
49
|
+
*/
|
|
50
|
+
export function attributeCommitToHu(commit, hus) {
|
|
51
|
+
if (!commit || !Array.isArray(hus) || hus.length === 0) return null;
|
|
52
|
+
const msg = String(commit.message || "");
|
|
53
|
+
const branch = String(commit.branch || "");
|
|
54
|
+
for (const hu of hus) {
|
|
55
|
+
if (!hu?.id) continue;
|
|
56
|
+
const matcher = huIdMatcher(hu.id);
|
|
57
|
+
if (matcher.test(msg)) return hu.id;
|
|
58
|
+
if (branch && matcher.test(branch)) return hu.id;
|
|
59
|
+
}
|
|
60
|
+
// Fallback: title slug
|
|
61
|
+
for (const hu of hus) {
|
|
62
|
+
const slug = slugify(hu.title);
|
|
63
|
+
if (!slug || slug.length < 6) continue;
|
|
64
|
+
if (slugify(msg).includes(slug)) return hu.id;
|
|
65
|
+
}
|
|
66
|
+
return null;
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
/**
|
|
70
|
+
* Component 1: commit attribution. Returns null when the run produced
|
|
71
|
+
* no commits at all (avoids penalising analysis-only or doc-only runs).
|
|
72
|
+
*/
|
|
73
|
+
export function computeCommitAttribution(commits, plan) {
|
|
74
|
+
if (!Array.isArray(commits) || commits.length === 0) return null;
|
|
75
|
+
if (!plan?.hus || plan.hus.length === 0) return null;
|
|
76
|
+
let attributed = 0;
|
|
77
|
+
for (const commit of commits) {
|
|
78
|
+
if (attributeCommitToHu(commit, plan.hus)) attributed += 1;
|
|
79
|
+
}
|
|
80
|
+
return Math.round((attributed / commits.length) * 100);
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
/**
|
|
84
|
+
* Component 2: acceptance test coverage. Heuristic: each HU declares
|
|
85
|
+
* `acceptance_tests`. We check whether at least one commit from that
|
|
86
|
+
* HU's pool has a `test:`-prefixed message. Per-HU score (1 or 0),
|
|
87
|
+
* averaged across HUs that DO declare tests. Returns null when no HU
|
|
88
|
+
* declares acceptance tests.
|
|
89
|
+
*/
|
|
90
|
+
export function computeAcceptanceTestCoverage(commits, plan) {
|
|
91
|
+
if (!plan?.hus || plan.hus.length === 0) return null;
|
|
92
|
+
if (!Array.isArray(commits) || commits.length === 0) return null;
|
|
93
|
+
const husWithTests = plan.hus.filter(
|
|
94
|
+
(hu) => Array.isArray(hu?.acceptance_tests) && hu.acceptance_tests.length > 0,
|
|
95
|
+
);
|
|
96
|
+
if (husWithTests.length === 0) return null;
|
|
97
|
+
|
|
98
|
+
let covered = 0;
|
|
99
|
+
for (const hu of husWithTests) {
|
|
100
|
+
const huCommits = (commits || []).filter(
|
|
101
|
+
(c) => attributeCommitToHu(c, [hu]) === hu.id,
|
|
102
|
+
);
|
|
103
|
+
const hasTestCommit = huCommits.some((c) =>
|
|
104
|
+
/^test[:(]/i.test(String(c.message || "").trim()),
|
|
105
|
+
);
|
|
106
|
+
if (hasTestCommit) covered += 1;
|
|
107
|
+
}
|
|
108
|
+
return Math.round((covered / husWithTests.length) * 100);
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
/**
|
|
112
|
+
* Component 3: scope discipline. For each file changed by the run, we
|
|
113
|
+
* try to match it against at least one HU's id slug, title slug, or
|
|
114
|
+
* `scope` text. A file that matches no HU is flagged as out-of-scope.
|
|
115
|
+
* Returns null when the run changed no files.
|
|
116
|
+
*/
|
|
117
|
+
export function computeScopeDiscipline(filesChanged, plan) {
|
|
118
|
+
if (!Array.isArray(filesChanged) || filesChanged.length === 0) return null;
|
|
119
|
+
if (!plan?.hus || plan.hus.length === 0) return null;
|
|
120
|
+
const huKeywords = plan.hus.map((hu) => ({
|
|
121
|
+
id: hu.id,
|
|
122
|
+
matchers: [
|
|
123
|
+
slugify(hu.id),
|
|
124
|
+
slugify(hu.title),
|
|
125
|
+
// Pull alpha-rich words from `scope` text (length >= 4).
|
|
126
|
+
...String(hu.scope || "")
|
|
127
|
+
.toLowerCase()
|
|
128
|
+
.split(/[^a-z]+/)
|
|
129
|
+
.filter((w) => w.length >= 4),
|
|
130
|
+
].filter(Boolean),
|
|
131
|
+
}));
|
|
132
|
+
let inScope = 0;
|
|
133
|
+
for (const file of filesChanged) {
|
|
134
|
+
const slug = slugify(file);
|
|
135
|
+
const matched = huKeywords.some((hk) =>
|
|
136
|
+
hk.matchers.some((m) => slug.includes(m) || file.includes(m)),
|
|
137
|
+
);
|
|
138
|
+
if (matched) inScope += 1;
|
|
139
|
+
}
|
|
140
|
+
return Math.round((inScope / filesChanged.length) * 100);
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
/**
|
|
144
|
+
* Component 4: dependency order. For every `(parent, child)` edge
|
|
145
|
+
* declared via `child.blocked_by`, check that the EARLIEST commit
|
|
146
|
+
* attributed to `parent` is older than the EARLIEST commit attributed
|
|
147
|
+
* to `child`. Returns null when no dependency edges exist.
|
|
148
|
+
*
|
|
149
|
+
* `commits` are expected oldest-first (the shape `listCommitsBetween`
|
|
150
|
+
* returns). When that's not the case the score still makes sense — we
|
|
151
|
+
* just compare relative positions in the array.
|
|
152
|
+
*/
|
|
153
|
+
export function computeDependencyOrder(commits, plan) {
|
|
154
|
+
if (!plan?.hus || plan.hus.length === 0) return null;
|
|
155
|
+
const edges = [];
|
|
156
|
+
for (const hu of plan.hus) {
|
|
157
|
+
for (const parentId of hu.blocked_by || []) {
|
|
158
|
+
edges.push([parentId, hu.id]);
|
|
159
|
+
}
|
|
160
|
+
}
|
|
161
|
+
if (edges.length === 0) return null;
|
|
162
|
+
if (!Array.isArray(commits) || commits.length === 0) return null;
|
|
163
|
+
|
|
164
|
+
// Build huId → first commit index (oldest first attribution).
|
|
165
|
+
const firstIndex = new Map();
|
|
166
|
+
commits.forEach((commit, idx) => {
|
|
167
|
+
const huId = attributeCommitToHu(commit, plan.hus);
|
|
168
|
+
if (huId && !firstIndex.has(huId)) firstIndex.set(huId, idx);
|
|
169
|
+
});
|
|
170
|
+
|
|
171
|
+
let respected = 0;
|
|
172
|
+
let evaluable = 0;
|
|
173
|
+
for (const [parentId, childId] of edges) {
|
|
174
|
+
const p = firstIndex.get(parentId);
|
|
175
|
+
const c = firstIndex.get(childId);
|
|
176
|
+
if (p === undefined || c === undefined) continue; // can't evaluate
|
|
177
|
+
evaluable += 1;
|
|
178
|
+
if (p < c) respected += 1;
|
|
179
|
+
}
|
|
180
|
+
if (evaluable === 0) return null;
|
|
181
|
+
return Math.round((respected / evaluable) * 100);
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
/**
|
|
185
|
+
* Aggregator: compute every component and weight them. Components
|
|
186
|
+
* that return `null` are excluded and their weight redistributed
|
|
187
|
+
* proportionally across the rest, so a run that legitimately has no
|
|
188
|
+
* acceptance tests or no dependencies isn't penalised.
|
|
189
|
+
*
|
|
190
|
+
* Returns the full structured result (see file header for shape).
|
|
191
|
+
*/
|
|
192
|
+
export function computePlanAdherenceScore({ commits, plan, filesChanged }) {
|
|
193
|
+
const breakdown = {
|
|
194
|
+
commit_attribution: computeCommitAttribution(commits, plan),
|
|
195
|
+
acceptance_tests: computeAcceptanceTestCoverage(commits, plan),
|
|
196
|
+
scope_discipline: computeScopeDiscipline(filesChanged, plan),
|
|
197
|
+
dependency_order: computeDependencyOrder(commits, plan),
|
|
198
|
+
};
|
|
199
|
+
|
|
200
|
+
// Weighted average over non-null components only.
|
|
201
|
+
let totalWeight = 0;
|
|
202
|
+
let weightedSum = 0;
|
|
203
|
+
for (const [key, weight] of Object.entries(WEIGHTS)) {
|
|
204
|
+
const value = breakdown[key];
|
|
205
|
+
if (value === null || value === undefined) continue;
|
|
206
|
+
totalWeight += weight;
|
|
207
|
+
weightedSum += value * weight;
|
|
208
|
+
}
|
|
209
|
+
const score = totalWeight === 0 ? null : Math.round(weightedSum / totalWeight);
|
|
210
|
+
|
|
211
|
+
// Per-HU detail: was the HU "touched" (attributed at least one commit)?
|
|
212
|
+
const hu_scores = (plan?.hus || []).map((hu) => {
|
|
213
|
+
const huCommits = (commits || []).filter(
|
|
214
|
+
(c) => attributeCommitToHu(c, [hu]) === hu.id,
|
|
215
|
+
);
|
|
216
|
+
const issues = [];
|
|
217
|
+
if (huCommits.length === 0) issues.push("no commit attributed");
|
|
218
|
+
return { huId: hu.id, attributed: huCommits.length > 0, issues };
|
|
219
|
+
});
|
|
220
|
+
|
|
221
|
+
return { score, breakdown, hu_scores };
|
|
222
|
+
}
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Golden tasks asserter (KJC-TSK-0374). Parses a `summary.md` produced
|
|
3
|
+
* by a real `kj run` and compares it against a GoldenTask's structural
|
|
4
|
+
* expectations. Pure: no I/O, no subprocess. Subprocess driver lands
|
|
5
|
+
* in PR-3a; the 3 fixtures + baseline + spec land in PR-3b.
|
|
6
|
+
*
|
|
7
|
+
* Three of the five assertion families live here (the ones derivable
|
|
8
|
+
* from summary.md alone): expected_commits_min, expected_audit_status,
|
|
9
|
+
* expected_plan_adherence_min. expected_test_files and
|
|
10
|
+
* allowed_loc_range need filesystem access — PR-3a wires those.
|
|
11
|
+
*/
|
|
12
|
+
|
|
13
|
+
const COMMIT_LINE = /^- `([^`]+)`\s+(.*)$/;
|
|
14
|
+
const SCORE_LINE = /^\*\*Score\*\*:\s*(\d+)\s*\/\s*100/m;
|
|
15
|
+
const AUDIT_ROW = /^\|\s*`audit`\s*\|\s*([^|]+?)\s*\|/m;
|
|
16
|
+
|
|
17
|
+
function extractSection(content, heading) {
|
|
18
|
+
const re = new RegExp(`^## ${heading}\\s*$([\\s\\S]*?)(?=^## |\\Z)`, "m");
|
|
19
|
+
const m = content.match(re);
|
|
20
|
+
return m ? m[1].trim() : "";
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
function parseCommits(content) {
|
|
24
|
+
const section = extractSection(content, "Commits");
|
|
25
|
+
if (!section || /No commits/.test(section)) return [];
|
|
26
|
+
return section
|
|
27
|
+
.split("\n")
|
|
28
|
+
.map((line) => line.match(COMMIT_LINE))
|
|
29
|
+
.filter(Boolean)
|
|
30
|
+
.map(([, hash, message]) => ({ hash, message }));
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
function parseAuditStatus(content) {
|
|
34
|
+
const m = content.match(AUDIT_ROW);
|
|
35
|
+
if (!m) return null;
|
|
36
|
+
const cell = m[1];
|
|
37
|
+
if (/✅|pass/i.test(cell)) return "pass";
|
|
38
|
+
if (/❌|fail/i.test(cell)) return "fail";
|
|
39
|
+
if (/⚠|warn/i.test(cell)) return "warning";
|
|
40
|
+
return null;
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
function parsePlanAdherence(content) {
|
|
44
|
+
const section = extractSection(content, "Plan adherence");
|
|
45
|
+
if (!section) return null;
|
|
46
|
+
const m = section.match(SCORE_LINE);
|
|
47
|
+
if (!m) return null;
|
|
48
|
+
const score = Number(m[1]);
|
|
49
|
+
return Number.isFinite(score) ? score : null;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
export function parseSummary(content) {
|
|
53
|
+
const text = String(content || "");
|
|
54
|
+
return {
|
|
55
|
+
commits: parseCommits(text),
|
|
56
|
+
auditStatus: parseAuditStatus(text),
|
|
57
|
+
planAdherenceScore: parsePlanAdherence(text),
|
|
58
|
+
};
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
export function assertFromSummary(task, parsed) {
|
|
62
|
+
const failures = [];
|
|
63
|
+
|
|
64
|
+
const fail = (kind, expected, actual, message) => failures.push({ kind, expected, actual, message });
|
|
65
|
+
|
|
66
|
+
if (parsed.commits.length < task.expected_commits_min) {
|
|
67
|
+
fail("commits_min", `>= ${task.expected_commits_min}`, parsed.commits.length,
|
|
68
|
+
`expected at least ${task.expected_commits_min} commit(s); got ${parsed.commits.length}`);
|
|
69
|
+
}
|
|
70
|
+
|
|
71
|
+
if (parsed.auditStatus !== task.expected_audit_status) {
|
|
72
|
+
const got = parsed.auditStatus === null ? "no audit row in summary" : `"${parsed.auditStatus}"`;
|
|
73
|
+
fail("audit_status", task.expected_audit_status, parsed.auditStatus,
|
|
74
|
+
`expected audit "${task.expected_audit_status}"; got ${got}`);
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
if (typeof task.expected_plan_adherence_min === "number") {
|
|
78
|
+
const score = parsed.planAdherenceScore;
|
|
79
|
+
const min = task.expected_plan_adherence_min;
|
|
80
|
+
if (score === null) {
|
|
81
|
+
fail("plan_adherence_missing", `>= ${min}`, null,
|
|
82
|
+
`expected plan adherence >= ${min}; summary has no Plan adherence section`);
|
|
83
|
+
} else if (score < min) {
|
|
84
|
+
fail("plan_adherence_below", `>= ${min}`, score,
|
|
85
|
+
`expected plan adherence >= ${min}; got ${score}`);
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
return { ok: failures.length === 0, failures };
|
|
90
|
+
}
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Golden tasks loader (KJC-TSK-0374). Reads + validates JSON files
|
|
3
|
+
* under a directory; aggregates errors; rejects duplicate ids.
|
|
4
|
+
*/
|
|
5
|
+
|
|
6
|
+
import fs from "node:fs/promises";
|
|
7
|
+
import path from "node:path";
|
|
8
|
+
import { safeParseGoldenTask } from "./schema.js";
|
|
9
|
+
|
|
10
|
+
export async function loadGoldenTasks(dir) {
|
|
11
|
+
const entries = await fs.readdir(dir, { withFileTypes: true }).catch((err) => {
|
|
12
|
+
throw new Error(`golden tasks directory not readable: ${dir} (${err.code || err.message})`);
|
|
13
|
+
});
|
|
14
|
+
const jsonFiles = entries
|
|
15
|
+
.filter((e) => e.isFile() && e.name.endsWith(".json"))
|
|
16
|
+
.map((e) => path.join(dir, e.name))
|
|
17
|
+
.sort();
|
|
18
|
+
|
|
19
|
+
const tasks = [];
|
|
20
|
+
const errors = [];
|
|
21
|
+
const seenIds = new Map();
|
|
22
|
+
|
|
23
|
+
for (const filePath of jsonFiles) {
|
|
24
|
+
let raw;
|
|
25
|
+
try {
|
|
26
|
+
const content = await fs.readFile(filePath, "utf8");
|
|
27
|
+
raw = JSON.parse(content);
|
|
28
|
+
} catch (err) {
|
|
29
|
+
errors.push(`${path.basename(filePath)}: invalid JSON (${err.message})`);
|
|
30
|
+
continue;
|
|
31
|
+
}
|
|
32
|
+
const result = safeParseGoldenTask(raw);
|
|
33
|
+
if (!result.ok) {
|
|
34
|
+
const summary = result.issues.map((iss) => ` - ${iss.path || "(root)"}: ${iss.message}`).join("\n");
|
|
35
|
+
errors.push(`${path.basename(filePath)}:\n${summary}`);
|
|
36
|
+
continue;
|
|
37
|
+
}
|
|
38
|
+
const existing = seenIds.get(result.task.id);
|
|
39
|
+
if (existing) {
|
|
40
|
+
errors.push(`${path.basename(filePath)}: duplicate id "${result.task.id}" (also in ${existing})`);
|
|
41
|
+
continue;
|
|
42
|
+
}
|
|
43
|
+
seenIds.set(result.task.id, path.basename(filePath));
|
|
44
|
+
tasks.push(result.task);
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
if (errors.length > 0) {
|
|
48
|
+
throw new Error(`golden tasks failed validation:\n${errors.join("\n")}`);
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
return { tasks };
|
|
52
|
+
}
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Golden tasks subprocess runner (KJC-TSK-0374). Spawns `kj run` in
|
|
3
|
+
* a sandbox dir, reads back summary.md, evaluates the schema's
|
|
4
|
+
* filesystem-dependent assertions (expected_test_files, allowed_loc_range),
|
|
5
|
+
* and merges them with `assertFromSummary` into a single result.
|
|
6
|
+
*
|
|
7
|
+
* The kj subprocess + diff calls are injected so unit tests can drive
|
|
8
|
+
* the runner without spawning anything (real e2e is opt-in via PR-3b).
|
|
9
|
+
*/
|
|
10
|
+
|
|
11
|
+
import fs from "node:fs/promises";
|
|
12
|
+
import path from "node:path";
|
|
13
|
+
import { runCommand as defaultRun } from "../utils/process.js";
|
|
14
|
+
import { parseSummary, assertFromSummary } from "./asserter.js";
|
|
15
|
+
|
|
16
|
+
async function findLatestSummary(workDir) {
|
|
17
|
+
const journalRoot = path.join(workDir, ".karajan", "sessions");
|
|
18
|
+
const sessions = await fs.readdir(journalRoot).catch(() => []);
|
|
19
|
+
if (sessions.length === 0) return null;
|
|
20
|
+
const stats = await Promise.all(sessions.map(async (id) => {
|
|
21
|
+
const file = path.join(journalRoot, id, "summary.md");
|
|
22
|
+
const stat = await fs.stat(file).catch(() => null);
|
|
23
|
+
return stat ? { file, mtimeMs: stat.mtimeMs } : null;
|
|
24
|
+
}));
|
|
25
|
+
const valid = stats.filter(Boolean).sort((a, b) => b.mtimeMs - a.mtimeMs);
|
|
26
|
+
return valid[0]?.file ?? null;
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
async function findMissingTestFiles(workDir, paths) {
|
|
30
|
+
const checks = await Promise.all(paths.map(async (rel) =>
|
|
31
|
+
[rel, await fs.stat(path.join(workDir, rel)).then(() => true).catch(() => false)]));
|
|
32
|
+
return checks.filter(([, ok]) => !ok).map(([rel]) => rel);
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
async function totalLocSince(workDir, baseSha, run) {
|
|
36
|
+
const res = await run("git", ["diff", "--numstat", `${baseSha}..HEAD`], { cwd: workDir }).catch(() => null);
|
|
37
|
+
if (!res || res.exitCode !== 0) return null;
|
|
38
|
+
return String(res.stdout || "").split("\n").filter(Boolean)
|
|
39
|
+
.map((l) => l.split("\t")).filter(([a]) => a !== "-")
|
|
40
|
+
.reduce((sum, [a]) => sum + Number(a || 0), 0);
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export async function runGoldenTask(task, opts = {}) {
|
|
44
|
+
const run = opts.run || defaultRun;
|
|
45
|
+
const fsApi = opts.fs || fs;
|
|
46
|
+
const workDir = opts.workDir;
|
|
47
|
+
if (!workDir) throw new Error("runGoldenTask requires opts.workDir");
|
|
48
|
+
|
|
49
|
+
const baseSha = opts.baseSha || null;
|
|
50
|
+
const kjResult = await run("kj", ["run", task.prompt, "-y"], {
|
|
51
|
+
cwd: workDir,
|
|
52
|
+
timeout: (task.timeout_minutes || 30) * 60_000,
|
|
53
|
+
});
|
|
54
|
+
|
|
55
|
+
const summaryPath = opts.summaryPath || await findLatestSummary(workDir);
|
|
56
|
+
if (!summaryPath) {
|
|
57
|
+
return { ok: false, kjExit: kjResult?.exitCode ?? -1, failures: [{
|
|
58
|
+
kind: "no_summary", expected: "summary.md", actual: null,
|
|
59
|
+
message: "kj run finished without producing a summary.md",
|
|
60
|
+
}] };
|
|
61
|
+
}
|
|
62
|
+
const content = await fsApi.readFile(summaryPath, "utf8");
|
|
63
|
+
const parsed = parseSummary(content);
|
|
64
|
+
const summaryAssertion = assertFromSummary(task, parsed);
|
|
65
|
+
|
|
66
|
+
const failures = [...summaryAssertion.failures];
|
|
67
|
+
|
|
68
|
+
if (Array.isArray(task.expected_test_files) && task.expected_test_files.length > 0) {
|
|
69
|
+
const missing = await findMissingTestFiles(workDir, task.expected_test_files);
|
|
70
|
+
if (missing.length > 0) {
|
|
71
|
+
failures.push({ kind: "test_files_missing", expected: task.expected_test_files, actual: missing,
|
|
72
|
+
message: `expected test files not produced: ${missing.join(", ")}` });
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
if (baseSha) {
|
|
77
|
+
const loc = await totalLocSince(workDir, baseSha, run);
|
|
78
|
+
const [min, max] = task.allowed_loc_range;
|
|
79
|
+
if (loc !== null && (loc < min || loc > max)) {
|
|
80
|
+
failures.push({ kind: "loc_out_of_range", expected: `${min}..${max}`, actual: loc,
|
|
81
|
+
message: `LOC added (${loc}) is outside the allowed range [${min}, ${max}]` });
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
return {
|
|
86
|
+
ok: failures.length === 0,
|
|
87
|
+
kjExit: kjResult?.exitCode ?? null,
|
|
88
|
+
summaryPath,
|
|
89
|
+
parsed,
|
|
90
|
+
failures,
|
|
91
|
+
};
|
|
92
|
+
}
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Golden tasks regression suite — schema (KJC-TSK-0374). Pure data,
|
|
3
|
+
* no I/O. See docs/golden-tasks.md (PR-3) for the full spec.
|
|
4
|
+
*/
|
|
5
|
+
|
|
6
|
+
import * as v from "valibot";
|
|
7
|
+
|
|
8
|
+
const NonEmptyString = v.pipe(v.string(), v.minLength(1));
|
|
9
|
+
const Percent = v.pipe(v.number(), v.minValue(0), v.maxValue(100));
|
|
10
|
+
|
|
11
|
+
const NonNegInt = v.pipe(v.number(), v.integer(), v.minValue(0));
|
|
12
|
+
const LocRangeSchema = v.pipe(v.array(NonNegInt), v.length(2),
|
|
13
|
+
v.check((arr) => arr[0] <= arr[1], "allowed_loc_range: min must be <= max"));
|
|
14
|
+
|
|
15
|
+
export const GoldenTaskSchema = v.looseObject({
|
|
16
|
+
id: v.pipe(NonEmptyString, v.regex(/^[a-z0-9_-]+$/, "id must be lowercase a-z0-9_-")),
|
|
17
|
+
title: NonEmptyString,
|
|
18
|
+
prompt: NonEmptyString,
|
|
19
|
+
description: v.optional(v.string()),
|
|
20
|
+
timeout_minutes: v.optional(v.pipe(v.number(), v.integer(), v.minValue(1)), 30),
|
|
21
|
+
expected_commits_min: NonNegInt,
|
|
22
|
+
expected_audit_status: v.picklist(["pass", "warning", "fail"]),
|
|
23
|
+
expected_test_files: v.array(NonEmptyString),
|
|
24
|
+
allowed_loc_range: LocRangeSchema,
|
|
25
|
+
expected_plan_adherence_min: v.optional(Percent),
|
|
26
|
+
});
|
|
27
|
+
|
|
28
|
+
/** @typedef {import("valibot").InferOutput<typeof GoldenTaskSchema>} GoldenTask */
|
|
29
|
+
|
|
30
|
+
export function parseGoldenTask(raw) {
|
|
31
|
+
return v.parse(GoldenTaskSchema, raw);
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
export function safeParseGoldenTask(raw) {
|
|
35
|
+
const result = v.safeParse(GoldenTaskSchema, raw);
|
|
36
|
+
if (result.success) return { ok: true, task: result.output };
|
|
37
|
+
return { ok: false, issues: result.issues.map((iss) => ({
|
|
38
|
+
path: (iss.path || []).map((p) => p.key).join("."),
|
|
39
|
+
message: iss.message,
|
|
40
|
+
})) };
|
|
41
|
+
}
|
|
@@ -23,7 +23,9 @@
|
|
|
23
23
|
|
|
24
24
|
import { generateDiff } from "../../review/diff-generator.js";
|
|
25
25
|
import { finalizeGitAutomation } from "../../git/automation.js";
|
|
26
|
-
import { listCommitsBetween } from "../../utils/git.js";
|
|
26
|
+
import { listCommitsBetween, listFilesChangedSince } from "../../utils/git.js";
|
|
27
|
+
import { loadPlan, projectSlug as slugFor } from "../../plan/plan-store.js";
|
|
28
|
+
import { computePlanAdherenceScore } from "../../audit/plan-adherence.js";
|
|
27
29
|
import { saveSession, markSessionStatus } from "../../session/store.js";
|
|
28
30
|
import {
|
|
29
31
|
setReviewerFeedback, setBudget, setRtkSavings, resetRetryCount,
|
|
@@ -166,6 +168,36 @@ export async function finalizeApprovedSession({ config, gitCtx, task, logger, se
|
|
|
166
168
|
try { allCommits = (await listCommitsBetween(session.head_at_start || session.session_start_sha)) || []; }
|
|
167
169
|
catch { allCommits = gitResult?.commits || []; }
|
|
168
170
|
const summaryCommits = allCommits.length > 0 ? allCommits : (gitResult?.commits || []);
|
|
171
|
+
|
|
172
|
+
// KJC-TSK-0376 — plan adherence metric (deepeval-inspired). When the
|
|
173
|
+
// run executed against a known plan (`kj run --plan <id>`), score how
|
|
174
|
+
// faithfully the coder followed it. Pure offline; safe-by-default
|
|
175
|
+
// (try/catch swallows missing helper or unreadable plan).
|
|
176
|
+
let planAdherence = null;
|
|
177
|
+
try {
|
|
178
|
+
const planId = session._planRef?.planId
|
|
179
|
+
|| stageResults?.huReviewer?.planId
|
|
180
|
+
|| null;
|
|
181
|
+
if (planId) {
|
|
182
|
+
const projectDir = config?.projectDir
|
|
183
|
+
|| session.config_snapshot?.projectDir
|
|
184
|
+
|| process.cwd();
|
|
185
|
+
const plan = await loadPlan(projectDir, planId).catch(() => null);
|
|
186
|
+
if (plan && Array.isArray(plan.hus) && plan.hus.length > 0) {
|
|
187
|
+
const filesChanged = await listFilesChangedSince(
|
|
188
|
+
session.head_at_start || session.session_start_sha,
|
|
189
|
+
).catch(() => []);
|
|
190
|
+
planAdherence = computePlanAdherenceScore({
|
|
191
|
+
commits: summaryCommits,
|
|
192
|
+
plan,
|
|
193
|
+
filesChanged,
|
|
194
|
+
});
|
|
195
|
+
}
|
|
196
|
+
}
|
|
197
|
+
} catch (err) {
|
|
198
|
+
logger?.warn?.(`Plan adherence calculation skipped: ${err.message}`);
|
|
199
|
+
}
|
|
200
|
+
|
|
169
201
|
await writeSummaryJournalV2(journalDir, {
|
|
170
202
|
task: session.task,
|
|
171
203
|
result: "APPROVED",
|
|
@@ -188,6 +220,7 @@ export async function finalizeApprovedSession({ config, gitCtx, task, logger, se
|
|
|
188
220
|
installed: session.autoInstalledSkills,
|
|
189
221
|
recommended: session.skillsRecommended,
|
|
190
222
|
},
|
|
223
|
+
planAdherence,
|
|
191
224
|
}, { logger });
|
|
192
225
|
logger.info(`Session journal written to ${journalDir}`);
|
|
193
226
|
} catch (err) {
|
|
@@ -361,7 +394,6 @@ export async function tryAutoStartBoard(config, logger, emitter, eventBase) {
|
|
|
361
394
|
|
|
362
395
|
try {
|
|
363
396
|
const { startBoard, renderBoardBanner } = await import("../../commands/board.js");
|
|
364
|
-
const { projectSlug: slugFor } = await import("../../plan/plan-store.js");
|
|
365
397
|
const boardPort = config.hu_board.port || 4000;
|
|
366
398
|
// Scope the board URL to the current run's project (`/p/<slug>`) so
|
|
367
399
|
// the user lands on a filtered view, not the global dashboard.
|
|
@@ -199,11 +199,50 @@ export function buildSummaryMarkdown(input) {
|
|
|
199
199
|
|
|
200
200
|
sections.push("", "## Commits", "", renderCommits(input.commits));
|
|
201
201
|
|
|
202
|
+
// KJC-TSK-0376 — plan adherence metric (deepeval-inspired). Only
|
|
203
|
+
// rendered when the run executed against a known plan and the
|
|
204
|
+
// calculation produced a non-null score (a run with no commits or
|
|
205
|
+
// no files changed yields nulls everywhere → omit the section).
|
|
206
|
+
const planAdherenceSection = renderPlanAdherence(input.planAdherence);
|
|
207
|
+
if (planAdherenceSection) {
|
|
208
|
+
sections.push("", "## Plan adherence", "", planAdherenceSection);
|
|
209
|
+
}
|
|
210
|
+
|
|
202
211
|
sections.push("", "## Journal files", "", renderJournalLinks(input.files));
|
|
203
212
|
|
|
204
213
|
return sections.join("\n") + "\n";
|
|
205
214
|
}
|
|
206
215
|
|
|
216
|
+
function renderPlanAdherence(planAdherence) {
|
|
217
|
+
if (!planAdherence || typeof planAdherence !== "object") return "";
|
|
218
|
+
if (planAdherence.score === null || planAdherence.score === undefined) return "";
|
|
219
|
+
const lines = [`**Score**: ${planAdherence.score}/100`, ""];
|
|
220
|
+
const breakdown = planAdherence.breakdown || {};
|
|
221
|
+
lines.push("| Component | Score | Weight |");
|
|
222
|
+
lines.push("| --- | --- | --- |");
|
|
223
|
+
const rows = [
|
|
224
|
+
["Commit attribution", breakdown.commit_attribution, "40%"],
|
|
225
|
+
["Acceptance tests", breakdown.acceptance_tests, "30%"],
|
|
226
|
+
["Scope discipline", breakdown.scope_discipline, "20%"],
|
|
227
|
+
["Dependency order", breakdown.dependency_order, "10%"],
|
|
228
|
+
];
|
|
229
|
+
for (const [label, value, weight] of rows) {
|
|
230
|
+
const cell = value === null || value === undefined ? "n/a" : `${value}/100`;
|
|
231
|
+
lines.push(`| ${label} | ${cell} | ${weight} |`);
|
|
232
|
+
}
|
|
233
|
+
const huScores = Array.isArray(planAdherence.hu_scores) ? planAdherence.hu_scores : [];
|
|
234
|
+
if (huScores.length > 0) {
|
|
235
|
+
const unattributed = huScores.filter((h) => !h.attributed);
|
|
236
|
+
if (unattributed.length > 0) {
|
|
237
|
+
lines.push("", `_${unattributed.length} HU(s) without an attributed commit:_`);
|
|
238
|
+
for (const h of unattributed) {
|
|
239
|
+
lines.push(`- \`${h.huId}\``);
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
}
|
|
243
|
+
return lines.join("\n");
|
|
244
|
+
}
|
|
245
|
+
|
|
207
246
|
/**
|
|
208
247
|
* Write summary.md to the given journal directory. Always writes (the file
|
|
209
248
|
* is the entry point; even a barebones session should have one).
|
package/src/utils/git.js
CHANGED
|
@@ -80,6 +80,33 @@ export async function revParse(ref) {
|
|
|
80
80
|
* @param {string} [toRef="HEAD"]
|
|
81
81
|
* @returns {Promise<Array<{hash: string, message: string}>>}
|
|
82
82
|
*/
|
|
83
|
+
/**
|
|
84
|
+
* List the file paths changed in the range `fromSha..toRef`. Used by the
|
|
85
|
+
* plan-adherence metric (KJC-TSK-0376) to score whether the coder's
|
|
86
|
+
* file changes fall within any HU's declared scope.
|
|
87
|
+
*
|
|
88
|
+
* Returns a deduplicated array of file paths, oldest commit's files
|
|
89
|
+
* first. Defensive: returns [] on missing `fromSha`, non-zero git exit,
|
|
90
|
+
* or thrown error — consistent with `listCommitsBetween`'s contract.
|
|
91
|
+
*
|
|
92
|
+
* @param {string|null|undefined} fromSha
|
|
93
|
+
* @param {string} [toRef="HEAD"]
|
|
94
|
+
* @returns {Promise<string[]>}
|
|
95
|
+
*/
|
|
96
|
+
export async function listFilesChangedSince(fromSha, toRef = "HEAD") {
|
|
97
|
+
if (!fromSha) return [];
|
|
98
|
+
const result = await run(
|
|
99
|
+
"git",
|
|
100
|
+
["diff", "--name-only", `${fromSha}..${toRef}`],
|
|
101
|
+
).catch(() => null);
|
|
102
|
+
if (!result || result.exitCode !== 0) return [];
|
|
103
|
+
const lines = String(result.stdout || "")
|
|
104
|
+
.split("\n")
|
|
105
|
+
.map((line) => line.trim())
|
|
106
|
+
.filter(Boolean);
|
|
107
|
+
return Array.from(new Set(lines));
|
|
108
|
+
}
|
|
109
|
+
|
|
83
110
|
export async function listCommitsBetween(fromSha, toRef = "HEAD") {
|
|
84
111
|
if (!fromSha) return [];
|
|
85
112
|
const result = await run(
|
|
@@ -63,3 +63,12 @@ Return a single valid JSON object and nothing else.
|
|
|
63
63
|
|
|
64
64
|
If the architecture is fully defined, `verdict: "ready"` and `questions: []`.
|
|
65
65
|
</output_format>
|
|
66
|
+
|
|
67
|
+
## PR atomicity (hard project rule)
|
|
68
|
+
|
|
69
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
70
|
+
|
|
71
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
72
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
73
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
74
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -61,3 +61,12 @@ If the architecture is fully defined with no open questions, return `verdict: "r
|
|
|
61
61
|
- Document WHY each pattern was chosen, not just WHAT
|
|
62
62
|
- If research context is provided, use it to inform architectural decisions
|
|
63
63
|
- Never invent requirements — if something is unclear, add it to questions
|
|
64
|
+
|
|
65
|
+
## PR atomicity (hard project rule)
|
|
66
|
+
|
|
67
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
68
|
+
|
|
69
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
70
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
71
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
72
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -50,3 +50,12 @@ Questions MUST include which isolation level (shared DB + tenant_id column vs sc
|
|
|
50
50
|
## Output (strict JSON)
|
|
51
51
|
|
|
52
52
|
Same shape as Example 1. Exactly one top-level object. No prose around it.
|
|
53
|
+
|
|
54
|
+
## PR atomicity (hard project rule)
|
|
55
|
+
|
|
56
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
57
|
+
|
|
58
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
59
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
60
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
61
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -49,3 +49,12 @@ You are the Architect. Design the technical architecture BEFORE implementation.
|
|
|
49
49
|
```
|
|
50
50
|
|
|
51
51
|
If fully defined → `verdict: "ready"` and `questions: []`.
|
|
52
|
+
|
|
53
|
+
## PR atomicity (hard project rule)
|
|
54
|
+
|
|
55
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
56
|
+
|
|
57
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
58
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
59
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
60
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -48,3 +48,12 @@ Rules for the JSON:
|
|
|
48
48
|
- Exact keys only. No extras.
|
|
49
49
|
- All arrays may be empty but never null.
|
|
50
50
|
- If `verdict` is "needs_clarification", `questions` must be non-empty.
|
|
51
|
+
|
|
52
|
+
## PR atomicity (hard project rule)
|
|
53
|
+
|
|
54
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
55
|
+
|
|
56
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
57
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
58
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
59
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -61,3 +61,12 @@ If the architecture is fully defined with no open questions, return `verdict: "r
|
|
|
61
61
|
- Document WHY each pattern was chosen, not just WHAT
|
|
62
62
|
- If research context is provided, use it to inform architectural decisions
|
|
63
63
|
- Never invent requirements — if something is unclear, add it to questions
|
|
64
|
+
|
|
65
|
+
## PR atomicity (hard project rule)
|
|
66
|
+
|
|
67
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
68
|
+
|
|
69
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
70
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
71
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
72
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -70,3 +70,12 @@ Return a single JSON object matching this schema:
|
|
|
70
70
|
}
|
|
71
71
|
```
|
|
72
72
|
</output_format>
|
|
73
|
+
|
|
74
|
+
## PR atomicity (hard project rule)
|
|
75
|
+
|
|
76
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
77
|
+
|
|
78
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
79
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
80
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
81
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -67,3 +67,12 @@ Return a JSON object:
|
|
|
67
67
|
"summary": "Human-readable summary of changes"
|
|
68
68
|
}
|
|
69
69
|
```
|
|
70
|
+
|
|
71
|
+
## PR atomicity (hard project rule)
|
|
72
|
+
|
|
73
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
74
|
+
|
|
75
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
76
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
77
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
78
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -76,3 +76,12 @@ An incomplete implementation is worse than an error — don't claim success if p
|
|
|
76
76
|
"summary": "Human-readable summary of changes"
|
|
77
77
|
}
|
|
78
78
|
```
|
|
79
|
+
|
|
80
|
+
## PR atomicity (hard project rule)
|
|
81
|
+
|
|
82
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
83
|
+
|
|
84
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
85
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
86
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
87
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -57,3 +57,12 @@ Return JSON:
|
|
|
57
57
|
"summary": "Human-readable summary of changes"
|
|
58
58
|
}
|
|
59
59
|
```
|
|
60
|
+
|
|
61
|
+
## PR atomicity (hard project rule)
|
|
62
|
+
|
|
63
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
64
|
+
|
|
65
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
66
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
67
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
68
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -47,3 +47,12 @@ Rules for the JSON:
|
|
|
47
47
|
- `approach` is one short sentence.
|
|
48
48
|
- `summary` is one short sentence.
|
|
49
49
|
- No extra keys. No text before or after the JSON.
|
|
50
|
+
|
|
51
|
+
## PR atomicity (hard project rule)
|
|
52
|
+
|
|
53
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
54
|
+
|
|
55
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
56
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
57
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
58
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
package/templates/roles/coder.md
CHANGED
|
@@ -12,6 +12,15 @@ You are the **Coder** in a multi-role AI pipeline. Your job is to write code and
|
|
|
12
12
|
- Before creating a new utility or helper, check if a similar one already exists in the codebase. Reuse existing code over creating duplicates.
|
|
13
13
|
- Follow existing code conventions and patterns in the repository.
|
|
14
14
|
|
|
15
|
+
## PR atomicity (hard project rule)
|
|
16
|
+
|
|
17
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan before writing:
|
|
18
|
+
|
|
19
|
+
- Stay under **~150 LOC per iteration** (safety margin against the 200 hard limit).
|
|
20
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
21
|
+
- If the task clearly requires >150 LOC, STOP and report the partition needed. Don't ship a 500-LOC PR — the gate rejects it and the work is redone, burning tokens twice.
|
|
22
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
23
|
+
|
|
15
24
|
## Task completeness
|
|
16
25
|
|
|
17
26
|
Before reporting done, verify that ALL parts of the task are addressed:
|
|
@@ -48,3 +48,12 @@ Any of:
|
|
|
48
48
|
}
|
|
49
49
|
```
|
|
50
50
|
</output_format>
|
|
51
|
+
|
|
52
|
+
## PR atomicity (hard project rule)
|
|
53
|
+
|
|
54
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
55
|
+
|
|
56
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
57
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
58
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
59
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -46,3 +46,12 @@ You are the **Planner** in a multi-role AI pipeline. Your job is to create an im
|
|
|
46
46
|
"summary": "Plan: 4 steps, estimated 2 files modified, 1 new file"
|
|
47
47
|
}
|
|
48
48
|
```
|
|
49
|
+
|
|
50
|
+
## PR atomicity (hard project rule)
|
|
51
|
+
|
|
52
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
53
|
+
|
|
54
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
55
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
56
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
57
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -64,3 +64,12 @@ Approach must cite concurrency model, queue semantics, shutdown. Steps include i
|
|
|
64
64
|
"summary": "Plan: N steps, M files modified, K new files"
|
|
65
65
|
}
|
|
66
66
|
```
|
|
67
|
+
|
|
68
|
+
## PR atomicity (hard project rule)
|
|
69
|
+
|
|
70
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
71
|
+
|
|
72
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
73
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
74
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
75
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -44,3 +44,12 @@ You are the Planner. Produce an implementation plan for the task.
|
|
|
44
44
|
"summary": "Plan: 4 steps, estimated 2 files modified, 1 new file"
|
|
45
45
|
}
|
|
46
46
|
```
|
|
47
|
+
|
|
48
|
+
## PR atomicity (hard project rule)
|
|
49
|
+
|
|
50
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
51
|
+
|
|
52
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
53
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
54
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
55
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -51,3 +51,12 @@ Rules for the JSON:
|
|
|
51
51
|
- Exact keys only. No extras.
|
|
52
52
|
- `order` is a positive integer.
|
|
53
53
|
- Arrays may be empty but never null.
|
|
54
|
+
|
|
55
|
+
## PR atomicity (hard project rule)
|
|
56
|
+
|
|
57
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
58
|
+
|
|
59
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
60
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
61
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
62
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -46,3 +46,12 @@ You are the **Planner** in a multi-role AI pipeline. Your job is to create an im
|
|
|
46
46
|
"summary": "Plan: 4 steps, estimated 2 files modified, 1 new file"
|
|
47
47
|
}
|
|
48
48
|
```
|
|
49
|
+
|
|
50
|
+
## PR atomicity (hard project rule)
|
|
51
|
+
|
|
52
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
53
|
+
|
|
54
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
55
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
56
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
57
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -37,3 +37,12 @@ You are the **Refactorer** in a multi-role AI pipeline. Your job is to improve c
|
|
|
37
37
|
"summary": "Refactored 2 files: extracted helper, improved naming"
|
|
38
38
|
}
|
|
39
39
|
```
|
|
40
|
+
|
|
41
|
+
## PR atomicity (hard project rule)
|
|
42
|
+
|
|
43
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
44
|
+
|
|
45
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
46
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
47
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
48
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -92,3 +92,12 @@ Return ONLY a single valid JSON object:
|
|
|
92
92
|
defines the contract; the code is what's measured against it.
|
|
93
93
|
- Keep changes minimal. One bug at a time. If the test has multiple
|
|
94
94
|
problems, fix the most blocking one first.
|
|
95
|
+
|
|
96
|
+
## PR atomicity (hard project rule)
|
|
97
|
+
|
|
98
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
99
|
+
|
|
100
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
101
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
102
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
103
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|
|
@@ -39,3 +39,12 @@ You are the **Tester** in a multi-role AI pipeline. You are a quality gate for t
|
|
|
39
39
|
"summary": "Tests pass with 85% coverage. 1 missing scenario identified (non-blocking)."
|
|
40
40
|
}
|
|
41
41
|
```
|
|
42
|
+
|
|
43
|
+
## PR atomicity (hard project rule)
|
|
44
|
+
|
|
45
|
+
Karajan projects MAY enforce a CI gate that fails any PR whose net delta exceeds **200 lines added** (the karajan-code repo itself enforces this since 2026-05-08). Plan your work to stay atomic:
|
|
46
|
+
|
|
47
|
+
- Aim for **~150 LOC** of changes per unit you produce (safety margin against the 200 hard limit).
|
|
48
|
+
- The gate counts the SUM of every changed file, not per-file. Tests count too. 5 files × 40 LOC = 200 = on the limit.
|
|
49
|
+
- Excluded from the count: lockfiles, snapshots, `dist/`, `node_modules/`, generated `tests/_diet/`, `public/docs/`. Source + tests count.
|
|
50
|
+
- Token-economy: oversized PRs get rejected at CI and the work is redone — partitioning upfront saves the round-trip.
|