codehere 2.1.0-rc.0 → 2.1.0-rc.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -2
- package/bin/codehere.js +7 -0
- package/package.json +1 -1
- package/src/audit.js +163 -0
- package/src/brain.js +64 -0
- package/src/commands.js +6 -1
- package/src/hooks.js +57 -5
- package/src/signals.js +40 -19
package/README.md
CHANGED
|
@@ -119,8 +119,11 @@ codehere show <id> read a help card
|
|
|
119
119
|
codehere reply <id> "answer" [--solved] answer; --solved makes a lesson
|
|
120
120
|
codehere status what Codehere has seen
|
|
121
121
|
codehere stats turns to green, loops stopped, context size
|
|
122
|
+
codehere audit commits of the last 90 days that deleted, skipped or weakened tests
|
|
122
123
|
```
|
|
123
124
|
|
|
125
|
+
`codehere audit` reads this repo's git history on your machine and sends nothing anywhere. It lists every commit that deleted a test file, skipped a test, or removed or weakened assertions in test files, and says which ones carry an agent's marker (`Co-Authored-By` naming Claude, Copilot or Codex, `Claude-Session:`, `Agent-Logs-Url:`, or the `copilot-swe-agent[bot]` author). The count is found by pattern: read each commit before drawing a conclusion. A commit with no marker may still be an agent's, since markers can be turned off.
|
|
126
|
+
|
|
124
127
|
When an agent runs `rule`, `lesson`, `help` or `reply`, the command prints a draft and saves nothing. Codehere saves it if your next message is a plain yes. Typed in your own terminal, they save at once.
|
|
125
128
|
|
|
126
129
|
## What lives where
|
|
@@ -164,14 +167,16 @@ npm test # end-to-end checks
|
|
|
164
167
|
- The "done" check needs a test command. Without one, only the tampering check runs.
|
|
165
168
|
- Approvals are per folder. Each git worktree or extra clone needs its own `codehere approve`.
|
|
166
169
|
- In `codex exec`, Codex shows you none of Codehere's lines. The agent still gets every message, and the finish check still sends it back.
|
|
167
|
-
- Copilot shows you none of Codehere's lines either, so on Copilot you do not see the ✓, the ⊘, or the notes about an approval, an uninstall
|
|
170
|
+
- Copilot shows you none of Codehere's lines either, so on Copilot you do not see the ✓, the ⊘, or the notes about an approval, an uninstall, a settings change or team files changed during a session. The agent gets every message except what Codehere adds to your prompt (Copilot drops that, so its agent is not told when a draft is saved or discarded), and when the finish check sends it back you see the reason as the next prompt.
|
|
168
171
|
- Copilot's cloud agent also runs the committed `.github/hooks/codehere.json`. `codehere` is not installed there, so by GitHub's docs each hook fails and the job carries on without Codehere.
|
|
169
172
|
- `codehere approve` and `codehere uninstall` ask only in your own terminal. They refuse when Claude Code, Codex or Copilot is among the processes above them, and inside Codex's sandbox. An agent that detaches its terminal on purpose can still answer, so the finish check of a session that was open tells you when an approval or an uninstall happened (not on Copilot, which shows none of these lines).
|
|
170
173
|
- Approving `npm test` approves that command, not the scripts behind it in `package.json`. And Claude Code runs any hook a teammate adds to the committed `.claude/settings.json`, as Copilot does in a trusted folder for `.github/hooks/` and `.claude/settings.json`; Codex asks each developer to trust a new or changed hook first. Both are your repo's code, with the same trust as running it yourself.
|
|
171
174
|
- When the agent changes `.codehere/config.json`, `.codehere/rules.md` (except a rule saved after your yes), the hook and env settings in `.claude/settings*.json` or `.github/copilot/settings*.json`, or `.codex/hooks.json`, `.codex/config.toml` or `.github/hooks/codehere.json` during a session, the finish check tells you. It cannot tell you if the session ends without a finish check, for example when you close it mid-turn. An agent that sets out to fake your terminal or rewrite Codehere's local files can also get around this.
|
|
175
|
+
- When the agent writes, changes or deletes a lesson or help card in `.codehere/` itself during a session, without a draft you said yes to, the finish check lists the files for you (not on Copilot or in `codex exec`, which show none of Codehere's lines). It does not stop the agent, and it does not list files that arrive committed, such as after a `git pull`, or a change the agent commits itself. A card a pull changes while it holds your own uncommitted changes is listed. A command that also deletes Codehere's local files, such as `git clean -x`, hides the change, and so can an agent that sets out to hide a change from git.
|
|
172
176
|
- Only the test command needs approval. A teammate who sets `verifyOnStop` to `false` in `.codehere/config.json` turns the finish check off for everyone; `codehere status` shows it as off.
|
|
173
177
|
- Sharing is through git. There is no server, so teammates see updates after `git pull`.
|
|
174
|
-
-
|
|
178
|
+
- `codehere audit` follows git's `--since`: git stops reading history at the first commit older than 90 days, so a newer commit behind a back-dated one is not read. A shallow clone has no history to read, and the audit says so.
|
|
179
|
+
- The yes before a draft is saved is checked by Codehere's hooks with a word list. An agent deliberately working around Codehere could fake that reply, as it could fake other hook input.
|
|
175
180
|
- A command you send through the agent drafts: Claude Code's `!` and scripts with no terminal take the agent's path, and so does every command on a machine without `ps`. In VS Code an agent's command can run in a real terminal with no agent above it, and then saves at once. No test has shown whether a `!` command reaches Codehere's hooks; if it does not, your yes gets a "nothing saved" line.
|
|
176
181
|
- A one-shot run (`claude -p`, `codex exec`) that is never resumed has no next message, so its drafts are never saved; a resumed turn (`--resume`, `codex exec resume`) can answer them. Unsaved drafts are removed after 7 days.
|
|
177
182
|
- Codex saves no drafts until you trust its hooks in `/hooks`.
|
package/bin/codehere.js
CHANGED
|
@@ -47,6 +47,7 @@ When you're stuck
|
|
|
47
47
|
|
|
48
48
|
codehere status what Codehere has seen
|
|
49
49
|
codehere stats turns to green, loops stopped, session size
|
|
50
|
+
codehere audit commits of the last 90 days that deleted, skipped or weakened tests
|
|
50
51
|
|
|
51
52
|
Run by an agent, rule, lesson, help and reply print a draft; it is saved if your next
|
|
52
53
|
message in the chat is a plain yes. Typed in your own terminal, they save at once.
|
|
@@ -86,6 +87,12 @@ const run = async () => {
|
|
|
86
87
|
return cmd.status();
|
|
87
88
|
case 'stats':
|
|
88
89
|
return cmd.stats();
|
|
90
|
+
case 'audit': {
|
|
91
|
+
// loaded only here: the hooks run on every agent turn and must stay fast
|
|
92
|
+
const { audit } = await import('../src/audit.js');
|
|
93
|
+
process.exitCode = await audit(process.cwd());
|
|
94
|
+
return;
|
|
95
|
+
}
|
|
89
96
|
default:
|
|
90
97
|
if (flags.version || command === '-v') {
|
|
91
98
|
return console.log(`codehere ${JSON.parse(fs.readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version}`);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "codehere",
|
|
3
|
-
"version": "2.1.0-rc.
|
|
3
|
+
"version": "2.1.0-rc.2",
|
|
4
4
|
"description": "A git-native team brain for vibe coders. Silent fixes for AI coding agents: breaks doom loops, verifies 'done', blocks test tampering, and shares what your team learned.",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
package/src/audit.js
ADDED
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
// codehere audit: the commits of the last 90 days that weakened tests, each listed for a person to read.
|
|
2
|
+
// Read-only: it runs git on this machine and sends nothing anywhere. The kinds come from the finish
|
|
3
|
+
// check's own detector. A count found by pattern is a list of candidates, never proof (DESIGN.md §1).
|
|
4
|
+
import { spawn, execFileSync } from 'node:child_process';
|
|
5
|
+
import readline from 'node:readline';
|
|
6
|
+
import { printable } from './brain.js';
|
|
7
|
+
import { tamperEvents, isTestFile } from './signals.js';
|
|
8
|
+
|
|
9
|
+
const DAYS = 90;
|
|
10
|
+
const KINDS = new Set(['test-deleted', 'test-skipped', 'assertion-weakened']);
|
|
11
|
+
const BIG = 512 * 1024 * 1024;
|
|
12
|
+
|
|
13
|
+
// How agents mark their commits. A commit without a marker is still counted and listed, unmarked:
|
|
14
|
+
// markers can be turned off, and some agents leave none.
|
|
15
|
+
const MARKERS = [
|
|
16
|
+
{ re: /^Co-authored-by:[^\n]*\b(Claude|Copilot|Codex)\b/im, name: (m) => `Co-Authored-By ${m[1]}` },
|
|
17
|
+
{ re: /^Claude-Session:/im, name: () => 'Claude-Session' },
|
|
18
|
+
{ re: /^Agent-Logs-Url:/im, name: () => 'Agent-Logs-Url' },
|
|
19
|
+
];
|
|
20
|
+
|
|
21
|
+
export function markersOf(author, body) {
|
|
22
|
+
const found = /copilot-swe-agent\[bot\]/i.test(author) ? ['copilot-swe-agent[bot]'] : [];
|
|
23
|
+
for (const m of MARKERS) {
|
|
24
|
+
const x = m.re.exec(body);
|
|
25
|
+
if (x) found.push(m.name(x));
|
|
26
|
+
}
|
|
27
|
+
return found;
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
function gitLine(root, args) {
|
|
31
|
+
try {
|
|
32
|
+
return execFileSync('git', args, { cwd: root, stdio: ['ignore', 'pipe', 'ignore'] }).toString().trim();
|
|
33
|
+
} catch {
|
|
34
|
+
return null;
|
|
35
|
+
}
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
// Every non-merge commit since `since`, with what the markers need.
|
|
39
|
+
function commitsSince(root, since) {
|
|
40
|
+
const out = execFileSync('git', ['log', '--no-merges', `--since=${since}`, '--format=%H%x1f%an <%ae>%x1f%cs%x1f%s%x1f%B%x1e'], {
|
|
41
|
+
cwd: root,
|
|
42
|
+
stdio: ['ignore', 'pipe', 'ignore'],
|
|
43
|
+
maxBuffer: BIG,
|
|
44
|
+
}).toString();
|
|
45
|
+
return out
|
|
46
|
+
.split('\x1e')
|
|
47
|
+
.map((r) => r.replace(/^\n/, ''))
|
|
48
|
+
.filter(Boolean)
|
|
49
|
+
.map((r) => {
|
|
50
|
+
const [sha, author, date, subject, body = ''] = r.split('\x1f');
|
|
51
|
+
return { sha, author, date, subject, body, events: [] };
|
|
52
|
+
});
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
// The path in `diff --git a/P b/P` when both sides name the same file, as they do for a deletion.
|
|
56
|
+
function samePath(line) {
|
|
57
|
+
const rest = line.slice('diff --git a/'.length);
|
|
58
|
+
const half = (rest.length - ' b/'.length) / 2;
|
|
59
|
+
return Number.isInteger(half) && rest.slice(half, half + 3) === ' b/' ? rest.slice(0, half) : '';
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
// One `git log -p`, read line by line: test-file hunks are kept, a deleted file by its name only, so
|
|
63
|
+
// memory stays small however large a commit is. Each commit's kinds go on its entry in bySha.
|
|
64
|
+
function scanChanges(root, since, bySha) {
|
|
65
|
+
return new Promise((resolve, reject) => {
|
|
66
|
+
const child = spawn('git', ['log', '--no-merges', `--since=${since}`, '-p', '--no-color', '--format=%x00%H'], {
|
|
67
|
+
cwd: root,
|
|
68
|
+
stdio: ['ignore', 'pipe', 'ignore'],
|
|
69
|
+
});
|
|
70
|
+
const lines = readline.createInterface({ input: child.stdout, crlfDelay: Infinity });
|
|
71
|
+
let commit = null;
|
|
72
|
+
let nameStatus = [];
|
|
73
|
+
let kept = [];
|
|
74
|
+
let keep = false;
|
|
75
|
+
let deleted = '';
|
|
76
|
+
const finish = () => {
|
|
77
|
+
if (commit) commit.events = tamperEvents(nameStatus.join('\n'), kept.join('\n')).filter((e) => KINDS.has(e.kind));
|
|
78
|
+
};
|
|
79
|
+
lines.on('line', (line) => {
|
|
80
|
+
if (line.startsWith('\0')) {
|
|
81
|
+
finish();
|
|
82
|
+
commit = bySha.get(line.slice(1)) || null;
|
|
83
|
+
nameStatus = [];
|
|
84
|
+
kept = [];
|
|
85
|
+
keep = false;
|
|
86
|
+
return;
|
|
87
|
+
}
|
|
88
|
+
if (!commit) return;
|
|
89
|
+
if (line.startsWith('diff --git ')) {
|
|
90
|
+
keep = false;
|
|
91
|
+
deleted = samePath(line);
|
|
92
|
+
return;
|
|
93
|
+
}
|
|
94
|
+
if (line.startsWith('deleted file mode')) {
|
|
95
|
+
if (deleted && isTestFile(deleted)) nameStatus.push(`D\t${deleted}`);
|
|
96
|
+
deleted = '';
|
|
97
|
+
keep = false;
|
|
98
|
+
return;
|
|
99
|
+
}
|
|
100
|
+
if (line.startsWith('+++ ')) {
|
|
101
|
+
const file = line.replace(/^\+\+\+ (b\/)?/, '');
|
|
102
|
+
keep = file !== '/dev/null' && isTestFile(file);
|
|
103
|
+
if (keep) kept.push(`+++ ${file}`);
|
|
104
|
+
return;
|
|
105
|
+
}
|
|
106
|
+
if (keep && !line.startsWith('--- ')) kept.push(line);
|
|
107
|
+
});
|
|
108
|
+
let failed = null;
|
|
109
|
+
child.on('error', (e) => (failed = e));
|
|
110
|
+
const ended = new Promise((r) => lines.on('close', r));
|
|
111
|
+
child.on('close', async (code) => {
|
|
112
|
+
await ended;
|
|
113
|
+
finish();
|
|
114
|
+
if (failed || code !== 0) reject(failed || new Error(`git log exited ${code}`));
|
|
115
|
+
else resolve();
|
|
116
|
+
});
|
|
117
|
+
});
|
|
118
|
+
}
|
|
119
|
+
|
|
120
|
+
function summary(events) {
|
|
121
|
+
const deletedFiles = events.filter((e) => e.kind === 'test-deleted');
|
|
122
|
+
const other = events.filter((e) => e.kind !== 'test-deleted').map((e) => e.why);
|
|
123
|
+
const parts = deletedFiles.length > 1 ? [`${deletedFiles.length} test files deleted`] : deletedFiles.map((e) => e.why);
|
|
124
|
+
parts.push(...other.slice(0, 3));
|
|
125
|
+
if (other.length > 3) parts.push(`and ${other.length - 3} more`);
|
|
126
|
+
return parts.join(' · ');
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
// Prints the audit; returns the exit code.
|
|
130
|
+
export async function audit(root, { out = console.log, err = console.error, now = Date.now() } = {}) {
|
|
131
|
+
if (gitLine(root, ['rev-parse', '--is-inside-work-tree']) !== 'true') {
|
|
132
|
+
err('codehere audit: not inside a git repository');
|
|
133
|
+
return 1;
|
|
134
|
+
}
|
|
135
|
+
if (gitLine(root, ['rev-parse', '--is-shallow-repository']) === 'true') {
|
|
136
|
+
err('codehere audit: this clone is shallow, so its history is not available › git fetch --unshallow');
|
|
137
|
+
return 1;
|
|
138
|
+
}
|
|
139
|
+
const since = new Date(now - DAYS * 24 * 3600 * 1000).toISOString();
|
|
140
|
+
let list;
|
|
141
|
+
try {
|
|
142
|
+
list = commitsSince(root, since);
|
|
143
|
+
if (list.length > 500) err(`codehere audit: reading ${list.length} commits`);
|
|
144
|
+
await scanChanges(root, since, new Map(list.map((c) => [c.sha, c])));
|
|
145
|
+
} catch {
|
|
146
|
+
err('codehere audit: git log failed, so nothing was counted');
|
|
147
|
+
return 1;
|
|
148
|
+
}
|
|
149
|
+
const flagged = list.filter((c) => c.events.length);
|
|
150
|
+
const marked = flagged.filter((c) => markersOf(c.author, c.body).length);
|
|
151
|
+
const n = (k, one, many) => `${k} ${k === 1 ? one : many}`;
|
|
152
|
+
out(`codehere audit · last ${DAYS} days · ${n(list.length, 'commit', 'commits')} read`);
|
|
153
|
+
out(`codehere › ${n(flagged.length, 'commit', 'commits')} weakened tests · ${n(marked.length, 'carries', 'carry')} an agent marker`);
|
|
154
|
+
if (!flagged.length) return 0;
|
|
155
|
+
out('codehere › each was found by pattern: read the commit before drawing a conclusion');
|
|
156
|
+
for (const c of flagged) {
|
|
157
|
+
const m = markersOf(c.author, c.body);
|
|
158
|
+
out('');
|
|
159
|
+
out(` ${c.sha.slice(0, 8)} · ${c.date} · ${m.length ? m.join(', ') : 'no agent marker'} · ${printable(c.subject)}`);
|
|
160
|
+
out(` ${printable(summary(c.events))}`);
|
|
161
|
+
}
|
|
162
|
+
return 0;
|
|
163
|
+
}
|
package/src/brain.js
CHANGED
|
@@ -342,6 +342,16 @@ export function git(root, args, raw = false) {
|
|
|
342
342
|
}
|
|
343
343
|
}
|
|
344
344
|
|
|
345
|
+
// A diff can be far larger than execFileSync's 1 MiB default, and an unread diff must never look
|
|
346
|
+
// like an empty one: this reads it whole and returns null when git fails.
|
|
347
|
+
export function gitDiff(root, args) {
|
|
348
|
+
try {
|
|
349
|
+
return execFileSync('git', ['--literal-pathspecs', ...args], { cwd: root, stdio: ['ignore', 'pipe', 'ignore'], maxBuffer: 512 * 1024 * 1024 }).toString();
|
|
350
|
+
} catch {
|
|
351
|
+
return null;
|
|
352
|
+
}
|
|
353
|
+
}
|
|
354
|
+
|
|
345
355
|
// Files with uncommitted changes, relative to root, which may be a subfolder of the repo.
|
|
346
356
|
export function statusFiles(root) {
|
|
347
357
|
const prefix = git(root, ['rev-parse', '--show-prefix']);
|
|
@@ -472,6 +482,60 @@ export function saveRecord(root, rec, proof = null) {
|
|
|
472
482
|
}
|
|
473
483
|
}
|
|
474
484
|
|
|
485
|
+
export const isMap = (x) => !!x && typeof x === 'object' && !Array.isArray(x);
|
|
486
|
+
|
|
487
|
+
// Lessons and help cards as hashes by path, for the finish check: one the agent writes itself is
|
|
488
|
+
// listed for the person. A link is not followed, and a linked folder is not read: it stands for
|
|
489
|
+
// itself, so swapping a folder for a link is a change too.
|
|
490
|
+
export function teamSnapshot(root) {
|
|
491
|
+
const snap = {};
|
|
492
|
+
for (const d of ['.codehere/lessons', '.codehere/help']) {
|
|
493
|
+
const dir = path.join(root, d);
|
|
494
|
+
if (!ownPath(root, dir)) {
|
|
495
|
+
let target = '';
|
|
496
|
+
try {
|
|
497
|
+
target = fs.readlinkSync(dir);
|
|
498
|
+
} catch {}
|
|
499
|
+
snap[d] = `link:${target}`;
|
|
500
|
+
continue;
|
|
501
|
+
}
|
|
502
|
+
let names = [];
|
|
503
|
+
try {
|
|
504
|
+
names = fs.readdirSync(dir);
|
|
505
|
+
} catch {}
|
|
506
|
+
for (const n of names) {
|
|
507
|
+
const f = path.join(dir, n);
|
|
508
|
+
try {
|
|
509
|
+
const st = fs.lstatSync(f);
|
|
510
|
+
const text = st.isSymbolicLink() ? `link:${fs.readlinkSync(f)}` : st.isFile() ? readPlain(f, null) : null;
|
|
511
|
+
if (text != null) snap[`${d}/${n}`] = crypto.createHash('sha256').update(text).digest('hex');
|
|
512
|
+
} catch {}
|
|
513
|
+
}
|
|
514
|
+
}
|
|
515
|
+
return snap;
|
|
516
|
+
}
|
|
517
|
+
|
|
518
|
+
// Before a save: the team files' hashes, and which files git sees as changed.
|
|
519
|
+
export function teamMark(root) {
|
|
520
|
+
return { hashes: teamSnapshot(root), dirty: new Set(statusFiles(root).map((f) => f.normalize('NFC'))) };
|
|
521
|
+
}
|
|
522
|
+
|
|
523
|
+
// A save Codehere made is not the agent's change: what it changed moves into the session's
|
|
524
|
+
// snapshot, unless that file had an uncommitted change made earlier in the session, which stays
|
|
525
|
+
// visible. A change that came committed (a pull) is not the agent's. True if anything moved.
|
|
526
|
+
export function keepSaved(s, mark, after) {
|
|
527
|
+
let moved = false;
|
|
528
|
+
if (!s || !isMap(s.team)) return moved;
|
|
529
|
+
const before = mark.hashes;
|
|
530
|
+
for (const f of new Set([...Object.keys(before), ...Object.keys(after)])) {
|
|
531
|
+
if (before[f] === after[f] || (s.team[f] !== before[f] && mark.dirty.has(f.normalize('NFC')))) continue;
|
|
532
|
+
if (after[f] === undefined) delete s.team[f];
|
|
533
|
+
else s.team[f] = after[f];
|
|
534
|
+
moved = true;
|
|
535
|
+
}
|
|
536
|
+
return moved;
|
|
537
|
+
}
|
|
538
|
+
|
|
475
539
|
function save(root, rec, proof) {
|
|
476
540
|
const p = paths(root);
|
|
477
541
|
const rel = (f) => path.relative(root, f);
|
package/src/commands.js
CHANGED
|
@@ -7,7 +7,7 @@ import {
|
|
|
7
7
|
findRoot, paths, ensureDirs, readConfig, writeConfig, readState, writeState,
|
|
8
8
|
readRules, listDocs, parseDoc, renderDoc, slug, stamp, today, git, isGitRepo, me,
|
|
9
9
|
readApprovals, isApproved, addApproval, removeApprovals, markRemoved, printable, noTick, commitId,
|
|
10
|
-
saveRecord, writeDraft, readPlain, writePlain, removePlain, ownPath,
|
|
10
|
+
saveRecord, writeDraft, readPlain, writePlain, removePlain, ownPath, teamSnapshot, teamMark, keepSaved,
|
|
11
11
|
} from './brain.js';
|
|
12
12
|
import { redact } from './signals.js';
|
|
13
13
|
import { summarize } from './gemma.js';
|
|
@@ -232,11 +232,16 @@ const indent = (text) => text.trimEnd().split('\n').map((l) => ` ${l}`);
|
|
|
232
232
|
// One line, built before the record, so the draft prints exactly what a yes saves.
|
|
233
233
|
const oneLine = (s) => String(s).replace(/\s*[\r\n]+\s*/g, ' ');
|
|
234
234
|
function saveOrExit(root, rec) {
|
|
235
|
+
const before = teamMark(root);
|
|
235
236
|
const saved = saveRecord(root, rec);
|
|
236
237
|
if (saved.error) {
|
|
237
238
|
console.error(saved.error);
|
|
238
239
|
process.exit(1);
|
|
239
240
|
}
|
|
241
|
+
// typed by the person, so not the agent's change in a session that is open now
|
|
242
|
+
const after = teamSnapshot(root);
|
|
243
|
+
const state = readState(root);
|
|
244
|
+
if (Object.values(state.sessions || {}).filter((x) => keepSaved(x, before, after)).length) writeState(root, state);
|
|
240
245
|
return saved;
|
|
241
246
|
}
|
|
242
247
|
|
package/src/hooks.js
CHANGED
|
@@ -7,8 +7,8 @@ import crypto from 'node:crypto';
|
|
|
7
7
|
import { spawnSync } from 'node:child_process';
|
|
8
8
|
import {
|
|
9
9
|
findRoot, paths, readConfig, readState, writeState, session, readRules, renderDoc,
|
|
10
|
-
listDocs, git, isGitRepo, me, isApproved, readApprovals, printable, commitId, statusFiles, gitHashes, noTick,
|
|
11
|
-
removedAt, expireDrafts, draftFile, saveRecord, readPlain, ownPath,
|
|
10
|
+
listDocs, git, gitDiff, isGitRepo, me, isApproved, readApprovals, printable, commitId, statusFiles, gitHashes, noTick,
|
|
11
|
+
removedAt, expireDrafts, draftFile, saveRecord, readPlain, ownPath, teamSnapshot, teamMark, keepSaved, isMap,
|
|
12
12
|
} from './brain.js';
|
|
13
13
|
import {
|
|
14
14
|
responseText, signature, errorLines, similarity, tamperFindings, failingTest, lowInfoPrompt, vagueFeature, testCounts, redact, plainYes,
|
|
@@ -119,9 +119,13 @@ function sessionStart(root, input, agent) {
|
|
|
119
119
|
s.startHead = git(root, 'rev-parse HEAD') || null;
|
|
120
120
|
s.baseline = dirtySnapshot(root);
|
|
121
121
|
s.settings = settingsSnapshot(root);
|
|
122
|
+
s.team = teamSnapshot(root);
|
|
123
|
+
s.teamHead = s.startHead;
|
|
122
124
|
} else if (input.source === 'resume') {
|
|
123
125
|
// While the session was closed only the developer or a pull could change settings.
|
|
124
126
|
s.settings = settingsSnapshot(root);
|
|
127
|
+
s.team = teamSnapshot(root);
|
|
128
|
+
s.teamHead = git(root, 'rev-parse HEAD') || null;
|
|
125
129
|
}
|
|
126
130
|
|
|
127
131
|
const out = [];
|
|
@@ -262,6 +266,35 @@ function settingsSnapshot(root) {
|
|
|
262
266
|
return snap;
|
|
263
267
|
}
|
|
264
268
|
|
|
269
|
+
// Lessons and help cards changed since the snapshot. A file with an uncommitted change is listed.
|
|
270
|
+
// Otherwise it is listed only if no commit since the snapshot touched it: a pull, or the person's
|
|
271
|
+
// own commit, changes files through commits, while an uncommitted save undone by `git stash` or
|
|
272
|
+
// `git checkout`, or an untracked file deleted, leaves no commit. What Codehere saved was moved
|
|
273
|
+
// into the snapshot (keepSaved).
|
|
274
|
+
function teamChanged(root, s) {
|
|
275
|
+
if (!isMap(s.team)) return [];
|
|
276
|
+
const now = teamSnapshot(root);
|
|
277
|
+
const diff = [...new Set([...Object.keys(now), ...Object.keys(s.team)])].filter((f) => now[f] !== s.team[f]);
|
|
278
|
+
if (!diff.length) return [];
|
|
279
|
+
const nfc = (f) => f.normalize('NFC');
|
|
280
|
+
const dirty = new Set(statusFiles(root).map(nfc));
|
|
281
|
+
let byCommit = null;
|
|
282
|
+
const committed = (f) => {
|
|
283
|
+
if (!byCommit) {
|
|
284
|
+
// the commits since the snapshot, and the trees' difference (a reset or a branch switch); a
|
|
285
|
+
// rename counts as a removal and an addition
|
|
286
|
+
const from = commitId(s.teamHead === undefined ? s.startHead : s.teamHead);
|
|
287
|
+
const paths = ['--', '.codehere/lessons', '.codehere/help'];
|
|
288
|
+
const names = (out) => out.split('\0').map((x) => nfc(x.replace(/^\n+/, ''))).filter(Boolean);
|
|
289
|
+
const log = names(git(root, ['log', '--name-only', '--no-renames', '--relative', '-z', '--format=', from ? `${from}..HEAD` : 'HEAD', ...paths], true));
|
|
290
|
+
const trees = from ? names(git(root, ['diff', '--name-only', '--no-renames', '--relative', '-z', from, 'HEAD', ...paths], true)) : [];
|
|
291
|
+
byCommit = new Set([...log, ...trees]);
|
|
292
|
+
}
|
|
293
|
+
return byCommit.has(nfc(f));
|
|
294
|
+
};
|
|
295
|
+
return diff.filter((f) => dirty.has(nfc(f)) || !committed(f)).sort();
|
|
296
|
+
}
|
|
297
|
+
|
|
265
298
|
function settingsChanged(root, s) {
|
|
266
299
|
if (!s.settings) return []; // no snapshot from session start: cannot tell
|
|
267
300
|
const now = settingsSnapshot(root);
|
|
@@ -677,6 +710,7 @@ function decideDrafts(root, state, s, prompt) {
|
|
|
677
710
|
rec = JSON.parse(content);
|
|
678
711
|
} catch {}
|
|
679
712
|
const before = settingsSnapshot(root)[RULES];
|
|
713
|
+
const teamBefore = teamMark(root);
|
|
680
714
|
const saved = saveRecord(root, rec, d.proof || null);
|
|
681
715
|
if (saved.error) {
|
|
682
716
|
notes.push(`Codehere could not save the draft: ${saved.error}`);
|
|
@@ -685,6 +719,8 @@ function decideDrafts(root, state, s, prompt) {
|
|
|
685
719
|
}
|
|
686
720
|
// a saved rule is not a settings change, unless rules.md had already changed this session
|
|
687
721
|
if (rec.kind === 'rule' && s.settings && s.settings[RULES] === before) s.settings[RULES] = settingsSnapshot(root)[RULES];
|
|
722
|
+
const teamAfter = teamSnapshot(root);
|
|
723
|
+
for (const x of Object.values(state.sessions)) keepSaved(x, teamBefore, teamAfter); // every open session: the save is not its agent's either
|
|
688
724
|
notes.push(`Codehere saved the draft after the user's yes: ${saved.file}${saved.lesson ? `, and the team lesson ${saved.lesson}` : ''}. Tell them it is saved. Teammates get it once the user commits and pushes it.`);
|
|
689
725
|
// the agent's own words: never a tick in a codehere line (DESIGN.md §1)
|
|
690
726
|
lines.push(`codehere › saved after your yes · ${rec.kind} · ${short(noTick(gist(rec)))}`);
|
|
@@ -728,6 +764,10 @@ function promptSubmit(root, input) {
|
|
|
728
764
|
s.transcript = input.transcript_path || s.transcript;
|
|
729
765
|
s.prompts = (s.prompts || 0) + 1;
|
|
730
766
|
if (!s.settings) s.settings = settingsSnapshot(root); // session start did not run for this session
|
|
767
|
+
if (!isMap(s.team)) {
|
|
768
|
+
s.team = teamSnapshot(root);
|
|
769
|
+
s.teamHead = git(root, 'rev-parse HEAD') || null;
|
|
770
|
+
}
|
|
731
771
|
const decided = decideDrafts(root, state, s, prompt);
|
|
732
772
|
|
|
733
773
|
// Only a failure from the agent's last turn counts: an older one may be fixed or forgotten.
|
|
@@ -882,7 +922,8 @@ function stop(root, input) {
|
|
|
882
922
|
const changed = changedThisSession(root, s);
|
|
883
923
|
const settings = settingsChanged(root, s);
|
|
884
924
|
const removed = removedAt(root);
|
|
885
|
-
|
|
925
|
+
const team = teamChanged(root, s);
|
|
926
|
+
if (!changed.length && !settings.length && !removed && !team.length) quiet(); // the agent changed nothing this session: stay silent
|
|
886
927
|
|
|
887
928
|
// Lines for the developer. Each is escaped as a whole, so text from the repo cannot fake a
|
|
888
929
|
// line, and text from the repo never carries a tick (noTick).
|
|
@@ -893,6 +934,7 @@ function stop(root, input) {
|
|
|
893
934
|
if (a.at > (s.startedAt || '')) all.push(`codehere › approved on this machine during this session: ${noTick(a.command)}`);
|
|
894
935
|
}
|
|
895
936
|
if (settings.length) all.push(`codehere ✕ settings changed during this session · ${settings.join(' · ')}`);
|
|
937
|
+
if (team.length) all.push(`codehere ✕ team files changed during this session, not saved by Codehere · ${noTick(team.slice(0, 5).join(' · '))}${team.length > 5 ? ` · and ${team.length - 5} more` : ''}`);
|
|
896
938
|
if (removed) all.push(removedLine(removed));
|
|
897
939
|
return all.map(printable).join('\n');
|
|
898
940
|
}
|
|
@@ -901,12 +943,17 @@ function stop(root, input) {
|
|
|
901
943
|
if (m) emitJson({ systemMessage: m });
|
|
902
944
|
quiet();
|
|
903
945
|
}
|
|
946
|
+
// only lessons or help cards changed: the person gets the line, and the tests are not run
|
|
947
|
+
if (!changed.length && !settings.length && !removed) report();
|
|
904
948
|
|
|
905
949
|
// 1. tampering with tests or checks (only in files changed this session)
|
|
906
950
|
let findings = [];
|
|
951
|
+
let unread = false; // the session's diff could not be read, so weakened tests were not checked: never proven
|
|
907
952
|
if (changed.length) {
|
|
908
953
|
const nameStatus = git(root, ['diff', base, '--relative', '--name-status', '--', ...changed]);
|
|
909
|
-
|
|
954
|
+
const diff = gitDiff(root, ['diff', base, '--relative', '-U0', '--', ...changed]);
|
|
955
|
+
unread = diff === null;
|
|
956
|
+
findings = tamperFindings(nameStatus, diff || '');
|
|
910
957
|
}
|
|
911
958
|
const key = findings.join('|');
|
|
912
959
|
if (findings.length && s.tamperReported !== key && mayBlock) {
|
|
@@ -955,12 +1002,16 @@ function stop(root, input) {
|
|
|
955
1002
|
`codehere ⊘ ${noTick(cfg.testCommand)} · not run · not approved on this machine${last ? ` · last approved: ${noTick(last.command)}` : ''} › run codehere approve in your terminal`
|
|
956
1003
|
);
|
|
957
1004
|
}
|
|
1005
|
+
if (unread && !(cfg.verifyOnStop && approved)) lines.push("codehere ✕ this session's diff could not be read · weakened tests not checked");
|
|
958
1006
|
const configChanged = settings.includes('.codehere/config.json');
|
|
959
1007
|
const cmd = noTick(cfg.testCommand || '');
|
|
960
1008
|
let passLine = null;
|
|
961
1009
|
let proven = false; // passed here, now, with intact tests and an unchanged config
|
|
962
1010
|
if (cfg.verifyOnStop && approved) {
|
|
963
1011
|
const t = runTests(root, cfg);
|
|
1012
|
+
// saves made while the tests ran moved snapshots on disk, and this hook writes the whole state back
|
|
1013
|
+
const disk = readState(root).sessions || {};
|
|
1014
|
+
for (const [k, x] of Object.entries(state.sessions)) if (isMap(disk[k]?.team)) Object.assign(x, { team: disk[k].team, teamHead: disk[k].teamHead });
|
|
964
1015
|
if (!t.ok && !mayBlock) {
|
|
965
1016
|
lines.push(`codehere ✕ ${cmd} · ${runResult(t)} · not sent back, limit of 3 reached`);
|
|
966
1017
|
writeState(root, state);
|
|
@@ -983,9 +1034,10 @@ function stop(root, input) {
|
|
|
983
1034
|
}
|
|
984
1035
|
// ✓ only for intact evidence (DESIGN.md §1): not after a weakened test, nor after the
|
|
985
1036
|
// config that names the test command changed.
|
|
986
|
-
proven = !findings.length && !configChanged;
|
|
1037
|
+
proven = !findings.length && !configChanged && !unread;
|
|
987
1038
|
if (findings.length) passLine = `codehere ✕ ${cmd} passed, but a check was weakened · ${noTick(findings.join(' · '))}`;
|
|
988
1039
|
else if (configChanged) passLine = `codehere ✕ ${cmd} passed, but .codehere/config.json changed during this session`;
|
|
1040
|
+
else if (unread) passLine = `codehere ✕ ${cmd} passed, but this session's diff could not be read · weakened tests not checked`;
|
|
989
1041
|
else passLine = `codehere ✓ ${cmd} · ${runResult(t)} · re-run here · ${t.seconds.toFixed(1)} s`;
|
|
990
1042
|
if (proven) {
|
|
991
1043
|
state.lastGreen = { commit: git(root, 'rev-parse HEAD'), at: new Date().toISOString() };
|
package/src/signals.js
CHANGED
|
@@ -132,27 +132,35 @@ export function similarity(a, b) {
|
|
|
132
132
|
// ---------- test tampering ----------
|
|
133
133
|
|
|
134
134
|
const TEST_FILE = /(^|\/)(test|tests|__tests__|spec)\/|\.(test|spec)\.[a-z]+$|_test\.(go|py)$|(^|\/)test_[^/]+\.py$/i;
|
|
135
|
+
export const isTestFile = (file) => TEST_FILE.test(file);
|
|
136
|
+
|
|
137
|
+
// Kinds: the first three weaken the tests themselves, which is what `codehere audit` counts;
|
|
138
|
+
// checks-off turns off a check around them (hooks, types, the test script's exit code).
|
|
139
|
+
const DELETED = 'test-deleted';
|
|
140
|
+
const SKIPPED = 'test-skipped';
|
|
141
|
+
const WEAKENED = 'assertion-weakened';
|
|
142
|
+
const CHECKS_OFF = 'checks-off';
|
|
135
143
|
|
|
136
144
|
const TAMPER_ADDED = [
|
|
137
|
-
{ re: /\b(it|test|describe)\.skip\s*\(/, why: 'a test was skipped (.skip)' },
|
|
138
|
-
{ re: /\b(xit|xdescribe|xtest)\s*\(/, why: 'a test was disabled (xit/xdescribe)' },
|
|
139
|
-
{ re: /@pytest\.mark\.skip|unittest\.skip/, why: 'a Python test was skipped' },
|
|
140
|
-
{ re: /@ts-nocheck/, why: 'type checking was turned off (@ts-nocheck)' },
|
|
141
|
-
{ re: /--no-verify/, why: 'git hooks were bypassed (--no-verify)' },
|
|
142
|
-
{ re: /expect\([^)]*\)\.(toBe|toEqual)\(\s*(true|1)\s*\)\s*;?\s*\/\/\s*(todo|temp|hack)/i, why: 'an assertion was replaced with a placeholder' },
|
|
145
|
+
{ re: /\b(it|test|describe)\.skip\s*\(/, kind: SKIPPED, why: 'a test was skipped (.skip)' },
|
|
146
|
+
{ re: /\b(xit|xdescribe|xtest)\s*\(/, kind: SKIPPED, why: 'a test was disabled (xit/xdescribe)' },
|
|
147
|
+
{ re: /@pytest\.mark\.skip|unittest\.skip/, kind: SKIPPED, why: 'a Python test was skipped' },
|
|
148
|
+
{ re: /@ts-nocheck/, kind: CHECKS_OFF, why: 'type checking was turned off (@ts-nocheck)' },
|
|
149
|
+
{ re: /--no-verify/, kind: CHECKS_OFF, why: 'git hooks were bypassed (--no-verify)' },
|
|
150
|
+
{ re: /expect\([^)]*\)\.(toBe|toEqual)\(\s*(true|1)\s*\)\s*;?\s*\/\/\s*(todo|temp|hack)/i, kind: WEAKENED, why: 'an assertion was replaced with a placeholder' },
|
|
143
151
|
];
|
|
144
152
|
|
|
145
153
|
// Only in test files: elsewhere `{ skip: true }` or `assert.equal(1, 1)` can be ordinary code.
|
|
146
154
|
const LITERAL = String.raw`(\d+(?:\.\d+)?|true|false|null|'[^']*'|"[^"]*")`;
|
|
147
155
|
const TAMPER_ADDED_IN_TESTS = [
|
|
148
156
|
// test('name', { skip: true }, ...): the name cannot run past its closing quote, so the options are this test's
|
|
149
|
-
{ re: /(?<![.\w])(it|test|describe|suite)\s*\(\s*(['"`])(?:(?!\2)[^\\]|\\.)*\2\s*,\s*\{[^}]*\b(skip|todo)\s*:\s*(true\b|['"`])/, why: 'a test was skipped ({ skip: true })' },
|
|
157
|
+
{ re: /(?<![.\w])(it|test|describe|suite)\s*\(\s*(['"`])(?:(?!\2)[^\\]|\\.)*\2\s*,\s*\{[^}]*\b(skip|todo)\s*:\s*(true\b|['"`])/, kind: SKIPPED, why: 'a test was skipped ({ skip: true })' },
|
|
150
158
|
// fit/fdescribe only in jest's form, starting a statement with a quoted name then a callback:
|
|
151
159
|
// `fit(` alone is a common function name
|
|
152
|
-
{ re: /(?<![.\w])(it|test|describe|context|suite)\.only\s*\(|^[+-]\s*(fit|fdescribe)\s*\(\s*(['"`])(?:(?!\3)[^\\]|\\.)*\3\s*,\s*(async\s+)?(\(|function\b|\w+\s*=>)/, why: 'other tests were turned off (.only)' },
|
|
153
|
-
{ re: /\bassert(\.ok)?\(\s*true\s*\)|^\+\s*assert\s+True\s*$/, why: 'an assertion that cannot fail was added' },
|
|
154
|
-
{ re: new RegExp(String.raw`\bassert\.\w+\(\s*${LITERAL}\s*,\s*\1\s*\)`), why: 'an assertion that cannot fail was added' },
|
|
155
|
-
{ re: new RegExp(String.raw`\bexpect\(\s*${LITERAL}\s*\)\.(toBe|toEqual|toStrictEqual)\(\s*\1\s*\)`), why: 'an assertion that cannot fail was added' },
|
|
160
|
+
{ re: /(?<![.\w])(it|test|describe|context|suite)\.only\s*\(|^[+-]\s*(fit|fdescribe)\s*\(\s*(['"`])(?:(?!\3)[^\\]|\\.)*\3\s*,\s*(async\s+)?(\(|function\b|\w+\s*=>)/, kind: SKIPPED, why: 'other tests were turned off (.only)' },
|
|
161
|
+
{ re: /\bassert(\.ok)?\(\s*true\s*\)|^\+\s*assert\s+True\s*$/, kind: WEAKENED, why: 'an assertion that cannot fail was added' },
|
|
162
|
+
{ re: new RegExp(String.raw`\bassert\.\w+\(\s*${LITERAL}\s*,\s*\1\s*\)`), kind: WEAKENED, why: 'an assertion that cannot fail was added' },
|
|
163
|
+
{ re: new RegExp(String.raw`\bexpect\(\s*${LITERAL}\s*\)\.(toBe|toEqual|toStrictEqual)\(\s*\1\s*\)`), kind: WEAKENED, why: 'an assertion that cannot fail was added' },
|
|
156
164
|
];
|
|
157
165
|
// assert(...), assert.equal(...), expect(...), self.assertEqual(...), and pytest's bare `assert x`;
|
|
158
166
|
// not expect.objectContaining(...) and the other matchers inside one assertion
|
|
@@ -161,11 +169,24 @@ const ASSERT = /\b(expect\s*\(|assert[\w.]*\s*\()|^[+-]\s*assert\s/;
|
|
|
161
169
|
const ALWAYS_PASSES = /"test"\s*:\s*"[^"]*(\|\|\s*(true|:|exit\s+0|echo\b[^"&|;]*)|;\s*(true|exit\s+0))\s*"/;
|
|
162
170
|
const COMMENT = /^[+-]\s*(\/\/|#|\/\*|\*)\s*/;
|
|
163
171
|
|
|
172
|
+
// What the finish check prints: the events' text, in order, each once.
|
|
164
173
|
export function tamperFindings(nameStatus, diff) {
|
|
165
|
-
|
|
174
|
+
return tamperEvents(nameStatus, diff).map((e) => e.why);
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
// Each finding as { kind, file, why }; file is undefined for the count of removed assertions,
|
|
178
|
+
// which spans the whole diff. A why already seen is not repeated.
|
|
179
|
+
export function tamperEvents(nameStatus, diff) {
|
|
180
|
+
const events = [];
|
|
181
|
+
const seen = new Set();
|
|
182
|
+
const add = (kind, file, why) => {
|
|
183
|
+
if (seen.has(why)) return;
|
|
184
|
+
seen.add(why);
|
|
185
|
+
events.push({ kind, file, why });
|
|
186
|
+
};
|
|
166
187
|
for (const line of nameStatus.split('\n')) {
|
|
167
188
|
const [status, file] = line.split(/\s+/);
|
|
168
|
-
if (status === 'D' && file && TEST_FILE.test(file))
|
|
189
|
+
if (status === 'D' && file && TEST_FILE.test(file)) add(DELETED, file, `test file deleted: ${file}`);
|
|
169
190
|
}
|
|
170
191
|
let current = '';
|
|
171
192
|
let removedAsserts = 0;
|
|
@@ -184,14 +205,14 @@ export function tamperFindings(nameStatus, diff) {
|
|
|
184
205
|
const probe = line.slice(0, 10000); // the patterns can take quadratic time on a long line
|
|
185
206
|
if (/(^|\/)package\.json$/.test(current) && ALWAYS_PASSES.test(probe)) {
|
|
186
207
|
if (line.startsWith('-')) hadFallback = true;
|
|
187
|
-
else if (!hadFallback)
|
|
208
|
+
else if (!hadFallback) add(CHECKS_OFF, current, `the test script was changed to always exit 0 in ${current}`);
|
|
188
209
|
}
|
|
189
210
|
if (line.startsWith('+')) {
|
|
190
|
-
for (const t of TAMPER_ADDED) if (t.re.test(probe))
|
|
211
|
+
for (const t of TAMPER_ADDED) if (t.re.test(probe)) add(t.kind, current, `${t.why} in ${current}`);
|
|
191
212
|
if (!inTest) continue;
|
|
192
|
-
for (const t of TAMPER_ADDED_IN_TESTS) if (t.re.test(probe))
|
|
213
|
+
for (const t of TAMPER_ADDED_IN_TESTS) if (t.re.test(probe)) add(t.kind, current, `${t.why} in ${current}`);
|
|
193
214
|
if (COMMENT.test(line)) {
|
|
194
|
-
if (removed.has(line.replace(COMMENT, '').trim()))
|
|
215
|
+
if (removed.has(line.replace(COMMENT, '').trim())) add(WEAKENED, current, `an assertion was commented out in ${current}`);
|
|
195
216
|
} else if (ASSERT.test(line)) addedAsserts++;
|
|
196
217
|
} else if (line.startsWith('-') && inTest && !COMMENT.test(line) && ASSERT.test(line)) {
|
|
197
218
|
removedAsserts++;
|
|
@@ -199,9 +220,9 @@ export function tamperFindings(nameStatus, diff) {
|
|
|
199
220
|
}
|
|
200
221
|
}
|
|
201
222
|
if (removedAsserts - addedAsserts >= 2) {
|
|
202
|
-
|
|
223
|
+
add(WEAKENED, undefined, `${removedAsserts - addedAsserts} assertions removed from test files`);
|
|
203
224
|
}
|
|
204
|
-
return
|
|
225
|
+
return events;
|
|
205
226
|
}
|
|
206
227
|
|
|
207
228
|
// ---------- redaction for anything that leaves the machine ----------
|