sawbuck 0.1.24 → 0.1.26
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRACT.md +5 -7
- package/README.md +32 -18
- package/bins.mjs +23 -0
- package/contract.mjs +50 -0
- package/init.mjs +3 -17
- package/memory.mjs +3 -10
- package/package.json +24 -7
- package/rank.mjs +2 -2
- package/reflect.mjs +6 -17
- package/sawbuck-hook.mjs +28 -2
- package/sawbuck.mjs +125 -110
- package/GLOSSARY.md +0 -73
- package/run.mjs +0 -47
- package/yank.mjs +0 -87
package/CONTRACT.md
CHANGED
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
*Put your coding agent under contract: a win counts only if it survives verification from the source alone, not because the agent said so. The terms below are not a promise the agent makes, they are a result anyone can re-prove.*
|
|
4
4
|
|
|
5
|
-
**Status:** Draft v0.1 · **License:** Apache-2.0 · **Reference implementation:** [Sawbuck](https://github.com/sawbuck-ai/sawbuck) (the
|
|
5
|
+
**Status:** Draft v0.1 · **License:** Apache-2.0 · **Reference implementation:** [Sawbuck](https://github.com/sawbuck-ai/sawbuck) (the CLI), [promptwheel](https://github.com/promptwheel-ai/promptwheel) (the check), [@promptwheel/logbook](https://github.com/promptwheel-ai/logbook) (the default memory provider)
|
|
6
6
|
|
|
7
|
-
This is an open interface for making an AI coding harness trustworthy. It defines how a loop's wins get verified, what a verdict looks like, what counts as gaming the goalposts, and how a repo's history is carried along. Any harness can emit or consume it without using this code. Sawbuck is a reference
|
|
7
|
+
This is an open interface for making an AI coding harness trustworthy. It defines how a loop's wins get verified, what a verdict looks like, what counts as gaming the goalposts, and how a repo's history is carried along. Any harness can emit or consume it without using this code. Sawbuck is a reference implementation; it is not the only possible one.
|
|
8
8
|
|
|
9
9
|
The contract has one property that holds all of it together: a verdict is **deterministic and reproducible from `(base, head, config)` alone**. No LLM is in the path, and nothing is read from the agent's trajectory, so the result cannot be obfuscated against and does not change when you ask it twice.
|
|
10
10
|
|
|
@@ -39,9 +39,7 @@ The agent consumes this as a per-turn reward it cannot inflate. The harness cons
|
|
|
39
39
|
"head": "e4f5g6h",
|
|
40
40
|
"mode": "refs",
|
|
41
41
|
"repeat": 3,
|
|
42
|
-
"cohort": "ci",
|
|
43
42
|
"verdict": "gamed",
|
|
44
|
-
"subsystems": ["api", "auth"],
|
|
45
43
|
"metrics": [
|
|
46
44
|
{
|
|
47
45
|
"name": "tests_pass",
|
|
@@ -87,11 +85,11 @@ A win is **`gamed`** when it does not survive a **source-only re-run**: revert e
|
|
|
87
85
|
|
|
88
86
|
**Protected terms (declared, optionally enforced).** `evaluatorPaths` is also the declaration surface for immutability: the files a repo names there are its fixed terms, the goalposts the agent does not get to move. The baseline is detection (a win that depended on changing a protected term reads `gamed`). A stronger conformance level *enforces* it: the audit gate (CI, or a hook the agent cannot bypass) fails any change to a protected term unless a human signs off, so the terms cannot be moved mid-game. Note the honest limit: a file cannot be made truly immutable to an agent that has write access to the repo. The guarantee comes from the un-bypassable gate, not from the filesystem.
|
|
89
87
|
|
|
90
|
-
**Declared override (an intentional test change).** The check flags a win that came only from moving the goalposts, not from the source. Sometimes that move is legitimate: a test that asserted the wrong thing, an obsolete test removed, a guard intentionally retired. The sanctioned way through is to DECLARE it, not to silently weaken the gate. A `Sawbuck-Allow: <reason>` trailer on the commit (or `--allow "<reason>"` for a working-tree or CI run) acknowledges the change, so a would-be `gamed` reads `pass` with the acknowledgment recorded in `allowed` (the reason, and the metric or gate names it cleared). The check does not judge intent; it forces intent to be explicit and auditable. An UNdeclared test-only win is still `gamed`; a declared one is on the record, so a reviewer sees the reason instead of a silent green. It covers the edit, the delete, and the new file alike. The declaration lives in the commit, so it is immutable and attributable: revoking it is a review act, not a deletion, and a reasonless trailer does not override.
|
|
88
|
+
**Declared override (an intentional test change).** The check flags a win that came only from moving the goalposts, not from the source. Sometimes that move is legitimate: a test that asserted the wrong thing, an obsolete test removed, a guard intentionally retired. The sanctioned way through is to DECLARE it, not to silently weaken the gate. A `Sawbuck-Allow: <reason>` trailer on the commit (`Promptwheel-Allow:` is the same thing; or `--allow "<reason>"` for a working-tree or CI run) acknowledges the change, so a would-be `gamed` reads `pass` with the acknowledgment recorded in `allowed` (the reason, and the metric or gate names it cleared). The check does not judge intent; it forces intent to be explicit and auditable. An UNdeclared test-only win is still `gamed`; a declared one is on the record, so a reviewer sees the reason instead of a silent green. It covers the edit, the delete, and the new file alike. The declaration lives in the commit, so it is immutable and attributable: revoking it is a review act, not a deletion, and a reasonless trailer does not override.
|
|
91
89
|
|
|
92
90
|
## 4. The context record (the trail knowledge)
|
|
93
91
|
|
|
94
|
-
A conforming
|
|
92
|
+
A conforming implementation also carries the repo's history as deterministic memory: a ledger derived from git (hotspots, reverted approaches, suppressed or weakened tests, fragile areas), surfaced per turn as a ranked slice for the files being touched. It is a projection of git, never a second source of truth, so it is always reconstructable and never drifts from what actually happened.
|
|
95
93
|
|
|
96
94
|
---
|
|
97
95
|
|
|
@@ -102,7 +100,7 @@ A conforming saddle also carries the repo's history as deterministic memory: a l
|
|
|
102
100
|
- A conforming **harness** runs the check as the audit gate (so the agent cannot self-clear its own win).
|
|
103
101
|
- A conforming **context record** is a deterministic projection of git (§4).
|
|
104
102
|
|
|
105
|
-
An implementation may support a subset and say so. The reference
|
|
103
|
+
An implementation may support a subset and say so. The reference implementation (Sawbuck) implements all four.
|
|
106
104
|
|
|
107
105
|
## Versioning
|
|
108
106
|
|
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
**Sign your agent to a contract so it does what you want.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
You write the contract (what a real win looks like); Sawbuck enforces it, re-proving every win from the agent's **source** edits alone, so a result that only passed because the agent edited the test, mocked the grader, or loosened a rule reads `gamed` and goes back. Deterministic and zero-LLM: no model in the verdict, nothing read from the agent's trajectory. Attach it and it carries your repo's history along: what broke in these files before, surfaced before you touch them.
|
|
6
6
|
|
|
7
7
|
> Sawbuck is the reference implementation of the [Verified-Outcome Contract](CONTRACT.md): the open interface for a coding loop you can trust.
|
|
8
8
|
|
|
@@ -10,27 +10,37 @@ A saddle for your coding harness. You write the contract (what a real win looks
|
|
|
10
10
|
# 1 · try it (no install, no commitment): verify your latest change right now
|
|
11
11
|
npx sawbuck check
|
|
12
12
|
|
|
13
|
-
# 2 ·
|
|
13
|
+
# 2 · enforce it: attach once to this repo
|
|
14
14
|
npm i -g sawbuck
|
|
15
|
-
sawbuck attach
|
|
15
|
+
sawbuck attach # every commit (a git post-commit hook)
|
|
16
|
+
sawbuck attach --claude # or every Claude Code turn (a project-scoped Stop hook)
|
|
16
17
|
```
|
|
17
18
|
|
|
18
19
|
**Zero-config:** it auto-detects your test command, so there is nothing to write to start.
|
|
19
20
|
On attach it tells you exactly what it can measure here (and how to fix it if it can't).
|
|
20
|
-
`sawbuck init` is only for *customizing* what
|
|
21
|
-
In CI
|
|
21
|
+
`sawbuck init` is only for *customizing* what counts (coverage, latency, a cost budget, a protected path).
|
|
22
|
+
In CI, the same check runs as the [promptwheel GitHub Action](https://github.com/promptwheel-ai/promptwheel#in-ci-and-in-claude-code) (`uses: promptwheel-ai/promptwheel@v0`); the debt gate below is this repo's own Action. For every project at once: `sawbuck attach --user`.
|
|
22
23
|
|
|
23
|
-
|
|
24
|
+
One entry point each: a person types `sawbuck`; CI and libraries use `promptwheel` (the engine underneath, `import { check }`). `sawbuck check` passes every `promptwheel run` flag through, so you never need both.
|
|
25
|
+
|
|
26
|
+
What a check looks like:
|
|
24
27
|
|
|
25
28
|
```
|
|
26
29
|
$ sawbuck check
|
|
27
|
-
|
|
30
|
+
🐴 🚩 gamed: a win here came from editing the test, config, or grader, not the source.
|
|
31
|
+
It does NOT survive a source-only re-run, so it does not count.
|
|
32
|
+
|
|
33
|
+
$ sawbuck check # after fixing the source and restoring the test
|
|
34
|
+
🐴 ✓ verified: your working change earned it. "tests_pass" held, and the source carried the win.
|
|
28
35
|
```
|
|
29
36
|
|
|
37
|
+
Exit codes: `0` verified · `1` regression · `2` gamed · `3` inconclusive · `64` could not run (not a git repo, no commit, no test command).
|
|
38
|
+
|
|
30
39
|
## What it does
|
|
31
40
|
|
|
32
|
-
- **Verifies the win.** `check` re-runs your metric from the source diff alone. A faked green (a deleted, weakened, or skipped test; a mocked grader; an edited golden file) reads `gamed`. Deterministic, zero-LLM, $0, and it cannot be
|
|
33
|
-
- **
|
|
41
|
+
- **Verifies the win.** `check` re-runs your metric from the source diff alone. A faked green (a deleted, weakened, or skipped test; a mocked grader; an edited golden file; a loosened config) reads `gamed`. Deterministic, zero-LLM, $0, and it cannot be argued with because it never reads the trajectory. Red-teamed across 9 language ecosystems and 26 adversarial scenarios ([promptwheel/bench/REDTEAM.md](https://github.com/promptwheel-ai/promptwheel/blob/main/bench/REDTEAM.md)).
|
|
42
|
+
- **Says what it cannot see.** A cheat that lives inside the source itself (a value hardcoded to the one visible case, logic keyed to the test environment) is invisible to a source-only re-run, by construction. That limit is in [the contract](CONTRACT.md#3-the-integrity-boundary-what-counts-as-gaming), not hidden.
|
|
43
|
+
- **Remembers what broke.** `context` surfaces the relevant history (reverts, fragile areas, test-weakening) for the files you are touching: a ranked trail guide, before you act.
|
|
34
44
|
- **Shows you everything.** `status` is the inventory: what it has caught for you over time, the debt accruing now, and whether it is armed here, in one view. On-demand, so it is quiet until you go looking.
|
|
35
45
|
- **Rides on, does not replace.** Sawbuck is the verify + memory layer. Your agent and your loop stay yours.
|
|
36
46
|
- **Clean on and off.** `attach` touches nothing in your worktree (the hook lives in `.git/`, the record is excluded locally); `detach` is a true inverse. `trace` shows every byte it placed, and `detach --verify` proves the repo came back unchanged.
|
|
@@ -39,16 +49,18 @@ sawbuck: gamed recorded on test.js (the win came from editing the test, no
|
|
|
39
49
|
|
|
40
50
|
| command | what |
|
|
41
51
|
|---|---|
|
|
42
|
-
| `sawbuck
|
|
43
|
-
| `sawbuck
|
|
44
|
-
| `sawbuck
|
|
45
|
-
| `sawbuck
|
|
46
|
-
| `sawbuck debt` | the whole-repo live suppression debt: what is still silenced in HEAD now, disabled tests first, heaviest files ranked |
|
|
52
|
+
| `sawbuck check [--base R --head R] [--repeat N] [--allow "reason"] [--json]` | verify your latest change (working tree if dirty, else the last commit, or a range); zero-config, no attach, no record. Every `promptwheel run` flag passes through: this is the engine with a plain-spoken verdict |
|
|
53
|
+
| `sawbuck init` | write the contract (`promptwheel.config.json`), tuned by what your git history shows actually breaks |
|
|
54
|
+
| `sawbuck attach [--claude \| --user]` | enforce it (git post-commit / project Claude hooks / user-level for every project) |
|
|
55
|
+
| `sawbuck detach [--claude \| --user]` | remove it; no flag reverses whatever is attached here |
|
|
47
56
|
| `sawbuck status` · `sawbuck log` | the inventory (caught over time + debt now + armed state), in one view; and the recent per-check feed |
|
|
57
|
+
| `sawbuck debt` | the whole-repo live suppression debt: what is still silenced in HEAD now, disabled tests first, heaviest files ranked |
|
|
58
|
+
| `sawbuck context` | the ranked relevant history for the current change |
|
|
59
|
+
| `sawbuck frame` | the contract's hard rules (mandates, inherited ones included) as one compact block; silent without a contract |
|
|
60
|
+
| `sawbuck turn` | frame + context in one block: what `attach --claude` runs on every prompt and again right after a compaction |
|
|
48
61
|
| `sawbuck recall` | related notes for your current change from extra memory providers you compose in (bring your own) |
|
|
49
|
-
| `sawbuck run --attempt '<agent cmd>'` | thin harness: load context, run one attempt through the keep-if-verified gate |
|
|
50
62
|
| `sawbuck trace` · `sawbuck doctor` · `detach --verify` | show the footprint; check the attachment resolves; prove a clean removal |
|
|
51
|
-
| `sawbuck where` · `sawbuck statusline` | list every repo it is attached to; a one-line status-line segment (🐴 watching / N caught) |
|
|
63
|
+
| `sawbuck where` · `sawbuck statusline` | list every repo it is attached to; a one-line status-line segment (🐴 watching / N caught / steer %) |
|
|
52
64
|
|
|
53
65
|
## Give your agent the contract
|
|
54
66
|
|
|
@@ -60,7 +72,9 @@ sawbuck: gamed recorded on test.js (the win came from editing the test, no
|
|
|
60
72
|
Run `sawbuck context` first to see what broke in these files before (reverts, weakened tests,
|
|
61
73
|
fragile spots). Never skip, weaken, or delete a test to go green.
|
|
62
74
|
|
|
63
|
-
|
|
75
|
+
**Measured, not assumed.** On a scenario whose natural fix sits inside a protected module, driven past two forced compactions (and again on Haiku, and under "urgent hotfix" pressure): with no rule the agent edited the protected file 17 of 17 times; with a rule in any form, including a mandate, 0 of 45. The model honors a rule it was told. The gate is for the rule nobody told it, the session that outlived the summary, and the proof that it held: `attach --claude` re-reads the mandates from the repo on every turn, re-impresses them after a compaction (`sawbuck turn`), and blocks a turn that leaves a protected path changed. Numbers and method: [sawbuck.ai/contracts](https://sawbuck.ai/contracts).
|
|
76
|
+
|
|
77
|
+
Without a `promptwheel.config.json` the check measures your detected test command. To make it count more (coverage, lint, latency, an eval) or to protect a path with a mandate, `sawbuck init` writes the config and explains each line with evidence from your history.
|
|
64
78
|
|
|
65
79
|
## Gate debt on pull requests
|
|
66
80
|
|
|
@@ -87,7 +101,7 @@ Debt also rises on its own: once attached, the per-turn `context` hook surfaces
|
|
|
87
101
|
|
|
88
102
|
Sawbuck is thin glue over two roles, each behind an open interface that points in opposite directions:
|
|
89
103
|
|
|
90
|
-
- **the check** : the deterministic outcome gate, a published standard (the [Verified-Outcome Contract](CONTRACT.md)) that others conform to. Reference implementation: **promptwheel**. You plug in *what* to measure (your tests, lint, coverage, an eval); the
|
|
104
|
+
- **the check** : the deterministic outcome gate, a published standard (the [Verified-Outcome Contract](CONTRACT.md)) that others conform to. Reference implementation: **promptwheel**. You plug in *what* to measure (your tests, lint, coverage, an eval); the source re-proof is the fixed part. Embed it with `import { check }`, or check any tool's verdict with `promptwheel conformance`.
|
|
91
105
|
- **the memory** : the relevant history for the files you touch, plus the live debt. A socket you can fill. Default provider: **@promptwheel/logbook** (git history as agent memory; also ships an MCP server). Sawbuck reads it through one adapter, so you can *compose in your own* memory: `SAWBUCK_MEMORY=logbook,obsidian` keeps logbook as the git-truth + debt provider and adds your notes, then `sawbuck recall`. Four reference providers ship in `providers/`: **obsidian** (local notes vault; set `OBSIDIAN_VAULT`), **local** (a plain JSON file, zero setup), **claude** (read-only: your project's `CLAUDE.md` / `.claude` memory, already there, nothing to configure), and **mem0** (semantic memory; the one that reaches the network, so it needs a `MEM0_API_KEY` or a `~/.mem0` key file, unlike the offline default; verified against Mem0's v3 API). Any bin that speaks the three contract verbs works; `sawbuck memory-conformance <bin>` checks one. Composed providers are **folder-scoped and isolated by default**: each repo sees only its own memory, never a sibling's. Drop a `.sawbuck-root` marker (or set `SAWBUCK_MEMORY_ROOT`) in a parent folder to let repos under it inherit that folder's shared memory, top down. logbook is the default, not a hard requirement of the idea.
|
|
92
106
|
|
|
93
107
|
In one line: the check is a standard we publish for others to meet; the memory is a socket we expose for others to fill. Sawbuck composes the two.
|
package/bins.mjs
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
// bins.mjs: resolve the two engine CLIs once, for every sawbuck entry point.
|
|
2
|
+
// Order: env override (PW_BIN / LB_BIN), the installed package, then the dev sibling checkout.
|
|
3
|
+
// The memory engine is reached only through memory.mjs (the Memory Contract seam); LB lives
|
|
4
|
+
// here so the resolver is written once, not per file.
|
|
5
|
+
|
|
6
|
+
import { dirname, join } from 'node:path';
|
|
7
|
+
import { fileURLToPath } from 'node:url';
|
|
8
|
+
import { createRequire } from 'node:module';
|
|
9
|
+
|
|
10
|
+
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
11
|
+
const require = createRequire(import.meta.url);
|
|
12
|
+
|
|
13
|
+
function resolveBin(envVar, pkgSubpath, devPath) {
|
|
14
|
+
if (process.env[envVar]) return process.env[envVar];
|
|
15
|
+
// deep bin subpath (no-exports pkgs, e.g. promptwheel), then the bare package name
|
|
16
|
+
// (pkgs whose exports map "." to the bin, e.g. @promptwheel/logbook), then the dev sibling.
|
|
17
|
+
const bare = pkgSubpath.replace(/\/bin\/[^/]*$/, '');
|
|
18
|
+
for (const spec of [pkgSubpath, bare]) { try { return require.resolve(spec); } catch {} }
|
|
19
|
+
return devPath;
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
export const PW = resolveBin('PW_BIN', 'promptwheel/bin/promptwheel.mjs', join(HERE, '..', 'promptwheel', 'bin', 'promptwheel.mjs'));
|
|
23
|
+
export const LB = resolveBin('LB_BIN', '@promptwheel/logbook/bin/logbook.mjs', join(HERE, '..', 'logbook', 'bin', 'logbook.mjs'));
|
package/contract.mjs
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
// contract.mjs: the contract's hard rules, read the way promptwheel reads them. Shared by the CLI
|
|
2
|
+
// (frame / turn) and the Stop hook (the turn-level wall), so there is one resolver and one renderer.
|
|
3
|
+
// Read-only, never throws, zero-dep.
|
|
4
|
+
import { readFileSync } from 'node:fs';
|
|
5
|
+
import { join, dirname, resolve } from 'node:path';
|
|
6
|
+
|
|
7
|
+
// mandates accumulate down the `extends` chain; a local mandate of the same name overrides an inherited one
|
|
8
|
+
export function resolvedMandates(repo) {
|
|
9
|
+
const seen = new Set(); const byName = new Map();
|
|
10
|
+
const load = (file) => {
|
|
11
|
+
const p = resolve(file); if (seen.has(p)) return; seen.add(p);
|
|
12
|
+
let cfg = null; try { cfg = JSON.parse(readFileSync(p, 'utf8')); } catch { return; }
|
|
13
|
+
if (!cfg || typeof cfg !== 'object') return;
|
|
14
|
+
const ext = cfg.extends == null ? [] : Array.isArray(cfg.extends) ? cfg.extends : [cfg.extends];
|
|
15
|
+
for (const e of ext) if (typeof e === 'string') load(resolve(dirname(p), e));
|
|
16
|
+
for (const m of (Array.isArray(cfg.mandates) ? cfg.mandates : [])) if (m && m.name) byName.set(String(m.name), m);
|
|
17
|
+
};
|
|
18
|
+
load(join(repo, 'promptwheel.config.json'));
|
|
19
|
+
return [...byName.values()].filter((m) => Array.isArray(m.paths) && m.paths.some((s) => typeof s === 'string' && s.trim()));
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
// the same glob dialect as promptwheel's evaluatorPaths / mandates (ported verbatim): a bare name
|
|
23
|
+
// matches that basename at any depth; a pattern with "/" is anchored at the repo root; "*" stops at
|
|
24
|
+
// "/", "**" crosses it, "?" is one char; a leading "/" anchors to the root explicitly.
|
|
25
|
+
export function globToRe(glob) {
|
|
26
|
+
let g = glob; const anchored = g.startsWith('/'); if (anchored) g = g.slice(1);
|
|
27
|
+
const body = g.replace(/[.+^${}()|[\]\\]/g, '\\$&')
|
|
28
|
+
.replace(/\*\*/g, '\x00').replace(/\*/g, '[^/]*').replace(/\?/g, '[^/]').replace(/\x00/g, '.*');
|
|
29
|
+
const pre = (anchored || glob.includes('/')) ? '^' : '(^|/)';
|
|
30
|
+
return new RegExp(`${pre}${body}$`);
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
// which changed paths violate which mandate: [{ name, reason, files }]
|
|
34
|
+
export function mandateHits(repo, changedPaths) {
|
|
35
|
+
const out = [];
|
|
36
|
+
for (const m of resolvedMandates(repo)) {
|
|
37
|
+
const res = m.paths.filter((s) => typeof s === 'string' && s.trim()).map(globToRe);
|
|
38
|
+
const files = changedPaths.filter((f) => res.some((re) => re.test(f)));
|
|
39
|
+
if (files.length) out.push({ name: String(m.name), reason: m.reason ? String(m.reason).slice(0, 300) : '', files: files.slice(0, 20) });
|
|
40
|
+
}
|
|
41
|
+
return out;
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
// the prompt face of the contract: one compact block, or '' when there is no contract
|
|
45
|
+
export function frameBlock(repo) {
|
|
46
|
+
const mandates = resolvedMandates(repo);
|
|
47
|
+
if (!mandates.length) return '';
|
|
48
|
+
const lines = mandates.map((m) => ` ${m.paths.filter((s) => typeof s === 'string' && s.trim()).join(' · ')}${m.reason ? ` (${String(m.reason).slice(0, 160)})` : ''}`);
|
|
49
|
+
return '▣ contract (sawbuck) · hard rules in force this session, enforced by the gate on every turn:\n protected, do not touch:\n' + lines.join('\n');
|
|
50
|
+
}
|
package/init.mjs
CHANGED
|
@@ -17,23 +17,9 @@
|
|
|
17
17
|
|
|
18
18
|
import { execFileSync } from 'node:child_process';
|
|
19
19
|
import { readFileSync, writeFileSync, existsSync, unlinkSync } from 'node:fs';
|
|
20
|
-
import {
|
|
21
|
-
import { join, dirname } from 'node:path';
|
|
22
|
-
import { fileURLToPath } from 'node:url';
|
|
20
|
+
import { join } from 'node:path';
|
|
23
21
|
import { createInterface } from 'node:readline';
|
|
24
|
-
|
|
25
|
-
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
26
|
-
const require = createRequire(import.meta.url);
|
|
27
|
-
const engineBin = (envVar, pkgSubpath, devPath) => {
|
|
28
|
-
if (process.env[envVar]) return process.env[envVar];
|
|
29
|
-
// deep bin subpath (no-exports pkgs, e.g. promptwheel), then the bare package name
|
|
30
|
-
// (pkgs whose exports map "." to the bin, e.g. @promptwheel/logbook), then the dev sibling.
|
|
31
|
-
const bare = pkgSubpath.replace(/\/bin\/[^/]*$/, '');
|
|
32
|
-
for (const spec of [pkgSubpath, bare]) { try { return require.resolve(spec); } catch {} }
|
|
33
|
-
return devPath;
|
|
34
|
-
};
|
|
35
|
-
const PW = engineBin('PW_BIN', 'promptwheel/bin/promptwheel.mjs', join(HERE, '..', 'promptwheel', 'bin', 'promptwheel.mjs'));
|
|
36
|
-
const LB = engineBin('LB_BIN', '@promptwheel/logbook/bin/logbook.mjs', join(HERE, '..', 'logbook', 'bin', 'logbook.mjs'));
|
|
22
|
+
import { PW, LB } from './bins.mjs';
|
|
37
23
|
|
|
38
24
|
const argv = process.argv.slice(2);
|
|
39
25
|
const has = (f) => argv.includes(f);
|
|
@@ -145,7 +131,7 @@ function report() {
|
|
|
145
131
|
for (const m of cfg.metrics) out(' - ' + describeMetric(m));
|
|
146
132
|
if (evidence) { out('\n investigated this repo:'); for (const l of evidenceLines()) out(l); }
|
|
147
133
|
out('\n next: sawbuck attach # enforce the contract every commit');
|
|
148
|
-
out(' sawbuck
|
|
134
|
+
out(' sawbuck check # verify your uncommitted changes now');
|
|
149
135
|
}
|
|
150
136
|
|
|
151
137
|
function runWizard() {
|
package/memory.mjs
CHANGED
|
@@ -14,28 +14,21 @@
|
|
|
14
14
|
// never by printing []. git-truth stays authoritative; other providers add supplementary notes.
|
|
15
15
|
|
|
16
16
|
import { execFileSync } from 'node:child_process';
|
|
17
|
-
import { createRequire } from 'node:module';
|
|
18
17
|
import { existsSync } from 'node:fs';
|
|
18
|
+
import { LB } from './bins.mjs';
|
|
19
19
|
import { join, dirname, relative, resolve, basename, sep } from 'node:path';
|
|
20
20
|
import { fileURLToPath } from 'node:url';
|
|
21
21
|
import { homedir } from 'node:os';
|
|
22
22
|
|
|
23
23
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
24
|
-
const require = createRequire(import.meta.url);
|
|
25
24
|
|
|
26
|
-
function logbookBin() {
|
|
27
|
-
if (process.env.SAWBUCK_MEMORY_BIN) return process.env.SAWBUCK_MEMORY_BIN; // back-compat single-provider override
|
|
28
|
-
if (process.env.LB_BIN) return process.env.LB_BIN;
|
|
29
|
-
for (const spec of ['@promptwheel/logbook/bin/logbook.mjs', '@promptwheel/logbook']) { try { return require.resolve(spec); } catch {} }
|
|
30
|
-
return join(HERE, '..', 'logbook', 'bin', 'logbook.mjs');
|
|
31
|
-
}
|
|
32
25
|
const envKey = (name) => 'SAWBUCK_MEMORY_' + name.toUpperCase().replace(/[^A-Z0-9]/g, '_') + '_BIN';
|
|
33
26
|
// the ordered provider list. Each: { name, bin, dialect: 'logbook' | 'contract' }.
|
|
34
27
|
const SHIPPED = new Set(['obsidian', 'mem0', 'local', 'claude']); // reference contract providers that ship in providers/
|
|
35
28
|
export function providers() {
|
|
36
29
|
const spec = (process.env.SAWBUCK_MEMORY || 'logbook').split(',').map((s) => s.trim()).filter(Boolean);
|
|
37
30
|
return spec.map((name) => {
|
|
38
|
-
if (name === 'logbook') return { name, bin:
|
|
31
|
+
if (name === 'logbook') return { name, bin: LB, dialect: 'logbook' };
|
|
39
32
|
if (SHIPPED.has(name)) return { name, bin: process.env[envKey(name)] || join(HERE, 'providers', name + '.mjs'), dialect: 'contract' };
|
|
40
33
|
return { name, bin: process.env[envKey(name)] || name, dialect: 'contract' }; // a bin path, or a name with an env-set bin
|
|
41
34
|
});
|
|
@@ -103,7 +96,7 @@ const classify = (r, source) => ({
|
|
|
103
96
|
file: r.file, line: r.line, kind: String(r.kind ?? '').trim(), content: r.content ?? '',
|
|
104
97
|
since: r.since ?? null, sha: r.sha ?? null,
|
|
105
98
|
documented: r.content ? isDocumented(r.content) : false,
|
|
106
|
-
testSkip: isTestSkip(r.kind ?? r.content ?? ''),
|
|
99
|
+
testSkip: typeof r.testSkip === 'boolean' ? r.testSkip : isTestSkip(r.kind ?? r.content ?? ''), // logbook >=0.9.8 classifies it; the regex is the fallback for older providers
|
|
107
100
|
source,
|
|
108
101
|
});
|
|
109
102
|
|
package/package.json
CHANGED
|
@@ -1,13 +1,31 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sawbuck",
|
|
3
|
-
"version": "0.1.
|
|
4
|
-
"description": "Sign your agent to a contract
|
|
3
|
+
"version": "0.1.26",
|
|
4
|
+
"description": "Sign your agent to a contract: a deterministic, zero-LLM check re-proves every win from the agent's source edits alone, so a green that came from editing the test, grader, or config is caught and sent back. Works in Claude Code, Codex, git hooks, and CI.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "Apache-2.0",
|
|
7
|
+
"engines": {
|
|
8
|
+
"node": ">=18"
|
|
9
|
+
},
|
|
7
10
|
"repository": {
|
|
8
11
|
"type": "git",
|
|
9
12
|
"url": "git+https://github.com/sawbuck-ai/sawbuck.git"
|
|
10
13
|
},
|
|
14
|
+
"homepage": "https://sawbuck.ai",
|
|
15
|
+
"bugs": {
|
|
16
|
+
"url": "https://github.com/sawbuck-ai/sawbuck/issues"
|
|
17
|
+
},
|
|
18
|
+
"keywords": [
|
|
19
|
+
"ai-agents",
|
|
20
|
+
"claude-code",
|
|
21
|
+
"codex",
|
|
22
|
+
"reward-hacking",
|
|
23
|
+
"code-quality",
|
|
24
|
+
"ci",
|
|
25
|
+
"git-hooks",
|
|
26
|
+
"developer-tools",
|
|
27
|
+
"promptwheel"
|
|
28
|
+
],
|
|
11
29
|
"scripts": {
|
|
12
30
|
"test": "node --test"
|
|
13
31
|
},
|
|
@@ -15,11 +33,13 @@
|
|
|
15
33
|
"sawbuck": "sawbuck.mjs"
|
|
16
34
|
},
|
|
17
35
|
"dependencies": {
|
|
18
|
-
"promptwheel": "^0.
|
|
19
|
-
"
|
|
36
|
+
"@promptwheel/logbook": "^0.9.6",
|
|
37
|
+
"promptwheel": "^0.7.5"
|
|
20
38
|
},
|
|
21
39
|
"files": [
|
|
22
40
|
"sawbuck.mjs",
|
|
41
|
+
"bins.mjs",
|
|
42
|
+
"contract.mjs",
|
|
23
43
|
"init.mjs",
|
|
24
44
|
"reflect.mjs",
|
|
25
45
|
"dispatch.mjs",
|
|
@@ -28,14 +48,11 @@
|
|
|
28
48
|
"debt.mjs",
|
|
29
49
|
"status.mjs",
|
|
30
50
|
"memory.mjs",
|
|
31
|
-
"yank.mjs",
|
|
32
51
|
"providers/obsidian.mjs",
|
|
33
52
|
"providers/mem0.mjs",
|
|
34
53
|
"providers/local.mjs",
|
|
35
54
|
"providers/claude.mjs",
|
|
36
|
-
"run.mjs",
|
|
37
55
|
"README.md",
|
|
38
|
-
"GLOSSARY.md",
|
|
39
56
|
"CONTRACT.md"
|
|
40
57
|
]
|
|
41
58
|
}
|
package/rank.mjs
CHANGED
|
@@ -45,7 +45,7 @@ const codeish = (p) => /\.(m?[jt]sx?|cjs|py|go|rs|rb|java|kt|c|cc|cpp|h|hpp|php|
|
|
|
45
45
|
// eslint-disable). The relevance lead: sharpen the signal, don't flatten it.
|
|
46
46
|
const isTest = (p) => /(^|\/)(tests?|__tests__|specs?)\//i.test(p) || /[._-](test|spec)\.[a-z0-9]+$/i.test(p) || /(^|\/)test_[^/]+\.py$/i.test(p);
|
|
47
47
|
|
|
48
|
-
// Per-language suppression normalization.
|
|
48
|
+
// Per-language suppression normalization. A 100-repo study found a suppression's
|
|
49
49
|
// raw count is an idiom artifact, not a risk level: a `# noqa` is background noise in Python (141
|
|
50
50
|
// markers/repo) but a `t.Skip` is notable in Go. So a hit is discounted by how ubiquitous its
|
|
51
51
|
// idiom is in the FILE's language. Discount-only [0.4, 1]: common idioms shrink, rare ones stay at
|
|
@@ -80,7 +80,7 @@ const normMult = (p, kind) => {
|
|
|
80
80
|
// with SAWBUCK_DEBT_MIN. This is how debt rises by itself in the per-turn context, no command run.
|
|
81
81
|
const DEBT_MIN = Math.max(1, +(process.env.SAWBUCK_DEBT_MIN) || 6);
|
|
82
82
|
|
|
83
|
-
// Still-live filter
|
|
83
|
+
// Still-live filter + ambient debt, via the memory adapter (the Memory Contract's `debt`
|
|
84
84
|
// capability), scoped to the touched paths so the per-turn hook stays cheap. A suppression on a path
|
|
85
85
|
// now CLEAN of suppressions is churn, not current risk (the study: AutoMapper carries 4 live
|
|
86
86
|
// `[Ignore]` vs ~158/repo across C# history). The adapter runs the provider's own detection + gating
|
package/reflect.mjs
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
// reflect.mjs, the Sawbuck feedback edge
|
|
2
|
+
// reflect.mjs, the Sawbuck feedback edge.
|
|
3
3
|
//
|
|
4
4
|
// Run the check (the promptwheel engine) on the latest commit; on a gamed or
|
|
5
5
|
// regression result, deterministic ground truth, record a finding in the
|
|
@@ -18,23 +18,12 @@ import { execFileSync } from 'node:child_process';
|
|
|
18
18
|
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
|
|
19
19
|
import { join, dirname } from 'node:path';
|
|
20
20
|
import { fileURLToPath } from 'node:url';
|
|
21
|
-
import { createRequire } from 'node:module';
|
|
22
21
|
import { record as memRecord } from './memory.mjs';
|
|
22
|
+
import { PW } from './bins.mjs';
|
|
23
23
|
|
|
24
24
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
25
|
-
|
|
26
|
-
//
|
|
27
|
-
// install), then the monorepo sibling (dev checkout). The MEMORY engine is reached only
|
|
28
|
-
// through memory.mjs (the Memory Contract seam), never resolved or shelled directly here.
|
|
29
|
-
const engineBin = (envVar, pkgSubpath, devPath) => {
|
|
30
|
-
if (process.env[envVar]) return process.env[envVar];
|
|
31
|
-
// deep bin subpath (no-exports pkgs, e.g. promptwheel), then the bare package name
|
|
32
|
-
// (pkgs whose exports map "." to the bin, e.g. @promptwheel/logbook), then the dev sibling.
|
|
33
|
-
const bare = pkgSubpath.replace(/\/bin\/[^/]*$/, '');
|
|
34
|
-
for (const spec of [pkgSubpath, bare]) { try { return require.resolve(spec); } catch {} }
|
|
35
|
-
return devPath;
|
|
36
|
-
};
|
|
37
|
-
const PW = engineBin('PW_BIN', 'promptwheel/bin/promptwheel.mjs', join(HERE, '..', 'promptwheel', 'bin', 'promptwheel.mjs'));
|
|
25
|
+
// The MEMORY engine is reached only through memory.mjs (the Memory Contract seam), never
|
|
26
|
+
// resolved or shelled directly here.
|
|
38
27
|
|
|
39
28
|
const flag = (name, def) => { const i = process.argv.indexOf(name); return i > -1 ? process.argv[i + 1] : def; };
|
|
40
29
|
const repo = flag('--repo', process.cwd());
|
|
@@ -83,8 +72,8 @@ const reason = kind === 'gamed'
|
|
|
83
72
|
|
|
84
73
|
// --- 3) record the finding in memory (unreviewed, machine-authored note) ----
|
|
85
74
|
// The Memory Contract's `record` capability, through the adapter (no direct engine shell here).
|
|
86
|
-
//
|
|
87
|
-
//
|
|
75
|
+
// Kept simple: a commit-attached digest note. A later pass could add --span grounding
|
|
76
|
+
// and true coupling (the fix diff -> "change X also needs Y").
|
|
88
77
|
memRecord(repo, { sha, reason: reason.slice(0, 280), side: 'diff', evidenceFile, by: 'sawbuck' });
|
|
89
78
|
|
|
90
79
|
// --- 4) tick the results record (the visible proof-of-work) -----------------
|
package/sawbuck-hook.mjs
CHANGED
|
@@ -1,15 +1,16 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
// sawbuck-hook.mjs: the Sawbuck Claude Code Stop-hook entry
|
|
2
|
+
// sawbuck-hook.mjs: the Sawbuck Claude Code Stop-hook entry.
|
|
3
3
|
// Each turn: if a new commit landed, run reflect on it; then render the dispatch
|
|
4
4
|
// on a fresh catch OR every N turns (config). State in .sawbuck/state.json.
|
|
5
5
|
// Zero-dep. Usage (wired as a Stop hook): node sawbuck-hook.mjs --repo <path>
|
|
6
6
|
|
|
7
7
|
import { execFileSync } from 'node:child_process';
|
|
8
|
-
import { readFileSync, writeFileSync, mkdirSync, existsSync } from 'node:fs';
|
|
8
|
+
import { readFileSync, writeFileSync, mkdirSync, existsSync, appendFileSync } from 'node:fs';
|
|
9
9
|
import { join, dirname } from 'node:path';
|
|
10
10
|
import { fileURLToPath } from 'node:url';
|
|
11
11
|
|
|
12
12
|
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
13
|
+
import { mandateHits } from './contract.mjs';
|
|
13
14
|
const flag = (n, d) => { const i = process.argv.indexOf(n); return i > -1 ? process.argv[i + 1] : d; };
|
|
14
15
|
const repo = flag('--repo', process.cwd());
|
|
15
16
|
const by = flag('--by', ''); // attribution from the harness wiring: the Claude hook passes 'claude', the raw-git hook passes nothing
|
|
@@ -46,6 +47,31 @@ try { rec = JSON.parse(readFileSync(join(stDir, 'results.json'), 'utf8')); } cat
|
|
|
46
47
|
const total = rec ? (rec.regressions || 0) + (rec.gamed || 0) : 0;
|
|
47
48
|
const saveState = () => { try { mkdirSync(stDir, { recursive: true }); writeFileSync(stPath, JSON.stringify(st, null, 2) + '\n'); } catch {} };
|
|
48
49
|
|
|
50
|
+
// 2b) THE WALL AT TURN LEVEL (Claude Code Stop hook only): a mandate is checked against the WORKING
|
|
51
|
+
// tree on every stop, not only on commits, because an agent usually ends a turn with uncommitted
|
|
52
|
+
// edits. A deterministic path match, milliseconds, no metric run. Block the stop (bounded per prompt)
|
|
53
|
+
// and send the agent back to revert the protected files and make the fix elsewhere.
|
|
54
|
+
if (stop && !process.env.SAWBUCK_NO_ENFORCE) {
|
|
55
|
+
const changed = [...new Set([...git('diff', '--name-only', 'HEAD').split('\n'), ...git('ls-files', '--others', '--exclude-standard').split('\n')].filter(Boolean))];
|
|
56
|
+
const hits = changed.length ? mandateHits(repo, changed) : [];
|
|
57
|
+
if (hits.length) {
|
|
58
|
+
const MAX_BLOCKS = +(cfg.enforce?.maxBlocks) || 3;
|
|
59
|
+
const promptId = hookInput.prompt_id || 'none';
|
|
60
|
+
if (st.mandatePromptId !== promptId) { st.mandatePromptId = promptId; st.mandateBlocks = 0; }
|
|
61
|
+
const desc = hits.map((h) => `"${h.name}"${h.reason ? ' (' + h.reason + ')' : ''}: ${h.files.join(', ')}`).join('; ');
|
|
62
|
+
try { mkdirSync(stDir, { recursive: true }); appendFileSync(join(stDir, 'activity.jsonl'), JSON.stringify({ at: new Date().toISOString(), kind: 'mandate', file: hits[0].files[0], reason: `mandate ${hits[0].name} violated in the working tree`, by: by || 'hook' }) + '\n'); } catch {}
|
|
63
|
+
if ((st.mandateBlocks || 0) < MAX_BLOCKS) {
|
|
64
|
+
st.mandateBlocks = (st.mandateBlocks || 0) + 1;
|
|
65
|
+
saveState();
|
|
66
|
+
process.stdout.write(JSON.stringify({ decision: 'block', reason: `Sawbuck: a mandate in the contract was violated, the turn is not done. Protected paths changed: ${desc}. Revert every change under the protected paths and make the fix elsewhere (the rule is a wall, not a reminder; the gate fails any change that touches them).` }));
|
|
67
|
+
process.exit(0);
|
|
68
|
+
}
|
|
69
|
+
saveState();
|
|
70
|
+
process.stdout.write(JSON.stringify({ systemMessage: `Sawbuck stopped blocking after ${MAX_BLOCKS} tries: protected paths are still changed (${desc}).` }));
|
|
71
|
+
process.exit(0);
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
|
|
49
75
|
// 3) CONTRACT ENFORCEMENT (Claude Code Stop hook only): if THIS turn's commit was gamed,
|
|
50
76
|
// block the stop and send the agent back to earn the green in the source. Bounded per
|
|
51
77
|
// prompt, because Claude Code's built-in block cap resets on every tool call and so cannot
|
package/sawbuck.mjs
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
|
-
// sawbuck, the
|
|
2
|
+
// sawbuck, the CLI. Zero-dep.
|
|
3
3
|
// `attach` wires Sawbuck's reflex into a repo (a git post-commit hook by default,
|
|
4
4
|
// or Claude Code hooks); `detach` removes it and leaves the repo exactly as it
|
|
5
5
|
// was. Isolation invariant: the repo never imports Sawbuck, and detach is a true
|
|
@@ -17,18 +17,13 @@ import { readFileSync, writeFileSync, existsSync, chmodSync, mkdirSync, unlinkSy
|
|
|
17
17
|
import { join, dirname, resolve } from 'node:path';
|
|
18
18
|
import { fileURLToPath } from 'node:url';
|
|
19
19
|
import { homedir } from 'node:os';
|
|
20
|
-
import { createRequire } from 'node:module';
|
|
21
20
|
import { recall as memRecall, conform as memConform } from './memory.mjs';
|
|
22
|
-
import {
|
|
21
|
+
import { PW } from './bins.mjs';
|
|
22
|
+
import { frameBlock } from './contract.mjs';
|
|
23
23
|
|
|
24
24
|
const SELF = fileURLToPath(import.meta.url);
|
|
25
25
|
const HERE = dirname(SELF);
|
|
26
26
|
const HOOK = join(HERE, 'sawbuck-hook.mjs');
|
|
27
|
-
const require = createRequire(import.meta.url);
|
|
28
|
-
// resolve the promptwheel engine bin: env override, installed package, then the dev sibling
|
|
29
|
-
const PW = process.env.PW_BIN
|
|
30
|
-
|| ['promptwheel/bin/promptwheel.mjs', 'promptwheel'].reduce((r, s) => { try { return r || require.resolve(s); } catch { return r; } }, '')
|
|
31
|
-
|| join(HERE, '..', 'promptwheel', 'bin', 'promptwheel.mjs');
|
|
32
27
|
// run the engine and parse its JSON verdict (it prints JSON even when it exits nonzero)
|
|
33
28
|
function gateReport(repo, a) {
|
|
34
29
|
let o = '', code = 0;
|
|
@@ -61,51 +56,48 @@ const dry = flags.has('--dry-run');
|
|
|
61
56
|
let version = '0.0.0';
|
|
62
57
|
try { version = JSON.parse(readFileSync(join(HERE, 'package.json'), 'utf8')).version; } catch {}
|
|
63
58
|
|
|
64
|
-
const help = `sawbuck ${version}:
|
|
59
|
+
const help = `sawbuck ${version}: sign your agent to a contract
|
|
65
60
|
|
|
66
61
|
Usage:
|
|
62
|
+
sawbuck check [repo] [--base R --head R] [--repeat N] [--allow "<reason>"] [--json]
|
|
63
|
+
Verify your latest change right now (working tree if dirty, else the last
|
|
64
|
+
commit, or the range you name). Zero-config, no attach, no hook. The
|
|
65
|
+
fastest way to see it work. Every promptwheel run flag passes through.
|
|
66
|
+
Exit 0 verified · 1 regression · 2 gamed · 3 inconclusive · 64 could not run
|
|
67
67
|
sawbuck init [--yes] [--wizard] [--no-investigate] [--no-agent-block]
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
--claude instead wire project Claude Code hooks (writes .claude/settings.local.json)
|
|
68
|
+
Write the contract (promptwheel.config.json), tuned and explained by what your
|
|
69
|
+
git history shows actually breaks, plus a short contract guide in AGENTS.md
|
|
70
|
+
(--no-agent-block to skip). Wizard in a terminal, non-interactive in a pipe/CI (or --yes).
|
|
71
|
+
sawbuck attach [repo] Enforce the contract on every commit (a git post-commit hook)
|
|
72
|
+
--claude instead on every Claude Code turn (project hooks in .claude/settings.local.json)
|
|
74
73
|
--shared with --claude, write the tracked .claude/settings.json (team-shared, visible in git)
|
|
75
|
-
--user
|
|
74
|
+
--user user-level Claude hooks that follow your cwd (attach once, every project)
|
|
76
75
|
--dry-run print the exact footprint it would leave, and write nothing
|
|
77
76
|
sawbuck detach [repo] [--claude | --user] [--all] [--verify] [--purge]
|
|
78
|
-
Remove
|
|
79
|
-
--purge also deletes the record (.sawbuck
|
|
80
|
-
sawbuck
|
|
81
|
-
sawbuck
|
|
77
|
+
Remove it and restore the repo (no flag: whatever is attached here); --verify
|
|
78
|
+
proves it came back clean; --purge also deletes the record (.sawbuck/)
|
|
79
|
+
sawbuck status [repo] The inventory: caught over time + debt accruing now + whether it is armed. --json
|
|
80
|
+
sawbuck log [repo] [-n N] Recent checks, newest first (verified / gamed / regression)
|
|
81
|
+
sawbuck debt [repo] [--since REF] [--fail-on undocumented|test|any|N]
|
|
82
|
+
The live suppression debt in HEAD, risk-ranked, undocumented flagged. --since REF
|
|
83
|
+
shows only what a branch adds; --fail-on exits nonzero (a CI gate)
|
|
84
|
+
sawbuck context [repo] Relevant history for the files you are changing (reverts, weakened tests, hot spots)
|
|
85
|
+
sawbuck frame [repo] The contract's hard rules (mandates, inherited ones included) as one compact block;
|
|
86
|
+
silent without a contract (--speak explains)
|
|
87
|
+
sawbuck turn [repo] frame + context in one block: what attach --claude runs on every prompt
|
|
88
|
+
|
|
89
|
+
sawbuck trace [repo] Show every file Sawbuck has placed in this repo (the footprint)
|
|
90
|
+
sawbuck doctor [repo] Check the attachment still resolves to a live sawbuck; --all for every repo
|
|
82
91
|
sawbuck where List every repo Sawbuck is attached to
|
|
83
|
-
sawbuck statusline One-line segment for your status line (🐴 watching | N caught)
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
sawbuck check [repo] Try it: verify your latest change right now (working tree if dirty, else the
|
|
88
|
-
last commit). Zero-config, no attach, no hook. The fastest way to see it work.
|
|
89
|
-
sawbuck rank [--base R --head R] [--file P ...]
|
|
90
|
-
Show the relevant past findings for the paths a change touches
|
|
91
|
-
sawbuck context [repo] Ranked relevant history for your current working changes (the read face)
|
|
92
|
-
sawbuck recall [repo] Related notes for your current changes from extra memory providers
|
|
93
|
-
(SAWBUCK_MEMORY=logbook,obsidian + OBSIDIAN_VAULT); supplementary to context
|
|
92
|
+
sawbuck statusline One-line segment for your status line (🐴 watching | N caught | steer %)
|
|
93
|
+
|
|
94
|
+
sawbuck recall [repo] Related notes for your changes from extra memory providers
|
|
95
|
+
(SAWBUCK_MEMORY=logbook,obsidian + OBSIDIAN_VAULT), alongside context
|
|
94
96
|
sawbuck memory-conformance <bin>
|
|
95
97
|
Check a provider bin speaks the Memory Contract (recall/debt/record)
|
|
96
|
-
sawbuck debt [repo] [--since REF] [--fail-on undocumented|test|any|N]
|
|
97
|
-
The whole-repo live suppression debt: what is still silenced in HEAD now,
|
|
98
|
-
risk-ranked (severity x file fragility), undocumented flagged. --since REF
|
|
99
|
-
shows only the debt a branch adds; --fail-on exits nonzero for a CI gate
|
|
100
|
-
sawbuck run --attempt '<cmd>' [--loops N]
|
|
101
|
-
Thin harness: load context, run one attempt through the keep-if-verified gate
|
|
102
|
-
sawbuck yank [--session <path>] [--json]
|
|
103
|
-
The steering meter: how much you had to steer (yank the reins) this session,
|
|
104
|
-
read from your Claude Code transcript. The check guards the turns you did not
|
|
105
|
-
steer — a miss there is the risk. Yank rate falling = the reins getting looser.
|
|
106
98
|
sawbuck --help | --version
|
|
107
99
|
|
|
108
|
-
|
|
100
|
+
Engines: promptwheel (the check) + @promptwheel/logbook (memory). Zero-LLM, offline, free.`;
|
|
109
101
|
|
|
110
102
|
const die = (msg) => { process.stderr.write(`sawbuck: ${msg}\n`); process.exit(1); };
|
|
111
103
|
const out = (s) => process.stdout.write(s.endsWith('\n') ? s : s + '\n');
|
|
@@ -130,22 +122,42 @@ switch (cmd) {
|
|
|
130
122
|
// one-shot: gate your latest change right now (working tree if dirty, else the last commit),
|
|
131
123
|
// zero-config (the engine auto-detects your test command), and say plainly whether the win was
|
|
132
124
|
// earned. No attach, no hook, no record. The try-it-first on-ramp.
|
|
133
|
-
|
|
125
|
+
// `sawbuck check` IS `promptwheel run` with a plain-spoken verdict: every engine flag passes
|
|
126
|
+
// through (--base/--head/--repeat/--allow/--json/--no-detect-gaming), so there is one entry
|
|
127
|
+
// point for a person and the engine stays the one for CI and libraries.
|
|
128
|
+
const VALUE_FLAGS = new Set(['--base', '--head', '--repeat', '--allow']);
|
|
129
|
+
const pass = []; const pos = []; let json = false, base = null, head = null;
|
|
130
|
+
for (let i = 0; i < rest.length; i++) {
|
|
131
|
+
const a = rest[i];
|
|
132
|
+
if (a === '--json') { json = true; continue; }
|
|
133
|
+
if (VALUE_FLAGS.has(a)) { const v = rest[++i]; if (v == null) die(`${a} needs a value`); if (a === '--base') base = v; else if (a === '--head') head = v; else pass.push(a, v); continue; }
|
|
134
|
+
if (a.startsWith('--')) { pass.push(a); continue; }
|
|
135
|
+
pos.push(a);
|
|
136
|
+
}
|
|
137
|
+
const repo = resolve(pos[0] || '.');
|
|
134
138
|
// stderr ignored so a benign failing probe (e.g. rev-parse HEAD~1 on a one-commit repo) does not
|
|
135
139
|
// leak a scary "fatal:" line before sawbuck's own message.
|
|
136
140
|
const g = (...a) => { try { return execFileSync('git', ['-C', repo, ...a], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim(); } catch { return ''; } };
|
|
137
|
-
|
|
141
|
+
// 64 (EX_USAGE) for every "could not run" below, never a verdict code: a script or CI must not read it as gamed (2).
|
|
142
|
+
if (!g('rev-parse', '--git-dir')) { out(`🐴 sawbuck: ${repo} is not a git repo. The check re-proves a change against git history, so run it inside one (git init, then commit).`); process.exit(64); }
|
|
143
|
+
if (!g('rev-parse', '--verify', 'HEAD')) { out('🐴 sawbuck: no commit yet. Make one, then `sawbuck check` verifies your next change against it.'); process.exit(64); }
|
|
144
|
+
const dirty = !base && !head && g('status', '--porcelain').length > 0;
|
|
138
145
|
// nothing to verify yet: a clean tree with no previous commit to diff against (a fresh repo). Say so
|
|
139
146
|
// plainly instead of the generic "could not run" below, so the first run is not a dead end.
|
|
140
|
-
if (!dirty && !g('rev-parse', '--verify', 'HEAD~1')) { out('🐴 sawbuck: nothing to verify yet, the working tree is clean and this is the only commit. Make a change (or add one more commit), then `sawbuck check` re-proves the win from your source edits alone.'); process.exit(
|
|
141
|
-
const where = dirty ? 'working' : 'last';
|
|
142
|
-
const
|
|
143
|
-
|
|
147
|
+
if (!dirty && !base && !g('rev-parse', '--verify', 'HEAD~1')) { out('🐴 sawbuck: nothing to verify yet, the working tree is clean and this is the only commit. Make a change (or add one more commit), then `sawbuck check` re-proves the win from your source edits alone.'); process.exit(64); }
|
|
148
|
+
const where = dirty ? 'working' : (base || head) ? `${base || 'HEAD~1'}..${head || 'HEAD'}` : 'last';
|
|
149
|
+
const args = dirty ? ['run', '--working'] : ['run', '--base', base || 'HEAD~1', '--head', head || 'HEAD'];
|
|
150
|
+
const { rep, code } = gateReport(repo, [...args, ...pass]);
|
|
151
|
+
if (!rep || !rep.verdict) { out('🐴 sawbuck: could not run the check here. Make sure this is a git repo with a commit, and a test command (or run `sawbuck init`).'); process.exit(64); }
|
|
152
|
+
if (json) { out(JSON.stringify(rep)); process.exit(code); }
|
|
144
153
|
const m = (rep.metrics || [])[0];
|
|
154
|
+
for (const md of rep.mandates || []) out(`🐴 ⛔ mandate "${md.name}" violated${md.reason ? ': ' + md.reason : ''} (touched: ${(md.files || []).join(', ')})${md.removed ? ' [deleted in this same change; a mandate stays in force for the change that removes it]' : ''}`);
|
|
145
155
|
if (rep.verdict === 'pass') out(`🐴 ✓ verified: your ${where} change earned it. ${m ? '"' + m.name + '"' : 'The metric'} held, and the source carried the win (it survives a source-only re-run).`);
|
|
146
156
|
else if (rep.verdict === 'gamed') out('🐴 🚩 gamed: a win here came from editing the test, config, or grader, not the source. It does NOT survive a source-only re-run, so it does not count.');
|
|
157
|
+
else if (rep.verdict === 'fail' && (rep.mandates || []).length) out('🐴 ✗ sent back: a hard rule in the contract was broken, however green the rest is.');
|
|
147
158
|
else if (rep.verdict === 'fail') out(`🐴 ✗ regression: a guarded metric got worse (${(rep.metrics || []).filter((x) => x.guard && !x.ok).map((x) => x.name).join(', ') || 'a guard'}).`);
|
|
148
159
|
else out('🐴 ≈ inconclusive: the check ran but could not measure your metric (it did not run in isolation). Point it at your real test command: `sawbuck init`, or set setup/linkDirs in promptwheel.config.json.');
|
|
160
|
+
if (rep.gate && rep.gate.changed) out(`🐴 ⚠ the gate definition changed in this change (fingerprint ${rep.gate.fingerprint.base} -> ${rep.gate.fingerprint.head}): a metric, guard, evaluatorPath or mandate was added, removed or edited.`);
|
|
149
161
|
process.exit(code);
|
|
150
162
|
}
|
|
151
163
|
case 'log': {
|
|
@@ -178,23 +190,23 @@ switch (cmd) {
|
|
|
178
190
|
catch (e) { process.exit(e.status || 1); }
|
|
179
191
|
break;
|
|
180
192
|
}
|
|
181
|
-
case 'context': {
|
|
182
|
-
|
|
193
|
+
case 'context': { const o = contextBlock(repoArg(), flags.has('--speak')); if (o) out(o); break; }
|
|
194
|
+
case 'frame': {
|
|
195
|
+
// the prompt face of the contract: re-impress the hard rules (mandates, resolved through `extends`)
|
|
196
|
+
// as one compact block at the live end of the context, so a deep or freshly compacted session
|
|
197
|
+
// cannot forget them. Silent without a contract; --speak explains instead.
|
|
198
|
+
const o = frameBlock(repoArg());
|
|
199
|
+
if (o) out(o);
|
|
200
|
+
else if (flags.has('--speak')) out('🐴 sawbuck frame: no mandates in promptwheel.config.json yet. Add one (a protected path) and this re-asserts it every turn and after every compaction, so a deep context cannot forget it.');
|
|
201
|
+
break;
|
|
202
|
+
}
|
|
203
|
+
case 'turn': {
|
|
204
|
+
// the ONE per-prompt hook: the contract frame (forward: what must hold) and the relevant history
|
|
205
|
+
// (backward: what broke here before) in a single block, one process per prompt. Silent when
|
|
206
|
+
// there is nothing to say. Wired by `attach --claude` on UserPromptSubmit and after a compaction.
|
|
183
207
|
const repo = repoArg();
|
|
184
|
-
const
|
|
185
|
-
|
|
186
|
-
// hook's stdout is injected into context; "nothing notable" every turn would be noise).
|
|
187
|
-
// --speak forces a reply, for an explicit `sawbuck context` / the /sawbuck-context command.
|
|
188
|
-
const speak = flags.has('--speak');
|
|
189
|
-
let changed = g('diff', '--name-only', 'HEAD').split('\n').filter(Boolean);
|
|
190
|
-
if (!changed.length) changed = g('diff', '--name-only', 'HEAD~1', 'HEAD').split('\n').filter(Boolean);
|
|
191
|
-
if (!changed.length) { if (speak) out('🐴 sawbuck: nothing changed to look up (no working edits, and no previous commit to read).'); break; }
|
|
192
|
-
const fileArgs = changed.flatMap((f) => ['--file', f]);
|
|
193
|
-
let o = '';
|
|
194
|
-
try { o = execFileSync(process.execPath, [join(HERE, 'rank.mjs'), '--repo', repo, ...fileArgs, '--top', '3'], { encoding: 'utf8' }); }
|
|
195
|
-
catch (e) { o = (e.stdout || '').toString(); }
|
|
196
|
-
if (/relevant history/.test(o)) out(o);
|
|
197
|
-
else if (speak) out('🐴 sawbuck: no notable history for the files you are touching (a clean trail).');
|
|
208
|
+
const o = [frameBlock(repo), contextBlock(repo, false)].filter(Boolean).join('\n');
|
|
209
|
+
if (o) out(o);
|
|
198
210
|
break;
|
|
199
211
|
}
|
|
200
212
|
case 'recall': {
|
|
@@ -223,11 +235,6 @@ switch (cmd) {
|
|
|
223
235
|
out(r.pass ? ' ✓ conforms to the Memory Contract.' : ' ✗ does not conform (see above).');
|
|
224
236
|
process.exit(r.pass ? 0 : 1);
|
|
225
237
|
}
|
|
226
|
-
case 'run': {
|
|
227
|
-
try { execFileSync(process.execPath, [join(HERE, 'run.mjs'), ...rest], { stdio: 'inherit' }); }
|
|
228
|
-
catch (e) { process.exit(e.status || 1); }
|
|
229
|
-
break;
|
|
230
|
-
}
|
|
231
238
|
case '__hook': {
|
|
232
239
|
try { execFileSync(process.execPath, [HOOK, ...rest], { stdio: 'inherit' }); } catch (e) { process.exit(e.status || 0); }
|
|
233
240
|
break;
|
|
@@ -246,7 +253,14 @@ switch (cmd) {
|
|
|
246
253
|
const repo = repoArg();
|
|
247
254
|
if (flags.has('--user')) unwireClaude(repo, { user: true });
|
|
248
255
|
else if (flags.has('--claude')) unwireClaude(repo, {});
|
|
249
|
-
else
|
|
256
|
+
else {
|
|
257
|
+
// no flag: reverse whatever this repo's receipt says is attached (git and/or project Claude
|
|
258
|
+
// hooks), so a bare `detach` is always the inverse of the `attach` you ran.
|
|
259
|
+
const a = readManifest(manifestPath(repo)).attachments || {};
|
|
260
|
+
const hasClaude = !!(a.claude || a['claude-shared']);
|
|
261
|
+
if (hasClaude) unwireClaude(repo, {});
|
|
262
|
+
if (a.git || !hasClaude) unwireGit(repo);
|
|
263
|
+
}
|
|
250
264
|
if (purge) purgeRecord(repo);
|
|
251
265
|
if (flags.has('--verify')) verifyClean(repo);
|
|
252
266
|
break;
|
|
@@ -270,43 +284,47 @@ switch (cmd) {
|
|
|
270
284
|
const mem = existsSync(join(repo, 'LOGBOOK.md')) || existsSync(join(repo, 'events.jsonl'));
|
|
271
285
|
const seg = caught > 0 ? `🐴 sawbuck · ${caught} caught` : active ? `🐴 sawbuck · watching` : armed ? `🐴 sawbuck · ready` : mem ? `🐴 sawbuck · memory` : '';
|
|
272
286
|
const base = seg ? (last ? `${seg} · ${last}` : seg) : '';
|
|
273
|
-
// the live steering readout, when the wrapper forwards the session transcript path
|
|
274
|
-
let yk = '';
|
|
275
|
-
const ti = rest.findIndex((a) => a === '--transcript' || a === '--session');
|
|
276
|
-
if (ti > -1 && rest[ti + 1]) { try { const { yanks, turns } = tally(readFileSync(rest[ti + 1], 'utf8')); if (turns) yk = `🪢 ${yanks}/${turns} yanks`; } catch {} }
|
|
277
287
|
// live steering % from the reins engine's file (OFFLINE read; reins does the LLM work, never sawbuck)
|
|
278
288
|
let st = '';
|
|
279
|
-
try { const sj = JSON.parse(readFileSync(join(repo, '.sawbuck', 'steer.json'), 'utf8')); if (sj && sj.scored) { const p = Math.round((sj.steeringRate || 0) * 100); st = `${p >= 50 ? '
|
|
280
|
-
|
|
289
|
+
try { const sj = JSON.parse(readFileSync(join(repo, '.sawbuck', 'steer.json'), 'utf8')); if (sj && sj.scored) { const p = Math.round(((sj.ema != null ? sj.ema : sj.steeringRate) || 0) * 100); st = `${p >= 50 ? '✊' : '🐎'} steer ${p}%`; } } catch {}
|
|
290
|
+
// configurable template: SAWBUCK_STATUSLINE picks which segments show and in what order
|
|
291
|
+
// (tokens {status} {sawbuck} {verdict} {steer} plus any literal text); empty tokens collapse
|
|
292
|
+
// so you never get a dangling separator. Unset = the default composition. Mix in other
|
|
293
|
+
// tools (git branch, model, your own adapters) by composing in your statusline script.
|
|
294
|
+
const tpl = process.env.SAWBUCK_STATUSLINE;
|
|
295
|
+
let outp;
|
|
296
|
+
if (tpl && tpl.trim()) {
|
|
297
|
+
const tok = { status: base, sawbuck: seg, verdict: last, steer: st };
|
|
298
|
+
outp = tpl.replace(/\{(status|sawbuck|verdict|steer)\}/g, (_, k) => tok[k] || '')
|
|
299
|
+
.replace(/\s*·\s*·\s*/g, ' · ').replace(/^\s*·\s*/, '').replace(/\s*·\s*$/, '').replace(/\s{2,}/g, ' ').trim();
|
|
300
|
+
} else {
|
|
301
|
+
outp = [base, st].filter(Boolean).join(' · ');
|
|
302
|
+
}
|
|
281
303
|
if (outp) process.stdout.write(outp);
|
|
282
304
|
}
|
|
283
305
|
} catch {}
|
|
284
306
|
break;
|
|
285
307
|
}
|
|
286
|
-
case 'yank': {
|
|
287
|
-
// the live steering meter (the reins hook): how much did you have to steer this session?
|
|
288
|
-
// A MIRROR read from the Claude Code transcript; the check guards the turns you did NOT steer.
|
|
289
|
-
const si = rest.findIndex((a) => a === '--session' || a === '--transcript');
|
|
290
|
-
const sess = resolveSessionPath(si > -1 ? rest[si + 1] : null, process.cwd());
|
|
291
|
-
if (!sess) { out('🐴 sawbuck: no session transcript found to read (pass --session <path.jsonl>).'); process.exit(2); }
|
|
292
|
-
let text = ''; try { text = readFileSync(sess, 'utf8'); } catch { out('🐴 sawbuck: could not read the session transcript.'); process.exit(2); }
|
|
293
|
-
const { turns, yanks, rate, recent } = tally(text);
|
|
294
|
-
if (flags.has('--json')) { out(JSON.stringify({ turns, yanks, rate, recent })); break; }
|
|
295
|
-
const pct = Math.round(rate * 100);
|
|
296
|
-
out('🐴 reins: steering this session');
|
|
297
|
-
if (yanks === 0) {
|
|
298
|
-
out(` 0 hard yanks across ${turns} turns — you did not have to pull the reins hard.`);
|
|
299
|
-
} else {
|
|
300
|
-
out(` ${yanks} hard yank${yanks === 1 ? '' : 's'} across ${turns} turns (${pct}%) — times you pulled me back hard.`);
|
|
301
|
-
if (turns - yanks > 0) out(` the check guards the other ${turns - yanks} turns you trusted me with — a miss there is the risk.`);
|
|
302
|
-
if (recent.length) out(` recent: ${recent.map((r) => '"' + r + '"').join(' · ')}`);
|
|
303
|
-
}
|
|
304
|
-
out(' (terse course-corrections only; the full steer rate, incl. discursive redirects, is the harvest layer.)');
|
|
305
|
-
break;
|
|
306
|
-
}
|
|
307
308
|
default: die(`unknown command '${cmd}'\n\n${help}`);
|
|
308
309
|
}
|
|
309
310
|
|
|
311
|
+
// --- the per-turn block: frame (forward, from contract.mjs) + context (backward) ---------------
|
|
312
|
+
// the read face: ranked relevant history for what you're working on right now. Silent when there is
|
|
313
|
+
// nothing to say, so it is safe on every UserPromptSubmit (that hook's stdout is injected into context;
|
|
314
|
+
// "nothing notable" every turn would be noise). `speak` forces a reply for the explicit command.
|
|
315
|
+
function contextBlock(repo, speak) {
|
|
316
|
+
const g = (...a) => { try { return execFileSync('git', ['-C', repo, ...a], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim(); } catch { return ''; } };
|
|
317
|
+
let changed = g('diff', '--name-only', 'HEAD').split('\n').filter(Boolean);
|
|
318
|
+
if (!changed.length) changed = g('diff', '--name-only', 'HEAD~1', 'HEAD').split('\n').filter(Boolean);
|
|
319
|
+
if (!changed.length) return speak ? '🐴 sawbuck: nothing changed to look up (no working edits, and no previous commit to read).' : '';
|
|
320
|
+
const fileArgs = changed.flatMap((f) => ['--file', f]);
|
|
321
|
+
let o = '';
|
|
322
|
+
try { o = execFileSync(process.execPath, [join(HERE, 'rank.mjs'), '--repo', repo, ...fileArgs, '--top', '3'], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }); }
|
|
323
|
+
catch (e) { o = (e.stdout || '').toString(); }
|
|
324
|
+
if (/relevant history/.test(o)) return o.trimEnd();
|
|
325
|
+
return speak ? '🐴 sawbuck: no notable history for the files you are touching (a clean trail).' : '';
|
|
326
|
+
}
|
|
327
|
+
|
|
310
328
|
// after attach, prove the check can actually measure something here (the inert-metric trap):
|
|
311
329
|
// run the gate on the last commit and say what it can or cannot measure, so you never attach
|
|
312
330
|
// into silent nothing. Best-effort and bounded; the wire already succeeded regardless.
|
|
@@ -371,7 +389,7 @@ function wireGit(repo) {
|
|
|
371
389
|
const pc = join(repo, '.git', 'hooks', 'post-commit');
|
|
372
390
|
const existed = existsSync(pc);
|
|
373
391
|
let body = existed ? readFileSync(pc, 'utf8') : '#!/bin/sh\n';
|
|
374
|
-
if (body.includes(S_START)) { out(`sawbuck: already attached (git) in ${repo}`);
|
|
392
|
+
if (body.includes(S_START)) { out(`sawbuck: already attached (git) in ${repo}`); return; }
|
|
375
393
|
if (!body.startsWith('#!')) body = '#!/bin/sh\n' + body;
|
|
376
394
|
if (!body.endsWith('\n')) body += '\n';
|
|
377
395
|
if (dry) {
|
|
@@ -390,7 +408,6 @@ function wireGit(repo) {
|
|
|
390
408
|
writeManifest(manifestPath(repo), m);
|
|
391
409
|
regAdd(repo, 'git');
|
|
392
410
|
out(`sawbuck: attached (git post-commit) in ${repo}`);
|
|
393
|
-
warnNoConfig(repo);
|
|
394
411
|
}
|
|
395
412
|
function unwireGit(repo) {
|
|
396
413
|
const pc = join(repo, '.git', 'hooks', 'post-commit');
|
|
@@ -426,23 +443,26 @@ function wireClaude(repo, opts) {
|
|
|
426
443
|
const fileExisted = existsSync(f);
|
|
427
444
|
let cfg = {};
|
|
428
445
|
if (fileExisted) { try { cfg = JSON.parse(readFileSync(f, 'utf8')); } catch { die(`${f} exists but is not valid JSON; I will not overwrite it. Add the hooks by hand.`); } }
|
|
429
|
-
if (JSON.stringify(cfg).includes('sawbuck.mjs')) { out(`sawbuck: already attached (${user ? 'user' : 'claude'}) in ${f}`);
|
|
430
|
-
const ctx = user ? `node ${JSON.stringify(SELF)}
|
|
446
|
+
if (JSON.stringify(cfg).includes('sawbuck.mjs')) { out(`sawbuck: already attached (${user ? 'user' : 'claude'}) in ${f}`); return; }
|
|
447
|
+
const ctx = user ? `node ${JSON.stringify(SELF)} turn` : `node ${JSON.stringify(SELF)} turn ${JSON.stringify(repo)}`;
|
|
431
448
|
const chk = user ? `node ${JSON.stringify(SELF)} __hook --by claude --stop` : `node ${JSON.stringify(SELF)} __hook --repo ${JSON.stringify(repo)} --by claude --stop`;
|
|
432
449
|
if (dry) {
|
|
433
450
|
out(dryText([
|
|
434
|
-
`${fileExisted ? 'modify' : 'create'} ${f} (UserPromptSubmit + Stop hooks)`,
|
|
451
|
+
`${fileExisted ? 'modify' : 'create'} ${f} (UserPromptSubmit + SessionStart[compact] + Stop hooks: the per-turn block, and the check)`,
|
|
435
452
|
user ? null : `exclude ${excludeFile(repo)} (${RECORD_DIRS.join(', ')}${shared ? '' : ', .claude/settings.local.json'})`,
|
|
436
453
|
`create ${user ? userManifestPath() : manifestPath(repo)} (attach receipt)`,
|
|
437
454
|
].filter(Boolean)));
|
|
438
455
|
return;
|
|
439
456
|
}
|
|
440
457
|
cfg.hooks = cfg.hooks || {};
|
|
441
|
-
const put = (event, command) => {
|
|
458
|
+
const put = (event, command, matcher) => {
|
|
442
459
|
cfg.hooks[event] = (cfg.hooks[event] || []).filter((h) => !JSON.stringify(h).includes('sawbuck.mjs'));
|
|
443
|
-
cfg.hooks[event].push({ hooks: [{ type: 'command', command }] });
|
|
460
|
+
cfg.hooks[event].push({ ...(matcher ? { matcher } : {}), hooks: [{ type: 'command', command }] });
|
|
444
461
|
};
|
|
445
462
|
put('UserPromptSubmit', ctx);
|
|
463
|
+
// right after a compaction the summary has just dropped detail: re-impress the contract frame and the
|
|
464
|
+
// relevant history there too (SessionStart with the `compact` source injects stdout into context).
|
|
465
|
+
put('SessionStart', ctx, 'compact');
|
|
446
466
|
put('Stop', chk);
|
|
447
467
|
mkdirSync(dir, { recursive: true });
|
|
448
468
|
writeFileSync(f, JSON.stringify(cfg, null, 2) + '\n');
|
|
@@ -456,7 +476,6 @@ function wireClaude(repo, opts) {
|
|
|
456
476
|
out(`sawbuck: wired ${user ? 'USER-LEVEL' : shared ? 'shared project' : 'project'} Claude Code hooks in ${f}`);
|
|
457
477
|
if (user) out(' profile-switches by cwd: memory in any git repo, the check where a promptwheel.config.json exists.');
|
|
458
478
|
else if (!shared) out(' (settings.local.json is gitignored; use --shared to commit the hooks for your team)');
|
|
459
|
-
if (!user) warnNoConfig(repo);
|
|
460
479
|
}
|
|
461
480
|
function unwireClaude(repo, { user }) {
|
|
462
481
|
const mp = user ? userManifestPath() : manifestPath(repo);
|
|
@@ -487,7 +506,7 @@ function unwireClaude(repo, { user }) {
|
|
|
487
506
|
}
|
|
488
507
|
function removeClaudeEntries(cfg) {
|
|
489
508
|
if (!cfg.hooks) return;
|
|
490
|
-
for (const ev of ['UserPromptSubmit', 'Stop']) {
|
|
509
|
+
for (const ev of ['UserPromptSubmit', 'SessionStart', 'Stop']) {
|
|
491
510
|
if (!Array.isArray(cfg.hooks[ev])) continue;
|
|
492
511
|
cfg.hooks[ev] = cfg.hooks[ev].filter((h) => !JSON.stringify(h).includes('sawbuck.mjs'));
|
|
493
512
|
if (!cfg.hooks[ev].length) delete cfg.hooks[ev]; // no empty [] residue
|
|
@@ -595,7 +614,3 @@ function detachAll(purge) {
|
|
|
595
614
|
|
|
596
615
|
// --- shared helpers ---------------------------------------------------------
|
|
597
616
|
function dryText(plan) { return ['sawbuck --dry-run: would leave this footprint (writing nothing):', ...plan.map((p) => ' ' + p), ' all of it reversible with `detach`.'].join('\n'); }
|
|
598
|
-
function warnNoConfig() {
|
|
599
|
-
// superseded by verifyAttach, which runs after every attach and reports exactly what the check
|
|
600
|
-
// can measure (zero-config now auto-detects your test command, so a missing config is not an error).
|
|
601
|
-
}
|
package/GLOSSARY.md
DELETED
|
@@ -1,73 +0,0 @@
|
|
|
1
|
-
# Sawbuck: Glossary & Controlled Vocabulary
|
|
2
|
-
|
|
3
|
-
Sawbuck is a saddle: an isolated layer you attach to a repo that fits its shape
|
|
4
|
-
and responds when the repo moves. It reads the history (via its memory provider,
|
|
5
|
-
logbook by default) and a small config of what to measure, runs the check when a
|
|
6
|
-
change lands, and surfaces the relevant past finding before the agent repeats a
|
|
7
|
-
known mistake.
|
|
8
|
-
|
|
9
|
-
## The rule: one plain surface
|
|
10
|
-
|
|
11
|
-
Earlier drafts split terms into a "brand/human" surface (a mining metaphor) and a
|
|
12
|
-
plain "agent-facing" surface. That split is retired. There is now ONE surface, and
|
|
13
|
-
it is plain and scientific. The saddle is the brand and the architecture spine
|
|
14
|
-
(the name, `attach`, `fit`); it is NOT a vocabulary for every concept.
|
|
15
|
-
|
|
16
|
-
Why plain wins everywhere: an LLM has strong trained priors on words like
|
|
17
|
-
`regression`, `verified`, `gamed`. Metaphor jargon in the model's context forces
|
|
18
|
-
a gloss every session (token cost) and risks a misread (trajectory drift). Plain,
|
|
19
|
-
high-prior words serve the human reader and the agent equally, so we do not keep
|
|
20
|
-
two vocabularies.
|
|
21
|
-
|
|
22
|
-
## Canonical terms
|
|
23
|
-
|
|
24
|
-
| concept | term | notes |
|
|
25
|
-
|---|---|---|
|
|
26
|
-
| the product | **Sawbuck** | the only brand word |
|
|
27
|
-
| the memory (the role) | **memory**; default provider **Logbook** | the relevant history + live debt for what you touch. logbook (a lab logbook: plain, scientific) is a standalone product and the default fill; the role is a socket, swap your own via the Memory Contract |
|
|
28
|
-
| the verifier | **the check** (verb: *verify*) | runs the metrics on a change |
|
|
29
|
-
| a change that improved and holds | **verified** | |
|
|
30
|
-
| a change that made a guarded metric worse | **regression** | do-not-retry |
|
|
31
|
-
| a change that passed by faking the metric | **gamed** | the win came from editing the test, config, or grader, not the source (the engine verdict, and the word every Sawbuck surface uses) |
|
|
32
|
-
| the accumulated record of outcomes | **results** | `results.json` |
|
|
33
|
-
| what Sawbuck leaves you | **verified history** | the findings you can trust |
|
|
34
|
-
| a surfaced past outcome | **finding** | |
|
|
35
|
-
| mounting Sawbuck to a repo | **attach** / **cinch** | `sawbuck attach <repo>` |
|
|
36
|
-
| reading the repo's shape | **fit** | history + config; a concept, not a command (it is what `attach` reads to decide what it can measure) |
|
|
37
|
-
| code still silenced in HEAD now | **debt** (suppression debt) | disabled/weakened tests + lint/type suppressions still live; `sawbuck debt` ranks it by severity and file fragility |
|
|
38
|
-
| the one-view accumulation | **inventory** | `sawbuck status`: what it has caught over time + the debt accruing now + whether it is armed here |
|
|
39
|
-
|
|
40
|
-
## The two interfaces (a socket and a standard)
|
|
41
|
-
|
|
42
|
-
Sawbuck composes two roles, and they are open in opposite directions. Keeping them
|
|
43
|
-
straight is what stops "Sawbuck, then logbook?" confusion: logbook is not a second
|
|
44
|
-
brand bolted on, it is the default fill for one of the two roles.
|
|
45
|
-
|
|
46
|
-
- **the check** is a *standard we publish* (the Verified-Outcome Contract, see
|
|
47
|
-
CONTRACT.md): other tools conform to it, or embed it. promptwheel is the reference
|
|
48
|
-
implementation. It is not swappable, because the un-gameable source re-proof is the
|
|
49
|
-
asset; what you plug in is *what to measure*.
|
|
50
|
-
- **the memory** is a *socket we expose*: you fill it with any provider. logbook is
|
|
51
|
-
the default, carried along so it works out of the box, and it is its own product you
|
|
52
|
-
can also run on its own.
|
|
53
|
-
|
|
54
|
-
In one line: the check is a standard for others to meet; the memory is a socket for
|
|
55
|
-
others to fill.
|
|
56
|
-
|
|
57
|
-
## The one humility rule
|
|
58
|
-
|
|
59
|
-
Findings are **leads, not conclusions**, verify against the diff before acting.
|
|
60
|
-
|
|
61
|
-
## Dispatch template
|
|
62
|
-
|
|
63
|
-
```
|
|
64
|
-
Sawbuck · {N} regressions charted · {N} gamed changes caught
|
|
65
|
-
🆕 just learned: {reason} ({file})
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
## Note on engine verdicts
|
|
69
|
-
|
|
70
|
-
The check engine emits verdicts (`gamed`, `fail`, `pass`); the surface keeps **gamed**
|
|
71
|
-
as-is (it is vivid and used on every surface) and maps `fail` → **regression** and
|
|
72
|
-
`pass` → **verified**. ("Invalid" was an earlier softening of `gamed` that no surface
|
|
73
|
-
adopted; `gamed` is canonical.)
|
package/run.mjs
DELETED
|
@@ -1,47 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
// run.mjs, the thin reference harness. Each loop: load context, run ONE attempt
|
|
3
|
-
// through the keep-if-verified gate, repeat. It loops primitives that already
|
|
4
|
-
// exist (sawbuck context + promptwheel improve + YOUR agent); it is NOT an
|
|
5
|
-
// orchestrator (no planning, routing, or queue, that is the platform's job).
|
|
6
|
-
// Usage: node run.mjs --attempt '<agent cmd>' [--loops N] [--repo P]
|
|
7
|
-
// the attempt cmd can read $SAWBUCK_CONTEXT (a file of the ranked relevant history).
|
|
8
|
-
|
|
9
|
-
import { execFileSync } from 'node:child_process';
|
|
10
|
-
import { createRequire } from 'node:module';
|
|
11
|
-
import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
|
|
12
|
-
import { tmpdir } from 'node:os';
|
|
13
|
-
import { join, dirname } from 'node:path';
|
|
14
|
-
import { fileURLToPath } from 'node:url';
|
|
15
|
-
|
|
16
|
-
const HERE = dirname(fileURLToPath(import.meta.url));
|
|
17
|
-
const require = createRequire(import.meta.url);
|
|
18
|
-
const bin = (envVar, pkg, dev) => { if (process.env[envVar]) return process.env[envVar]; const bare = pkg.replace(/\/bin\/[^/]*$/, ''); for (const s of [pkg, bare]) { try { return require.resolve(s); } catch {} } return dev; };
|
|
19
|
-
const PW = bin('PW_BIN', 'promptwheel/bin/promptwheel.mjs', join(HERE, '..', 'promptwheel', 'bin', 'promptwheel.mjs'));
|
|
20
|
-
|
|
21
|
-
const arg = (n, d) => { const i = process.argv.indexOf(n); return i > -1 ? process.argv[i + 1] : d; };
|
|
22
|
-
const repo = arg('--repo', process.cwd());
|
|
23
|
-
const attempt = arg('--attempt', null);
|
|
24
|
-
const loops = Math.max(1, parseInt(arg('--loops', '1'), 10) || 1);
|
|
25
|
-
if (!attempt) { process.stderr.write("sawbuck run: --attempt '<agent cmd>' is required\n"); process.exit(2); }
|
|
26
|
-
|
|
27
|
-
const context = () => { try { return execFileSync(process.execPath, [join(HERE, 'sawbuck.mjs'), 'context', repo], { cwd: repo, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }); } catch (e) { return (e.stdout || '').toString(); } };
|
|
28
|
-
const parseResult = (s) => { try { return JSON.parse(s.slice(s.indexOf('{'), s.lastIndexOf('}') + 1)).result || 'error'; } catch { return 'error'; } };
|
|
29
|
-
|
|
30
|
-
const scratch = mkdtempSync(join(tmpdir(), 'sawbuck-run-'));
|
|
31
|
-
let kept = 0;
|
|
32
|
-
try {
|
|
33
|
-
for (let i = 1; i <= loops; i++) {
|
|
34
|
-
const ctxFile = join(scratch, `context-${i}.md`);
|
|
35
|
-
writeFileSync(ctxFile, context() || 'no prior findings yet.\n'); // load the trajectory
|
|
36
|
-
let result;
|
|
37
|
-
try {
|
|
38
|
-
const out = execFileSync(process.execPath, [PW, 'improve', '--attempt', attempt, '--json'],
|
|
39
|
-
{ cwd: repo, encoding: 'utf8', env: { ...process.env, SAWBUCK_CONTEXT: ctxFile, PW_PLAYBOOK: ctxFile }, timeout: 600000, stdio: ['ignore', 'pipe', 'ignore'] });
|
|
40
|
-
result = parseResult(out);
|
|
41
|
-
} catch (e) { result = parseResult((e.stdout || '').toString()); }
|
|
42
|
-
if (result === 'kept') kept++;
|
|
43
|
-
process.stdout.write(`sawbuck run: loop ${i}/${loops} -> ${result}\n`);
|
|
44
|
-
if (result === 'plateau' || result === 'error') break; // nothing left to improve, or it broke
|
|
45
|
-
}
|
|
46
|
-
} finally { rmSync(scratch, { recursive: true, force: true }); }
|
|
47
|
-
process.stdout.write(`sawbuck run: ${kept} verified improvement${kept === 1 ? '' : 's'} kept\n`);
|
package/yank.mjs
DELETED
|
@@ -1,87 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
// yank.mjs — the live steering meter (the "reins" hook). Reads the current Claude
|
|
3
|
-
// Code session transcript and counts YANKS: turns where you corrected or redirected
|
|
4
|
-
// the agent. Deterministic, zero-dep, with a transparent definition shown in the
|
|
5
|
-
// output. It is a MIRROR — a number about how much you had to steer this session —
|
|
6
|
-
// and it pairs with the check: the check guards the turns you did NOT steer, where
|
|
7
|
-
// a miss is the real risk. The yank rate falling over time is the reins getting
|
|
8
|
-
// looser (the agent learning your cues).
|
|
9
|
-
//
|
|
10
|
-
// A yank is counted CONSERVATIVELY (precision over recall): a human turn, after the
|
|
11
|
-
// agent has acted, that OPENS by correcting or redirecting ("no", "don't",
|
|
12
|
-
// "actually", "stop", "undo", "instead", ...). Discursive steers buried mid-
|
|
13
|
-
// paragraph are deliberately NOT counted here — that is the LLM-harvest layer's job;
|
|
14
|
-
// this stays cheap, instant, and honest (an undercount with a clear definition beats
|
|
15
|
-
// an inflated guess).
|
|
16
|
-
|
|
17
|
-
import { readFileSync, readdirSync, statSync, existsSync } from 'node:fs';
|
|
18
|
-
import { join } from 'node:path';
|
|
19
|
-
import { homedir } from 'node:os';
|
|
20
|
-
|
|
21
|
-
// leading correction/redirect markers at the START of the turn (case-insensitive)
|
|
22
|
-
const LEAD = /^\s*["'(]?\s*(no\b|no,|nope\b|don'?t\b|do not\b|actually\b|wait\b|stop\b|undo\b|revert\b|instead\b|nah\b|hold on|rather\b|not (that|like|quite|what|really)|that'?s (not|wrong)|hmm,? ?no|why (did|are) you|you (shouldn'?t|weren'?t))/i;
|
|
23
|
-
|
|
24
|
-
function humanText(o) {
|
|
25
|
-
if (!o || o.type !== 'user' || o.isMeta) return null;
|
|
26
|
-
if (o.userType && o.userType !== 'external') return null;
|
|
27
|
-
const c = o.message && o.message.content;
|
|
28
|
-
if (typeof c === 'string') return c;
|
|
29
|
-
if (Array.isArray(c)) {
|
|
30
|
-
if (c.some((b) => b && b.type === 'tool_result')) return null; // tool output, not you
|
|
31
|
-
const t = c.filter((b) => b && b.type === 'text').map((b) => b.text || '').join(' ');
|
|
32
|
-
return t || null;
|
|
33
|
-
}
|
|
34
|
-
return null;
|
|
35
|
-
}
|
|
36
|
-
const isNoise = (t) => {
|
|
37
|
-
const s = t.trim();
|
|
38
|
-
if (s.includes('<task-notification>') || s.startsWith('<task-id>')) return true; // injected
|
|
39
|
-
if (s.replace(/\[Image #\d+\]/g, '').trim().length < 3) return true; // image-only
|
|
40
|
-
return false;
|
|
41
|
-
};
|
|
42
|
-
|
|
43
|
-
// tally(transcriptText) -> { turns, yanks, rate, recent } over YOUR typed turns.
|
|
44
|
-
export function tally(text) {
|
|
45
|
-
let turns = 0, yanks = 0, seenHuman = false, prevAssistantActed = false;
|
|
46
|
-
const recent = [];
|
|
47
|
-
for (const line of String(text).split('\n')) {
|
|
48
|
-
if (!line.trim()) continue;
|
|
49
|
-
let o; try { o = JSON.parse(line); } catch { continue; }
|
|
50
|
-
if (o.type === 'assistant') { prevAssistantActed = true; continue; }
|
|
51
|
-
const t = humanText(o);
|
|
52
|
-
if (t === null || isNoise(t)) continue;
|
|
53
|
-
const opener = !seenHuman; seenHuman = true;
|
|
54
|
-
turns++;
|
|
55
|
-
if (!opener && prevAssistantActed && LEAD.test(t)) {
|
|
56
|
-
yanks++;
|
|
57
|
-
recent.push(t.replace(/\s+/g, ' ').trim().slice(0, 48));
|
|
58
|
-
}
|
|
59
|
-
prevAssistantActed = false;
|
|
60
|
-
}
|
|
61
|
-
return { turns, yanks, rate: turns ? yanks / turns : 0, recent: recent.slice(-5) };
|
|
62
|
-
}
|
|
63
|
-
|
|
64
|
-
// Resolve the newest session transcript for a cwd. Claude Code stores transcripts
|
|
65
|
-
// under ~/.claude/projects/<encoded-cwd>/<id>.jsonl (path separators -> dashes).
|
|
66
|
-
// Best-effort: prefer the dir matching this cwd, else the newest across all. Never throws.
|
|
67
|
-
export function resolveSessionPath(explicit, cwd = process.cwd()) {
|
|
68
|
-
if (explicit) return existsSync(explicit) ? explicit : null;
|
|
69
|
-
const root = join(homedir(), '.claude', 'projects');
|
|
70
|
-
if (!existsSync(root)) return null;
|
|
71
|
-
const key = cwd.replace(/[^A-Za-z0-9]/g, '-');
|
|
72
|
-
let dirs = [];
|
|
73
|
-
try { dirs = readdirSync(root).map((d) => join(root, d)).filter((p) => { try { return statSync(p).isDirectory(); } catch { return false; } }); }
|
|
74
|
-
catch { return null; }
|
|
75
|
-
const matched = dirs.filter((p) => p.endsWith('/' + key));
|
|
76
|
-
const pool = matched.length ? matched : dirs;
|
|
77
|
-
let best = null, bestT = 0;
|
|
78
|
-
for (const full of pool) {
|
|
79
|
-
let files = []; try { files = readdirSync(full); } catch { continue; }
|
|
80
|
-
for (const f of files) {
|
|
81
|
-
if (!f.endsWith('.jsonl')) continue;
|
|
82
|
-
const p = join(full, f);
|
|
83
|
-
try { const st = statSync(p); if (st.mtimeMs > bestT) { bestT = st.mtimeMs; best = p; } } catch {}
|
|
84
|
-
}
|
|
85
|
-
}
|
|
86
|
-
return best;
|
|
87
|
-
}
|