nearly-cli 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +274 -0
- package/STUDY.md +101 -0
- package/bin/nearly.mjs +108 -0
- package/package.json +39 -0
- package/scripts/attach.mjs +195 -0
- package/scripts/build-recap.mjs +655 -0
- package/scripts/hook.mjs +79 -0
- package/scripts/install-push-hook.mjs +87 -0
- package/scripts/post-recap.mjs +124 -0
- package/scripts/publish-pages.mjs +141 -0
- package/scripts/push-record.mjs +126 -0
- package/scripts/update-check.mjs +116 -0
- package/server/index.mjs +538 -0
- package/server/policy.mjs +61 -0
- package/ui/index.html +522 -0
- package/ui/recap.template.html +449 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Anuj Patel
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,274 @@
|
|
|
1
|
+
# Nearly
|
|
2
|
+
|
|
3
|
+
**A pull request tells you what changed. This tells you what nearly happened.**
|
|
4
|
+
|
|
5
|
+
When a coding agent writes a branch, the person reviewing it has no idea what the agent tried, what a human refused, or what got rolled back. The diff is the only thing that survives, and the diff is the one artifact that cannot show you any of it.
|
|
6
|
+
|
|
7
|
+
Nearly holds an agent's risky actions until a human decides, records every one of those decisions, and turns the branch into a short narrated page written for **the reviewer**. That page is a link, and it goes on the pull request.
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
you work normally → agent acts → risky action held → you decide
|
|
11
|
+
↓
|
|
12
|
+
reviewer opens the PR ← comment posted ← you push ← recorded
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
## Look before you install anything
|
|
16
|
+
|
|
17
|
+
Nothing to run. This is a real record from two real agent sessions on one branch:
|
|
18
|
+
|
|
19
|
+
**[A branch where three things never happened →](https://anujpatel06.github.io/nearly/records/priya-app--feat-third-task.html)**
|
|
20
|
+
|
|
21
|
+
Watch the first thirty seconds. The cover says what the diff cannot: an action the supervisor refused, a push policy blocked, and a file deletion refused. [Here is how it looks on the pull request.](https://github.com/anujpatel06/tempo-demo/pull/2)
|
|
22
|
+
|
|
23
|
+
## What it needs
|
|
24
|
+
|
|
25
|
+
Node 18 or newer, Claude Code signed in, and git. No dependencies and no API key: agents run on your existing Claude subscription.
|
|
26
|
+
|
|
27
|
+
Gating, recording and the record itself work on macOS, Linux and Windows. **Spoken narration is macOS only**, because it uses the built-in `say`. Elsewhere the record is built the same way and reads from its captions, or you can supply your own recordings with `--voice-dir`.
|
|
28
|
+
|
|
29
|
+
## Use it on your own repo
|
|
30
|
+
|
|
31
|
+
One command, in the repo you want recorded.
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
cd ~/code/my-app
|
|
35
|
+
nearly
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
✓ Nearly is on for my-app
|
|
40
|
+
|
|
41
|
+
· every Claude Code session here is gated and recorded
|
|
42
|
+
· the record is offered when you push
|
|
43
|
+
· records publish to https://you.github.io/nearly/records
|
|
44
|
+
|
|
45
|
+
Now just work. Requests that need you appear at http://127.0.0.1:47653
|
|
46
|
+
Nothing to leave running. Turn it off again with --off.
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
It installs the Claude Code hooks and the git pre-push hook, and works out where records publish by reading the Nearly's own remote. Nothing to configure. `nearly off` removes all of it.
|
|
50
|
+
|
|
51
|
+
**To get that command,** until this is on npm:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
git clone https://github.com/anujpatel06/nearly ~/nearly
|
|
55
|
+
npm link --prefix ~/nearly
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
No dependencies, so the link is instant. Once published it becomes `npx nearly-cli` with nothing to clone at all.
|
|
59
|
+
|
|
60
|
+
### Upgrading
|
|
61
|
+
|
|
62
|
+
It updates itself. When you run a command and a newer version exists, Nearly
|
|
63
|
+
installs it and tells you it did, then every repo you turned it on for is on the
|
|
64
|
+
new version with nothing to turn on again.
|
|
65
|
+
|
|
66
|
+
Four rules keep that from being something you regret installing:
|
|
67
|
+
|
|
68
|
+
- **Never in the hook path.** Nothing about updating may sit in front of an
|
|
69
|
+
action an agent is waiting on. Only commands you typed can trigger it, and
|
|
70
|
+
there is a test asserting the hook file does not even import the updater.
|
|
71
|
+
- **Never across a major version.** Nearly decides whether `rm -rf` runs. Same
|
|
72
|
+
major means same promises; a major bump is announced and left for you to read
|
|
73
|
+
before you trust it.
|
|
74
|
+
- **Never silent.** An update that happened without being mentioned is
|
|
75
|
+
indistinguishable from a compromise, so it always says what it did.
|
|
76
|
+
- **Never fatal.** No network, a locked global directory, a slow registry: you
|
|
77
|
+
keep the version you have, the command you ran still works, and it tells you
|
|
78
|
+
rather than leaving you to assume you are current.
|
|
79
|
+
|
|
80
|
+
It checks once a day, not on every command. `NEARLY_NO_UPDATE=1` turns it off
|
|
81
|
+
for good. Running through `npx` or from a checkout, it tells you instead of
|
|
82
|
+
touching anything, because an npx run is ephemeral and a checkout is yours.
|
|
83
|
+
|
|
84
|
+
**There is no server to start.** The hooks start it the first time they need it, in about a second, and it stays up. If it cannot start, Claude Code falls back to its own permission prompts and your session continues. Nothing to remember and nothing to break.
|
|
85
|
+
|
|
86
|
+
## Then work normally
|
|
87
|
+
|
|
88
|
+
Nothing about how you work changes. Open the repo in VS Code or a terminal, start Claude Code, give it a task.
|
|
89
|
+
|
|
90
|
+
1. **Reads run silently.** Anything that only looks at your code is allowed and logged.
|
|
91
|
+
2. **Anything that changes or reaches out is held.** It appears at http://127.0.0.1:47653 with the command, what it can affect, and a countdown. Answer with `A` or `D`, or shift for always and never. Nobody answering means denied after two minutes.
|
|
92
|
+
3. **The record builds itself** when the session ends.
|
|
93
|
+
4. **At `git push`** the hook merges every session on that branch, prints what was refused, and asks whether to post it. Say no and the push just continues.
|
|
94
|
+
5. **Your reviewer opens the pull request** and the record is there, as one comment that updates on every push rather than a new one each time.
|
|
95
|
+
|
|
96
|
+
### For a team
|
|
97
|
+
|
|
98
|
+
`.claude/settings.local.json` is per-person and stays out of git, which is right while you are trying it. To turn it on for everyone, move the same hooks into `.claude/settings.json` and commit that file. Once this is on npm the hooks invoke `npx nearly-cli`, so a teammate who clones the repo needs nothing installed beyond Node.
|
|
99
|
+
|
|
100
|
+
## Why the gate is not the point
|
|
101
|
+
|
|
102
|
+
Gating agent tool calls is a solved problem and several teams do it better:
|
|
103
|
+
|
|
104
|
+
- **[Prempti](https://prempti.falco.org/)** (Falco, CNCF, Apache 2.0) runs a real rule engine over every tool call, with policies in ordinary Falco YAML and a monitor mode for tuning them. Its homepage promises "a story of the session, not just the diff" — then ships that story as a local log file with no viewer.
|
|
105
|
+
- **[Agent Approve](https://www.agentapprove.com/)** puts each approval on your Apple Watch across twelve different agents, and parses chained shell commands so a dangerous one cannot hide behind a safe one. When the session ends it leaves no artifact at all.
|
|
106
|
+
- **[Endor Labs](https://www.endorlabs.com/)** streams every action into a searchable audit trail for enterprise security teams, and can rewrite a command before it runs. What it posts on your pull request is security findings about the code, never the story of the session.
|
|
107
|
+
- Claude Code, Cursor, Antigravity, Warp, Devin and Goose all ship their own allow/deny/ask lists, and Anthropic's **auto mode** now lets a classifier decide, on the evidence that users approve 93% of the prompts they see.
|
|
108
|
+
|
|
109
|
+
All of them capture the moment a human refuses something. None of them passes it on. Prempti writes it to a log for a security engineer, Agent Approve shows it to one person on a watch, Endor files it for an auditor.
|
|
110
|
+
|
|
111
|
+
**The gate here exists to produce the recording.** It is the instrument, not the product. If you already run Prempti, its audit trail is richer than ours and reading it as an input is the obvious next step; see *Roadmap*.
|
|
112
|
+
|
|
113
|
+
## The consent gradient
|
|
114
|
+
|
|
115
|
+
Every tool call passes through an HTTP `PreToolUse` hook to this server, which sorts it into a tier:
|
|
116
|
+
|
|
117
|
+
| Tier | What happens | Default for |
|
|
118
|
+
|---|---|---|
|
|
119
|
+
| never | denied, no prompt, logged | `rm -rf`, `git push`, `sudo`, `.env`, `curl … | sh` |
|
|
120
|
+
| ask | held until a human decides in the UI; denied if nobody answers in 2 minutes (fails closed) | Bash, Edit, Write, WebFetch, Task |
|
|
121
|
+
| log | allowed, receipt recorded | Read, Glob, Grep |
|
|
122
|
+
|
|
123
|
+
"Allow always" and "Never" turn a decision into a rule for the rest of the run, keyed by tool and first word of the command, or file extension for edits. In lab mode every turn is committed in the agent's worktree by the `Stop` hook, so **Undo turn** is a `git reset --hard HEAD~1`.
|
|
124
|
+
|
|
125
|
+
Agents in lab mode are real Claude Code sessions (`claude -p`) on your Claude subscription, each in its own git worktree under `workspace/.worktrees/`. No API key, no paid infrastructure, anywhere in this project.
|
|
126
|
+
|
|
127
|
+
## What the record actually contains
|
|
128
|
+
|
|
129
|
+
**Who it is for.** The reviewer, who was not in the room. So the narration names the supervisor rather than saying "you", and it leads with the actions that never happened. Pass `--audience supervisor` for the second-person version.
|
|
130
|
+
|
|
131
|
+
Scene by scene: the task verbatim, every held request with the answer given and how long it took, each round of changes as a diff, anything rolled back, and one closing view of what was asked against what the agent claims it did. Every figure is computed from the recording.
|
|
132
|
+
|
|
133
|
+
The push hook builds this for you. To build one by hand:
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
node scripts/build-recap.mjs --branch feat/x --repo ~/code/my-app # a branch
|
|
137
|
+
node scripts/build-recap.mjs latest # one session
|
|
138
|
+
node scripts/build-recap.mjs latest --llm # Claude rewrites the sentences, never the facts
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Output is one self-contained HTML file in `ui/records/`, served at `/records/…` while the server runs.
|
|
142
|
+
|
|
143
|
+
### The voice
|
|
144
|
+
|
|
145
|
+
macOS ships three tiers of every voice. The **compact** one is installed by default and is the robot everyone recognises. **Enhanced** and **Premium** are free downloads and sound dramatically better:
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
System Settings → Accessibility → Spoken Content → System Voice → Manage Voices
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
The builder picks the best tier it finds and tells you when all it has is compact. See what you have:
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
node scripts/build-recap.mjs --voices
|
|
155
|
+
node scripts/build-recap.mjs latest --voice "Ava (Premium)"
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
**Better still, read it yourself.** Synthesis is a stand-in for a person reading their own words, and for the one record you put in front of people it is worth ten minutes:
|
|
159
|
+
|
|
160
|
+
```bash
|
|
161
|
+
node scripts/build-recap.mjs latest --script # writes records/<slug>-script.md
|
|
162
|
+
# record each numbered line into a folder as 01.m4a, 02.m4a, …
|
|
163
|
+
node scripts/build-recap.mjs latest --voice-dir ~/Desktop/narration
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Lines you have not recorded fall back to the system voice, so you can do them a few at a time.
|
|
167
|
+
|
|
168
|
+
Rules the builder follows:
|
|
169
|
+
|
|
170
|
+
- Every number, diff and decision is computed from `recordings/<session>.jsonl` and the worktree's git history. With `--llm`, Claude only rewrites the narration sentences; it cannot add or change a fact, and the page says which mode produced it.
|
|
171
|
+
- Narration is macOS `say` converted to AAC and embedded, so the file needs no server and no API key. About 6 KB per second of speech.
|
|
172
|
+
- The storyboard is also written to `records/<agent>-<id>.json` for inspection.
|
|
173
|
+
|
|
174
|
+
## The branch is the unit, not the session
|
|
175
|
+
|
|
176
|
+
A reviewer opens a pull request, not a session. One branch collects several agent sessions over days, so the record merges all of them, numbering each instruction in order. A branch record supersedes the per-session records inside it, so the published index never shows the same story twice.
|
|
177
|
+
|
|
178
|
+
## Hand it over at push time
|
|
179
|
+
|
|
180
|
+
Pushing is the moment the work stops being yours and becomes someone else's to
|
|
181
|
+
review, so that is when the record should change hands.
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
node scripts/install-push-hook.mjs ~/code/my-app
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
That installs a `pre-push` hook. On your next push it builds the branch record,
|
|
188
|
+
prints what it found including anything refused, and — only if there is an open
|
|
189
|
+
pull request — asks whether to post it. Answer `y` and it comments; anything else
|
|
190
|
+
and the push just continues.
|
|
191
|
+
|
|
192
|
+
Three rules it follows:
|
|
193
|
+
|
|
194
|
+
- **It never blocks a push.** No sessions on the branch, no server, a crash, a
|
|
195
|
+
timeout: it prints one dim line at most and exits 0.
|
|
196
|
+
- **It never posts without you.** A record of what you refused is more revealing
|
|
197
|
+
than a diff. Publishing that to a shared pull request is your call, every time.
|
|
198
|
+
- **It stays fast.** Narration is skipped by default, because a minute of `say`
|
|
199
|
+
at every push is not acceptable. Set `NEARLY_AUDIO=1` when you want the good one.
|
|
200
|
+
|
|
201
|
+
Set `NEARLY_URL_BASE` to the hosted path so the comment can link to the page:
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
export NEARLY_URL_BASE=https://<user>.github.io/<repo>/records
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Today the poster speaks GitHub, through the `gh` CLI. Bitbucket and GitLab each
|
|
208
|
+
need their own small poster; the record itself is provider-agnostic, since it is
|
|
209
|
+
just a hosted page and a link.
|
|
210
|
+
|
|
211
|
+
## Publish the records
|
|
212
|
+
|
|
213
|
+
Recap pages are self-contained HTML, so GitHub Pages hosts them for free and the links in pull request comments resolve for anyone who can see the repo.
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
node scripts/publish-pages.mjs --base https://<user>.github.io/<repo>
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
That copies every built recap into `docs/records/` and writes `docs/index.html`, an index of the sessions on record. Commit `docs/`, then set **Settings → Pages → branch `main`, folder `/docs`**. Preview it locally first at http://127.0.0.1:47653/docs/ while the server is running.
|
|
220
|
+
|
|
221
|
+
## Why the hook fails open
|
|
222
|
+
|
|
223
|
+
Claude Code treats a hook that times out, errors, or returns anything other than `200` with JSON as a non-blocking error and lets the tool call proceed. So this server always answers with JSON, holds "ask" calls for at most `ASK_TIMEOUT_MS`, and denies when nobody decides. The hook's own timeout is set longer than that.
|
|
224
|
+
|
|
225
|
+
## Tests
|
|
226
|
+
|
|
227
|
+
```bash
|
|
228
|
+
npm test
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
41 tests, no dependencies, about 30 seconds. They run on a fresh clone with no
|
|
232
|
+
agent, no network and no Claude subscription, because the fixtures are the two
|
|
233
|
+
recorded sessions committed in `recordings/demo`.
|
|
234
|
+
|
|
235
|
+
What they hold the project to:
|
|
236
|
+
|
|
237
|
+
- **The consent gradient.** That destructive commands are denied without asking,
|
|
238
|
+
that a never pattern still fires when the command is buried in a chain, that an
|
|
239
|
+
unclassified tool is held rather than allowed, and that "always" for `git status`
|
|
240
|
+
can never become permission for `git push`.
|
|
241
|
+
- **The hook contract.** The exact JSON Claude Code reads, for every tier. That an
|
|
242
|
+
ask really is held until somebody answers, and that **nobody answering means
|
|
243
|
+
denied**, which is the assumption the whole design rests on.
|
|
244
|
+
- **The record's central claim.** That every figure on the page matches the
|
|
245
|
+
recording: the refusal count, who refused each one, every instruction verbatim
|
|
246
|
+
and in order. The test reads the recordings itself rather than trusting the
|
|
247
|
+
builder's own summary.
|
|
248
|
+
- **The failure paths**, which are the ones that lose you a user silently. That a
|
|
249
|
+
hook whose server cannot start stays quiet and exits 0 rather than wedging a
|
|
250
|
+
session. That two hooks racing for the port do not crash. That turning it on
|
|
251
|
+
twice installs nothing twice, turning it off removes everything, and neither
|
|
252
|
+
touches settings somebody else put there.
|
|
253
|
+
|
|
254
|
+
## Study
|
|
255
|
+
|
|
256
|
+
The claim this project makes is testable: a reviewer who sees the session record catches something a reviewer who sees only the diff misses. [`STUDY.md`](STUDY.md) is the protocol — two seeded-error tasks, three participants, and rules for reporting the result honestly including when it is negative.
|
|
257
|
+
|
|
258
|
+
## Roadmap
|
|
259
|
+
|
|
260
|
+
- **Read Prempti's audit trail as an input.** Their recording is structured, local, Apache-licensed and covers more than ours. The recap builder reads its own JSONL today; a second reader would let anyone already running Prempti get a session record without changing their gate.
|
|
261
|
+
- **A Cursor adapter.** The server speaks JSON in and JSON out, so a tool that runs a script instead of calling a URL needs about twenty lines of translation. Cursor's `beforeShellExecution` is the first target.
|
|
262
|
+
- **Port the recap player to React.** It is one self-contained page today.
|
|
263
|
+
|
|
264
|
+
## Files
|
|
265
|
+
|
|
266
|
+
- `server/index.mjs`, spawn sessions, hooks, policy, recorder, undo
|
|
267
|
+
- `ui/index.html`, sessions, triage of pending approvals, rules, log
|
|
268
|
+
- `scripts/attach.mjs`, install or remove the hooks in a repo of your own; `scripts/post-recap.mjs`, comment the recap on its PR
|
|
269
|
+
- `scripts/build-recap.mjs` + `ui/recap.template.html`, narrated recap page per session
|
|
270
|
+
- `scripts/publish-pages.mjs`, build the `docs/` folder GitHub Pages serves
|
|
271
|
+
- `scripts/install-push-hook.mjs` + `scripts/push-record.mjs`, hand the branch record over at `git push`
|
|
272
|
+
- `workspace/`, the repo agents work on (seeded with the Tempo demo)
|
|
273
|
+
- `recordings/<session>.jsonl`, every event and decision; `recordings/demo/` is committed so the records can be rebuilt from source
|
|
274
|
+
- `STUDY.md`, the protocol for testing whether any of this helps a reviewer
|
package/STUDY.md
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# Seeded-error study
|
|
2
|
+
|
|
3
|
+
**The claim to test:** a reviewer who sees the session record catches something a reviewer who sees only the diff misses.
|
|
4
|
+
|
|
5
|
+
Nobody in this space has published evidence that their surface changes what a person catches. Prempti, Agent Approve and Endor Labs all ship a product; none of them ships a number. This is the cheapest thing on the schedule that cannot be copied in a sprint.
|
|
6
|
+
|
|
7
|
+
Three participants, twenty minutes each, no incentives, run in person or over a call with screen share.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Design
|
|
12
|
+
|
|
13
|
+
Within-subject, two tasks per participant, order counterbalanced.
|
|
14
|
+
|
|
15
|
+
| | Condition A, control | Condition B, treatment |
|
|
16
|
+
|---|---|---|
|
|
17
|
+
| What they get | The pull request: title, description, diff | The same pull request, plus a link to the session record |
|
|
18
|
+
| What they do | Decide: approve, request changes, or block | Same |
|
|
19
|
+
|
|
20
|
+
Each participant does one task in each condition. Participant 1 gets A then B, participant 2 gets B then A, participant 3 gets A then B. With three people this does not remove order effects, it only stops them all pointing the same way. Say so in the write-up.
|
|
21
|
+
|
|
22
|
+
**Do not tell participants what the study is about.** Say: "you are reviewing a branch a coding agent wrote; decide whether to approve it." Nothing about refusals, nothing about Nearly.
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## The two tasks
|
|
27
|
+
|
|
28
|
+
Both run against the demo workspace. Both produce a diff that looks correct and passes a glance. The planted problem is not in the diff.
|
|
29
|
+
|
|
30
|
+
### Task 1 — the unrun test
|
|
31
|
+
|
|
32
|
+
The agent is asked to change behaviour **and run the test suite to confirm**. The supervisor refuses the test command. The agent edits the code correctly and reports honestly that tests did not run.
|
|
33
|
+
|
|
34
|
+
- **Diff shows:** a clean, plausible code change.
|
|
35
|
+
- **Diff hides:** that the change was never verified, because the verification step was refused.
|
|
36
|
+
- **Catch condition:** the participant says, unprompted, that the change is unverified or that tests did not run.
|
|
37
|
+
|
|
38
|
+
### Task 2 — the rolled-back turn
|
|
39
|
+
|
|
40
|
+
The agent makes two rounds of changes. The supervisor undoes the second. The branch ends in a state that looks deliberate.
|
|
41
|
+
|
|
42
|
+
- **Diff shows:** the surviving change only.
|
|
43
|
+
- **Diff hides:** that the agent tried a second approach which a human threw away, and why that matters for the reviewer's suggestion to "just also do X".
|
|
44
|
+
- **Catch condition:** the participant mentions that something was attempted and reverted, or asks what else was tried.
|
|
45
|
+
|
|
46
|
+
Record both sessions once and reuse the recordings for all three participants, so every participant sees identical material. Build with `--audience reviewer`, which is the default.
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Script
|
|
51
|
+
|
|
52
|
+
Read this aloud, the same way each time.
|
|
53
|
+
|
|
54
|
+
> You are reviewing a branch that a coding agent wrote. Another engineer supervised the agent while it worked. Take as long as you need, think out loud, and at the end tell me whether you would approve it, request changes, or block it.
|
|
55
|
+
|
|
56
|
+
Then, in the treatment condition only, add:
|
|
57
|
+
|
|
58
|
+
> There is also a link to a record of the session in the pull request comment. Use it or ignore it, whichever you would normally do.
|
|
59
|
+
|
|
60
|
+
Say nothing else. Do not point at the refusals. If they ask what the record is, say "have a look and tell me."
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## What to write down
|
|
65
|
+
|
|
66
|
+
For each task, per participant:
|
|
67
|
+
|
|
68
|
+
| Field | How to record it |
|
|
69
|
+
|---|---|
|
|
70
|
+
| Condition | A or B |
|
|
71
|
+
| Caught the planted problem | yes / no, and the exact words they used |
|
|
72
|
+
| Time to decision | stopwatch, from start to their verdict |
|
|
73
|
+
| Opened the record (B only) | yes / no, unprompted or after the prompt |
|
|
74
|
+
| Verdict | approve / request changes / block |
|
|
75
|
+
| Quote | one sentence in their words worth publishing |
|
|
76
|
+
|
|
77
|
+
The headline number is the first row: how many of three caught it with the record, how many without.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## Reporting honestly
|
|
82
|
+
|
|
83
|
+
With three people this is a **usability test, not an experiment**. Write it up that way:
|
|
84
|
+
|
|
85
|
+
- Report the raw count, never a percentage. "Two of three caught it with the record, none of three without" is honest. "67% improvement" is not.
|
|
86
|
+
- Report participants who caught it for the wrong reason as misses of the design, not hits.
|
|
87
|
+
- If the record does not help, say so and publish it anyway. A negative result honestly reported is still evidence you ran the test, and it is more interesting than another feature.
|
|
88
|
+
- Note every confound: three people, order effects, the author present, participants who know what the project is.
|
|
89
|
+
|
|
90
|
+
The essay sentence to aim for is a fact, not a claim: *given the same branch, N of three reviewers noticed the refused test run with the record, and N without.*
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Before the session
|
|
95
|
+
|
|
96
|
+
- [ ] Both recordings captured with a real agent, not simulated
|
|
97
|
+
- [ ] Both records built and reachable at a public link
|
|
98
|
+
- [ ] Both pull requests open, with the recap comment posted on the treatment one
|
|
99
|
+
- [ ] Recap opens in under two seconds on a cold load
|
|
100
|
+
- [ ] Practice run with someone who is not a participant, to fix the script
|
|
101
|
+
- [ ] Stopwatch, and a written sheet per participant, not memory
|
package/bin/nearly.mjs
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// The one command. Everything else is a subcommand of this.
|
|
3
|
+
//
|
|
4
|
+
// nearly turn it on for the repo you are in
|
|
5
|
+
// nearly off turn it off again
|
|
6
|
+
// nearly open open the dashboard
|
|
7
|
+
// nearly record build the record for the current branch
|
|
8
|
+
// nearly post put that record on the pull request
|
|
9
|
+
// nearly voices list the narration voices you have
|
|
10
|
+
// nearly server run the server in the foreground (it self-starts otherwise)
|
|
11
|
+
// nearly hook <ev> internal: what the Claude Code hooks call
|
|
12
|
+
//
|
|
13
|
+
// Run it with no arguments inside a git repo and it does the useful thing,
|
|
14
|
+
// because the useful thing is what people type first.
|
|
15
|
+
|
|
16
|
+
import { spawn, spawnSync, execFileSync } from 'node:child_process';
|
|
17
|
+
import { join, dirname, resolve } from 'node:path';
|
|
18
|
+
import { fileURLToPath } from 'node:url';
|
|
19
|
+
|
|
20
|
+
const root = resolve(join(dirname(fileURLToPath(import.meta.url)), '..'));
|
|
21
|
+
const s = (n) => join(root, 'scripts', n);
|
|
22
|
+
// Commands a person typed and is waiting on may mention an update. The hook
|
|
23
|
+
// never does: nothing goes in front of an agent's tool call.
|
|
24
|
+
const NOTIFY = new Set(['attach', 'on', 'init', 'record', 'recap', 'post', 'publish']);
|
|
25
|
+
|
|
26
|
+
const run = async (file, args = []) => {
|
|
27
|
+
const r = spawnSync(process.execPath, [file, ...args], { stdio: 'inherit' });
|
|
28
|
+
if (NOTIFY.has(cmd) && (r.status ?? 0) === 0) {
|
|
29
|
+
try {
|
|
30
|
+
const { checkForUpdate, applyUpdate } = await import('../scripts/update-check.mjs');
|
|
31
|
+
applyUpdate(await checkForUpdate());
|
|
32
|
+
} catch { /* an update notice is never worth an error */ }
|
|
33
|
+
}
|
|
34
|
+
process.exit(r.status ?? 0);
|
|
35
|
+
};
|
|
36
|
+
|
|
37
|
+
const [cmd = 'attach', ...rest] = process.argv.slice(2);
|
|
38
|
+
|
|
39
|
+
async function main() {
|
|
40
|
+
switch (cmd) {
|
|
41
|
+
case 'attach': case 'on': case 'init':
|
|
42
|
+
return run(s('attach.mjs'), rest);
|
|
43
|
+
|
|
44
|
+
case 'off': case 'detach':
|
|
45
|
+
return run(s('attach.mjs'), [...rest, '--off']);
|
|
46
|
+
|
|
47
|
+
// Used by attach to confirm a `nearly` on PATH really is this tool before
|
|
48
|
+
// pointing hooks at a bare command name.
|
|
49
|
+
case '--which':
|
|
50
|
+
console.log(root);
|
|
51
|
+
return process.exit(0);
|
|
52
|
+
|
|
53
|
+
case 'hook':
|
|
54
|
+
return run(s('hook.mjs'), rest);
|
|
55
|
+
|
|
56
|
+
case 'record': case 'recap': {
|
|
57
|
+
// Default to the branch you are on, since that is what gets reviewed.
|
|
58
|
+
if (rest.length) return run(s('build-recap.mjs'), rest);
|
|
59
|
+
try {
|
|
60
|
+
const branch = execFileSync('git', ['rev-parse', '--abbrev-ref', 'HEAD'], { encoding: 'utf8' }).trim();
|
|
61
|
+
return run(s('build-recap.mjs'), ['--branch', branch, '--repo', process.cwd()]);
|
|
62
|
+
} catch { return run(s('build-recap.mjs'), ['latest']); }
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
case 'post':
|
|
66
|
+
return run(s('post-recap.mjs'), rest.length ? rest : ['latest']);
|
|
67
|
+
|
|
68
|
+
case 'publish':
|
|
69
|
+
return run(s('publish-pages.mjs'), rest);
|
|
70
|
+
|
|
71
|
+
case 'voices':
|
|
72
|
+
return run(s('build-recap.mjs'), ['--voices']);
|
|
73
|
+
|
|
74
|
+
case 'server': {
|
|
75
|
+
console.log('Nearly on http://127.0.0.1:47653');
|
|
76
|
+
console.log('You do not normally need this: the hooks start it when they need it.');
|
|
77
|
+
return run(join(root, 'server', 'index.mjs'), rest);
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
case 'open': {
|
|
81
|
+
const url = 'http://127.0.0.1:47653';
|
|
82
|
+
spawn(process.platform === 'darwin' ? 'open' : process.platform === 'win32' ? 'start' : 'xdg-open',
|
|
83
|
+
[url], { stdio: 'ignore', detached: true, shell: process.platform === 'win32' }).unref();
|
|
84
|
+
console.log(url);
|
|
85
|
+
return process.exit(0);
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
case 'help': case '--help': case '-h': {
|
|
89
|
+
console.log(`
|
|
90
|
+
nearly turn it on for the repo you are in
|
|
91
|
+
nearly off turn it off again
|
|
92
|
+
nearly open open the dashboard
|
|
93
|
+
nearly record build the record for the current branch
|
|
94
|
+
nearly post put that record on the pull request
|
|
95
|
+
nearly voices list the narration voices you have
|
|
96
|
+
nearly server run the server in the foreground
|
|
97
|
+
`);
|
|
98
|
+
return process.exit(0);
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
default:
|
|
102
|
+
console.error(`Unknown command: ${cmd}`);
|
|
103
|
+
console.error('Try: nearly help');
|
|
104
|
+
process.exit(1);
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
|
|
108
|
+
await main();
|
package/package.json
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "nearly-cli",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "A pull request tells you what changed. Nearly tells you what nearly happened: the commands a human refused, the pushes policy blocked, the turns rolled back.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"nearly": "bin/nearly.mjs"
|
|
8
|
+
},
|
|
9
|
+
"files": [
|
|
10
|
+
"bin",
|
|
11
|
+
"scripts",
|
|
12
|
+
"server",
|
|
13
|
+
"ui/index.html",
|
|
14
|
+
"ui/recap.template.html",
|
|
15
|
+
"README.md",
|
|
16
|
+
"STUDY.md"
|
|
17
|
+
],
|
|
18
|
+
"engines": {
|
|
19
|
+
"node": ">=18"
|
|
20
|
+
},
|
|
21
|
+
"dependencies": {},
|
|
22
|
+
"keywords": [
|
|
23
|
+
"claude-code",
|
|
24
|
+
"coding-agent",
|
|
25
|
+
"code-review",
|
|
26
|
+
"pull-request",
|
|
27
|
+
"agent-supervision",
|
|
28
|
+
"hooks"
|
|
29
|
+
],
|
|
30
|
+
"license": "MIT",
|
|
31
|
+
"repository": {
|
|
32
|
+
"type": "git",
|
|
33
|
+
"url": "https://github.com/anujpatel06/nearly.git"
|
|
34
|
+
},
|
|
35
|
+
"homepage": "https://anujpatel06.github.io/nearly/",
|
|
36
|
+
"scripts": {
|
|
37
|
+
"test": "node --test --test-concurrency=1 test/"
|
|
38
|
+
}
|
|
39
|
+
}
|