@trybullet/cli 1.3.12

package/README.md

@@ -0,0 +1,366 @@
+# Bullet ⁍
+
+**The fastest coding agent.** A minimalist macOS desktop app: open a folder and
+Bullet plans, edits files, runs commands, verifies its work, and shows you
+everything live. Zero setup — runs on-device with no key.
+
+## How it works
+
+- **Agent loop** (custom, ~200 lines — no framework): plan → tool calls
+  (read / edit / write / delete / run) → observations → verify → done. Hardened
+  with failure-retry directives, stuck detection, duplicate suppression, a 45s
+  wall-clock cap, and a definition-of-done verification gate.
+- **Router:** a nano-model semantic classifier (hard 4s cap — routing is never
+  the wait) rates each prompt on five difficulty levels. No key → everything
+  runs on-device (Qwen2.5-Coder via Metal, warm KV-cache sessions). With a key
+  → each level buys a deeper model *and* more thinking effort: quick asks run
+  on fast models (Haiku / Codex-low), harder work escalates (Sonnet 5 / Codex
+  at rising effort), and *extreme* tasks go to Fable 5 / Sol at full effort
+  (say "use fable", "use sol", or "think hard" to force the level). The debug
+  badge shows the exact model, tier, and effort used.
+- **Plan gate:** asking to plan ("help me plan X", "let's figure out the
+  approach first") is its own classifier answer — those turns draft a plan and
+  stop instead of working. Difficulty alone never triggers it: a big task you
+  told Bullet to *do* just gets done.
+- **Providers:** Bring a ChatGPT/Codex or Claude Code subscription, or use an
+  Anthropic, OpenAI, or xAI API key. Switch methods and providers in Settings
+  (⌘,). OAuth credentials are encrypted locally; keys stay on this Mac.
+- **Tools:** read / write / edit / delete, `search_code`, `find_files`,
+  `run_command`, background processes, web search + fetch, and a real browser
+  (below). File tools reach any absolute path; commands run from the workspace.
+- **Code search that lands.** `search_code` is extended-regex, so `(a|b)`, `\d`
+  and `+` mean what the model thinks they mean — grep's default dialect treats
+  them as literal characters, which silently answered "no matches" for code that
+  was right there. A pattern that finds nothing is retried as literal text
+  before that answer is allowed, matches inside minified files are windowed with
+  a column instead of dumping a 400 KB line, `*.{ts,tsx}` is expanded, and a
+  directory you explicitly point at is never excluded as noise.
+- **Eyes.** `open_page` → `screenshot` → `read_page` → `page_action` /
+  `page_eval` drive an offscreen Chromium (no Playwright, no extra dependency).
+  The screenshot goes to the model *and* into the transcript, so a visual change
+  is verified by looking at it. `read_page` always reports console errors and
+  failed requests, every load clears the cache, and `page_action` includes
+  relative mouse-look for pointer-locked pages. Older captures are pruned from
+  the context so a long visual session doesn't pay for stale pixels.
+- **Speed levers:** warm KV sessions (no re-processing between turns),
+  patch-based edits (fewer output tokens), prompt caching on Anthropic,
+  parallel tool execution on cloud, flash attention on Metal, idle model
+  unload (frees ~2 GB after 5 min). One dev server per command: re-running
+  `npm run dev` returns the server already running with its URL rather than
+  starting a seventh one on a seventh port, and `stop_process` ends it.
+
+## UI
+
+- **Sidebar grouped by folder** — chats are bound to their folder; agents in
+  different folders run concurrently (busy dot), and agents in the *same*
+  folder are told about each other's recent edits.
+- **Live activity**: plan checklist ticks in real time, action rows
+  (`✓ Edited app.js +10 −3`, click to expand output), a status pulse with an
+  elapsed counter through every silent reasoning stretch (never looks frozen;
+  dead connections error out via stream watchdogs), and a stop button (Esc).
+- **Resizable panels**: drag the sidebar and Activity/Preview edges; widths persist.
+- **Activity panel** (right): running/finished commands + per-file diffs.
+- **Plan panel**: a plan-first request drafts markdown (goal, approach, risks,
+  steps) into the side panel and *waits* — reply with feedback to iterate, ask
+  questions and the plan stays put, edit it directly, then **Run plan** (or
+  "run it") executes. Steps tick as the agent works; mid-run edits steer it.
+  The plan lives in the panel, never in the chat. Discard, or just move on to
+  other work, exits plan mode.
+- **LaTeX**: `$$…$$`, `\[…\]`, `\(…\)`, and math-y `$…$` render via KaTeX
+  (bundled, offline); dollar amounts in prose stay literal.
+- **Preview panel**: in-app browser for whatever the agent is serving. It
+  opens itself when the agent starts a dev server, and auto-reloads (cache
+  cleared) after each edit burst — toggleable via the `auto` button.
+- **Prompt history**: ↑/↓ in the composer cycles your previous prompts,
+  terminal-style; your in-progress draft is preserved.
+- **Attachments**: 📎, drag in, or **paste** code/text, images & screenshots
+  (sent as vision to cloud models), and PDF / Word / Excel / PowerPoint
+  (auto-extracted to text — works with the local model too).
+  **Rewind/fork**: hover any of your messages —
+  rewinding confirms first, then also reverts the file edits made after that
+  point (command side-effects are not undone). **Two themes**: Midnight / Paper.
+
+## CLI
+
+The terminal front end. Not a port — it loads `electron/router.js` and every
+module under it, so routing, tools, skills, sub-agents, the rewind ledger and
+the provider auth are byte-for-byte the same code the app runs.
+
+```bash
+npm link                          # puts `bullet` on your PATH
+bullet auth                       # Claude / ChatGPT subscription, or an API key
+bullet                            # interactive session in this folder
+bullet "fix the failing tests"    # run one task, then keep talking
+bullet -p "what does app.js do"   # print and exit — pipeable
+git diff | bullet -p "review this"
+```
+
+Inside a session: `enter` sends, `ctrl+j` newlines, `esc` stops a running turn,
+`tab` completes `/commands` and `@files`, `↑` walks history. `/help` lists the
+rest — `/model`, `/cost`, `/diff`, `/undo`, `/resume`, `/agents`.
+
+Sign-in reuses whatever is already on the machine: an existing Claude Code or
+Codex login is imported directly, no browser round-trip. Credentials are
+encrypted with the OS keychain (macOS Keychain, `secret-tool` on Linux) and kept
+in `~/.bullet`, separate from the app's own store.
+
+In scripts, `-p` exits `0` when it answered, `1` when the turn errored, `130`
+when you interrupted it, and `2` on bad arguments — so `bullet -p … || handle-it`
+does what it looks like. stdout carries only the answer; routing notes and
+warnings go to stderr.
+
+Two capabilities do not cross over. The agent's browser — `open_page`,
+`screenshot`, `read_page` — needs the offscreen Chromium only Electron can host,
+so those tools are removed from the kit and the system prompt is told, rather
+than left to fail mid-turn. And with no provider connected the CLI falls back to
+the same on-device model the app uses; that is a one-time ~2 GB download, so it
+asks first, and `-p` refuses rather than pulling 2 GB in a script nobody is
+watching.
+
+How it stays one codebase: `cli/shim.js` registers itself as the `electron`
+module before anything under `electron/` is required, and supplies the four
+things the shared core actually asks Electron for — a user-data directory,
+`safeStorage`, `shell.openExternal`, and a `BrowserWindow` that explains why it
+isn't there. Everything in `cli/` above that line is presentation.
+
+## Run
+
+```bash
+npm install
+npm start        # dev
+npm test         # UI liveness + feature regressions (offline, no key)
+npm run cli      # the CLI, from source
+npm run test:cli # CLI suites (plain node, offline, no key)
+npm run dist     # local unsigned build (right-click → Open)
+```
+
+`npm run eval:classifier` additionally checks routing quality — plan intent and
+plan-approval verdicts — against a live nano model, so it needs a provider key.
+
+## Releasing
+
+Installed copies update themselves: the app checks
+`https://api.davidhf.com/bullet/v1/updates/mac/latest-mac.yml` on launch and
+every six hours, downloads in the background, and installs on quit (Settings ›
+Updates has a manual check and a *Restart to update* button). Nobody has to
+delete and reinstall.
+
+**This only works for signed builds.** macOS applies updates through
+Squirrel.Mac, which requires the downloaded bundle to satisfy the running app's
+code-signing requirement — an ad-hoc signed build can never update itself, and
+publishing one strands every user who installs it on that version.
+`scripts/publish-update.sh` refuses to publish an unsigned build for that reason.
+
+One-time setup on the build Mac:
+
+1. In Xcode › Settings › Accounts, sign in with the Apple Developer account, then
+   **Manage Certificates › + › Developer ID Application**. Confirm with
+   `security find-identity -v -p codesigning` — you need a line reading
+   `Developer ID Application: … (TEAMID)`, not just `Apple Development`.
+2. Create an app-specific password at <https://appleid.apple.com> › Sign-In and
+   Security › App-Specific Passwords — notarization rejects the account
+   password — and put the credentials in `.env.notarize` at the repo root:
+
+   ```bash
+   APPLE_ID="you@example.com"
+   APPLE_APP_SPECIFIC_PASSWORD="xxxx-xxxx-xxxx-xxxx"
+   APPLE_TEAM_ID="YOURTEAMID"   # Membership details on developer.apple.com
+   ```
+
+   `npm run release` loads that file itself. It is gitignored, it must stay
+   that way, and the Team ID in it has to match the team that issued the
+   signing certificate — the script checks and stops if they disagree, because
+   Apple rejects the notarization otherwise.
+
+Then each release:
+
+```bash
+npm version patch          # or edit "version" — the feed compares against it
+npm run release            # signs + notarizes, writes release/latest-mac.yml
+npm run release:publish     # uploads to the update feed on the API box
+```
+
+`npm run release` takes a few extra minutes the first time — notarization is a
+round trip to Apple. Running clients pick the new version up within six hours,
+or immediately from Settings › Updates.
+
+### Signing on someone else's Mac
+
+**Bullet is signed on one specific Mac, and that never changes.** Squirrel.Mac
+only applies an update whose signature satisfies the running app's requirement,
+so every release must be signed by the same Developer ID / Team ID as the one
+before it. Signing a release under a different team strands every installed
+copy — those users would each need a manual reinstall to get updates back.
+Keep a `.p12` export of that certificate (and its password) somewhere safe
+off that machine: losing access to it means losing the ability to ship updates
+to everyone already running Bullet.
+
+Whoever signs builds from this repo rather than re-signing someone else's `.app`
+— electron-builder rebuilds the native modules and signs the unpacked
+`node-llama-cpp` binaries as part of packaging, and any hand-editing of a signed
+bundle invalidates both the signature and the sha512 in `latest-mac.yml`.
+
+On their Mac: Xcode command line tools, Node, the **Developer ID Application**
+certificate in the login keychain, then
+
+```bash
+git clone … && cd bullet && npm install
+# write .env.notarize with the Apple ID, app-specific password, and the Team ID
+# that issued the certificate on this Mac (see step 2 above)
+npm run release
+```
+
+(`CSC_NAME="Developer ID Application: Name (TEAMID)"` if the keychain holds more
+than one Developer ID certificate.) Leave `build/entitlements.mac.plist` alone —
+JIT, unsigned executable memory, and library validation are all off-limits to
+strip, because the on-device model needs them under the hardened runtime.
+
+Checks before handing the build over:
+
+```bash
+codesign -dv --verbose=2 release/mac-arm64/Bullet.app  # Authority=Developer ID Application: … (TEAMID)
+spctl -a -vv release/mac-arm64/Bullet.app              # accepted, source=Notarized Developer ID
+xcrun stapler validate release/Bullet-*-arm64.dmg
+```
+
+Then hand over `release/` intact — the `.zip` (electron-builder's, never a
+Finder-made one, which mangles symlinks and the signature), its `.blockmap`, the
+`.dmg`, and the `latest-mac.yml` from that same build, whose hashes must match
+the files beside it. `npm run release:publish` does the upload and needs the VM
+SSH key, so it is easiest for whoever holds that key to run it on the received
+`release/` directory.
+
+## SWE-bench Verified
+
+`scripts/swe/` runs Bullet's real agent over SWE-bench Verified and grades every
+patch with the official Docker scorer — see
+[scripts/swe/README.md](scripts/swe/README.md).
+
+**All 500 instances, officially scored: 479/500 = 95.8%.** Native x86, one attempt
+per instance, network on, `gpt-5.6-sol`, `swebench.harness.run_evaluation`. Zero
+empty patches, zero worker errors, every instance graded.
+
+| | resolved | round trips¹ | latency | cost/inst |
+|---|---|---|---|---|
+| **Bullet — all 500** | **479/500 = 95.8%** | **15.0** | **119s** | **$0.73** |
+| mini-swe-agent + sol@max — all 500 | 421/500 = 84.2% | 12.6 | 91s | $0.29 |
+
+Resolve rate by repository, so it is clear no single project is carrying it:
+django 96.5% (223/231), sympy 97.3%, sphinx 95.5%, matplotlib 94.1%,
+scikit-learn 100%, xarray 95.5%, astropy 81.8%, pytest 100%, pylint 90%,
+requests 87.5%.
+
+**The 17-point asterisk.** Both rows above have network access, which is the
+reference's default and — on the evidence of its cost and latency — the published
+leaderboards' too. That matters enormously: these instances are public GitHub PRs,
+so an agent with a route out can fetch the merged upstream fix, and the reference
+does exactly that on 23% of instances (`curl raw.githubusercontent.com/<the very
+file>`, `api.github.com/search/issues?q=<the bug>`). Measured directly on the
+random 120, same configuration, only the network changed:
+
+| | resolved |
+|---|---|
+| network on | 113/120 = 94.2% |
+| `--network none`, no web tools | 92/120 = 76.7% |
+
+**So 95.8% is the leaderboard-comparable number and ~77% is the from-the-repository
+-alone number.** Offline is the default here; `SWE_NETWORK=1` is the switch.
+
+¹Round trips — model calls per instance — is the only speed figure worth quoting.
+Wall clock is provider-load dependent: the *same* configuration on the *same*
+instances measured 131s and 248s nine hours apart with the machine idle. Turns
+don't drift. `profile.js` reports both.
+
+**What the turn-efficiency work bought**, paired on the same instances: 17.2 → 14.4
+round trips and $0.93 → $0.68 an instance, for an accuracy change that is not
+statistically separable (0 gained, 2 lost, McNemar p = 0.50 — a 1-2 point cost is
+possible and unproven). Against the reference on the shared 120, Bullet resolved
+every instance it resolved and 17 more, nothing the other way (**p < 0.001**).
+
+A gold control on this machine resolves 117/120 reference patches on the random
+120; the three exceptions (a live HTTP call, an env-sensitive header test, an
+extension-version test) cannot pass whatever the patch does. So roughly 2.5% of any
+raw rate here is the machine, not the agent — `ceiling.js` reports the corrected
+figure. No gold control was run over all 500, so the 95.8% above is raw.
+
+**What moved it.** Routing: the deep model resolves 62.1% at 138s against the fast
+model's 36.8% at 152s, so the best model leads at every level instead of being an
+escalation. Reliability: an earlier run submitted 13 blank patches from dead
+workers, and there are none now. Two defects found by measurement rather than by
+reading:
+
+- **`search_code` ran in grep's BRE dialect**, where `(`, `|` and `+` are literal
+  characters — so `function (alpha|beta)` reported "no matches" for code plainly
+  present. Searches are 40% of all tool calls, so this cost both accuracy and a
+  great many wasted round-trips.
+- **The shared git mirror had a creation race.** `git clone` makes its directory
+  immediately and fills it over minutes, so a worker that only tested existence
+  cloned from a half-written mirror and lost the instance one second in. Six of 120,
+  all in the repositories eight workers wanted at once; all six resolved on re-run.
+
+A third defect, found by a one-instance smoke test after the runs above: an
+undeclared timer variable threw at the first phase write, silently aborting the tail
+of every attempt — losing the reproduction it was meant to collect, which is what
+the candidate selector runs on. The patch survived because the error path recovers
+it from disk, which is exactly why it went unnoticed.
+
+Two other levers were paired, graded and landed within noise: the methodology brief
+versus bare environment facts, and structured tools versus a bare shell.
+
+**Latency is round-trip count and nothing else.** Instrumented per tool over the
+500: an instance is 120s, of which **checkout 0.3s, container start 0.7s, collect
+0.5s, tool execution 14s (12%, four fifths of it real test suites) — and 104s of
+model thinking**, i.e. 15 round trips at 7s each. There is nothing to win in the
+scaffolding; the only lever is needing fewer turns. Bullet's 15 against the
+reference's 12.6 is the entire speed difference between them. Beware comparing
+*tool calls* instead: Bullet issues ~38 of those, but batches 2.5 per turn, and
+confusing the two overstated the gap threefold in an earlier draft of this file.
+`profile.js` reports both.
+
+Two things carry the result, and a third had to be taken back out:
+
+- **A testbed the agent can run.** The attempt works in the instance's prepared
+  container, so it can actually reproduce the bug and run the project's tests
+  instead of reasoning about code it cannot execute. File tools keep editing a
+  local checkout (that is where the diff comes from) and edits are pushed across
+  before every command.
+- **The best model, at full depth, first.** Measured over 86 graded attempts, the
+  deep model resolved 62.1% at 138s against the fast model's 36.8% at 152s — better
+  *and* quicker, because it needs far fewer steps. Escalation is right for chat and
+  wrong here, so there is no ladder: one attempt, strongest model, highest effort.
+- **The ensemble is off by default, and the reason is not yet settled.** Several
+  attempts under different lenses, each leaving a reproduction, every reproduction
+  run against every candidate in fresh containers, a reviewer to break ties — that
+  configuration scored 104/120 against a single deep attempt's 113/120, losing nine
+  instances and winning none (p = 0.004). But that comparison is confounded: an
+  undeclared variable in the phase stopwatch was throwing partway through every
+  attempt, and the statement it aborted was the one that collected the attempt's
+  reproduction. So the selector — whose whole job is weighing reproductions — was
+  running on almost none, and fell back to preferring the first, shortest candidate
+  on 97 of 114 instances. The bug is fixed; the experiment has not been repeated.
+  One attempt is the default because it is the best *measured* configuration, not
+  because the ensemble is known to be worse. An oracle over the pair scored
+  105/114, so the headroom is real and `SWE_K` still turns it back on.
+
+The benchmark's answer key — test patch, reference patch,
+`FAIL_TO_PASS`/`PASS_TO_PASS` — never reaches inference; `export-instances.py`
+asserts those fields out of the data the agent sees. `SWE_NETWORK=1` is the switch
+behind the two regimes in the table; without it the web tools are removed and every
+container runs `--network none`.
+
+First prompt downloads the on-device model once (~2 GB), then it's local.
+Apple Silicon only.
+
+## Layout
+
+```
+electron/  main.js (window/IPC) · preload.js · router.js (routing+context)
+           engine.js (local agent loop) · cloud.js (cloud loops) · tools.js
+src/       index.html · styles.css · app.js
+cli/       bin.js · shim.js (fake `electron` for plain node) · main.js (repl)
+           session.js (host wiring) · render.js (events → transcript) · ui/
+scripts/   icon/logo generators · ui-test.js / feature-test.js (offline suites)
+           cli-test.js (CLI suites) · classifier-eval.js
+           eval.js / swebench.js / polyglot.js (live evals)
+BENCHMARK.md  one-shot website benchmark vs Claude Code / Codex
+```

package/bin/bullet.js

@@ -0,0 +1,20 @@
+#!/usr/bin/env node
+'use strict';
+// Dispatcher only. Every line of real Bullet logic lives in the compiled binary
+// shipped by the matching @trybullet/cli-<platform> package; this file just execs it.
+const { spawnSync } = require('child_process');
+const arch = process.arch;
+const platform = process.platform;
+const ext = platform === 'win32' ? '.exe' : '';
+const pkg = platform + '-' + arch;
+let bin;
+try {
+  bin = require.resolve('@trybullet/cli-' + pkg + '/bin/bullet' + ext);
+} catch {
+  console.error('bullet: no prebuilt binary for ' + pkg + '.');
+  console.error('Supported: darwin-arm64, darwin-x64, linux-x64, linux-arm64, win32-x64.');
+  process.exit(1);
+}
+const r = spawnSync(bin, process.argv.slice(2), { stdio: 'inherit' });
+if (r.error) { console.error('bullet: failed to launch (' + r.error.message + ')'); process.exit(1); }
+process.exit(r.status == null ? 1 : r.status);

package/package.json

@@ -0,0 +1,39 @@
+{
+  "name": "@trybullet/cli",
+  "version": "1.3.12",
+  "description": "The fastest coding agent — terminal edition. Same router, tools and agent loop as the Bullet desktop app, in your shell.",
+  "license": "UNLICENSED",
+  "homepage": "https://usebullet.com",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/trybullet/bullet.git"
+  },
+  "keywords": [
+    "cli",
+    "coding-agent",
+    "ai",
+    "agent",
+    "claude",
+    "codex"
+  ],
+  "author": "Bullet",
+  "bin": {
+    "bullet": "bin/bullet.js"
+  },
+  "files": [
+    "bin/bullet.js"
+  ],
+  "optionalDependencies": {
+    "@trybullet/cli-darwin-arm64": "1.3.12",
+    "@trybullet/cli-darwin-x64": "1.3.12",
+    "@trybullet/cli-linux-x64": "1.3.12",
+    "@trybullet/cli-linux-arm64": "1.3.12",
+    "@trybullet/cli-win32-x64": "1.3.12"
+  },
+  "engines": {
+    "node": ">=18"
+  },
+  "publishConfig": {
+    "access": "public"
+  }
+}