pi-ghost-autocomplete 0.5.0 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/README.md +513 -513
  2. package/dist/bench-cli.js +1 -1
  3. package/dist/bench.js +1 -1
  4. package/dist/index.js +1 -1
  5. package/dist/replay-cli.js +1 -1
  6. package/package.json +91 -84
package/README.md CHANGED
@@ -1,513 +1,513 @@
1
- # pi-ghost-autocomplete
2
-
3
- Inline ghost-text autocomplete for the [Pi coding agent](https://pi.dev/) (`@earendil-works/pi-coding-agent`).
4
-
5
- While you type at the Pi prompt, an LLM predicts the most likely continuation
6
- of the current line and renders it as dim grey "ghost text" right of the
7
- cursor. Press **Right Arrow** to accept. The cursor never moves until you do,
8
- the existing slash-command popup keeps **Tab**, and a slow or unreachable
9
- backend silently disables itself rather than spamming the chat.
10
-
11
- Four provider modes ship in the same package:
12
-
13
- | Mode | Backend | Default debounce | Default model |
14
- | --- | --- | --- | --- |
15
- | `cloud` | Groq / Cerebras / OpenRouter via `@earendil-works/pi-ai` | 150 ms | `groq/llama-3.1-8b-instant` |
16
- | `local` | Ollama / vLLM (OpenAI-compatible) | 400 ms | `qwen2.5-coder:1.5b` on `http://localhost:11434` |
17
- | `openai-compat` | Any OpenAI-compatible REST API (e.g. Mercury Edit) — also used as the **default** provider | 150 ms | `mercury-edit-2` on `https://api.inceptionlabs.ai/v1` |
18
- | `race` | Multiple providers in parallel; first valid response wins | 400 ms | per-member defaults |
19
-
20
- ## Getting started
21
-
22
- ### 1. Get a Mercury Edit 2 API key
23
-
24
- The default provider is **Mercury Edit 2** by Inception Labs — a diffusion-based model optimised for low-latency code completions. Sign up at [platform.inceptionlabs.ai](https://platform.inceptionlabs.ai) to get an API key.
25
-
26
- Set the `INCEPTION_API_KEY` environment variable so the extension can reach the API. Add it to your shell profile (e.g. `~/.bashrc`, `~/.zshrc`, or PowerShell `$PROFILE`) for persistence:
27
-
28
- ```bash
29
- # ~/.bashrc or ~/.zshrc
30
- export INCEPTION_API_KEY="your-key"
31
- ```
32
-
33
- ```powershell
34
- # PowerShell $PROFILE
35
- $env:INCEPTION_API_KEY = "your-key"
36
- ```
37
-
38
- Or pass it inline for a single session:
39
-
40
- ```bash
41
- INCEPTION_API_KEY="your-key" pi
42
- ```
43
-
44
- ```powershell
45
- $env:INCEPTION_API_KEY = "your-key"; pi
46
- ```
47
-
48
- To use a different provider instead (Groq, Cerebras, a local model, …), see [Configuration](#configuration).
49
-
50
- ### 2. Install the extension
51
-
52
- ```bash
53
- pi install npm:pi-ghost-autocomplete
54
- ```
55
-
56
- This registers the extension globally. Pi picks it up automatically on the next launch. To install it only for a single project, add `-l`:
57
-
58
- ```bash
59
- pi install npm:pi-ghost-autocomplete -l # writes to .pi/ in the current directory
60
- ```
61
-
62
- ### 3. Launch Pi and try it
63
-
64
- ```bash
65
- pi
66
- ```
67
-
68
- Start typing at the prompt. A dim ghost text should appear to the right of your cursor within ~150 ms. Press **Right Arrow** to accept it, or keep typing to dismiss it.
69
-
70
- > **The first session feels "cold."** The trie, cache, and profile all start
71
- > empty, so every ghost is a fresh LLM call — slower and more generic. It gets
72
- > noticeably faster and more accurate as you accept completions (the trie learns
73
- > your patterns) and the cache warms up. Give it ~15–20 min of real use before
74
- > judging quality.
75
-
76
- Run `/ghost` to confirm the extension loaded and see the active configuration:
77
-
78
- ```
79
- Pi Ghost: enabled
80
- minChars=3
81
- maxLineLength=240
82
- mode=openai-compat
83
- provider=mercury-edit
84
- baseUrl=https://api.inceptionlabs.ai/v1
85
- model=mercury-edit-2
86
- apiKey=4e24***
87
- debounceMs=150
88
- maxTokens=64
89
- maxRecentMessages=4
90
- ```
91
-
92
- ### 4. (Optional) Load the extension without installing
93
-
94
- Pass `-e` to load the package for a single session without a permanent install:
95
-
96
- ```bash
97
- pi -e npm:pi-ghost-autocomplete
98
- ```
99
-
100
- ## Install
101
-
102
- ```bash
103
- npm install pi-ghost-autocomplete
104
- ```
105
-
106
- ## Configuration
107
-
108
- All knobs are environment variables — no extra config file needed:
109
-
110
- | Variable | Default | Effect |
111
- | --- | --- | --- |
112
- | `PI_GHOST_DISABLED` | unset | Set to `1` to disable on launch |
113
- | `PI_GHOST_PROVIDER` | unset (Mercury Edit 2) | `cloud`, `local`, or `router` — set explicitly to switch away from Mercury Edit 2 (ignored when `PI_GHOST_RACE_PROVIDERS` is set). See [Register routing](#register-routing) for `router` |
114
- | `PI_GHOST_PROVIDER_NAME` | `groq` | Cloud only (`PI_GHOST_PROVIDER=cloud`): any `KnownProvider` from `pi-ai` (`groq`, `cerebras`, `openrouter`, …) |
115
- | `PI_GHOST_MODEL` | provider-specific | Model id, e.g. `llama-3.1-8b-instant`, `qwen2.5-coder:1.5b` |
116
- | `PI_GHOST_API_KEY` | env var lookup | Cloud only: explicit API key (else `pi-ai` env lookup is used) |
117
- | `PI_GHOST_BASE_URL` | `http://localhost:11434` | Local only: Ollama / vLLM base URL |
118
- | `PI_GHOST_DEBOUNCE_MS` | 150 (cloud) / 400 (local) | Inputs are debounced this long before a request is sent |
119
- | `PI_GHOST_MAX_TOKENS` | 64 (cloud) / 48 (local) | Cap on completion tokens |
120
- | `PI_GHOST_RECENT` | 4 (cloud) / 2 (local) | Number of recent user/assistant messages bundled as context |
121
- | `PI_GHOST_MIN_CHARS` | 3 | Minimum draft length before a prediction fires |
122
- | `PI_GHOST_MAX_LINE` | 240 | Predictions skipped if the current line exceeds this length |
123
- | `PI_GHOST_TEMPERATURE` | `0` | Sampling temperature [0, 2]. `0` is best-effort deterministic; see note below |
124
- | `PI_GHOST_MERCURY_PARALLEL_N` | `1` | Mercury FIM only: fire N parallel candidate completions for cycling. Requires `PI_GHOST_TEMPERATURE > 0` (downgrades silently to 1 at temp=0). Costs scale linearly with N |
125
- | `PI_GHOST_CHAT_MODEL` | `mercury-2` | Router only: chat model id for the natural-language leg (`/chat/completions`) |
126
- | `PI_GHOST_CHAT_BASE_URL` | = FIM base URL | Router only: override the chat leg's endpoint base (defaults to the same vendor as the FIM leg) |
127
- | `PI_GHOST_CHAT_API_KEY` | = Inception/FIM key | Router only: override the chat leg's API key (defaults to the FIM leg's key) |
128
- | `PI_GHOST_CHAT_MAX_TOKENS` | = `PI_GHOST_MAX_TOKENS` | Router only: cap on chat-leg completion tokens |
129
- | `PI_GHOST_CONTEXT_MAX_SCAN` | `200` | M6b: cap on the candidate pool scanned for relevance ranking |
130
- | `PI_GHOST_CONTEXT_MAX_SELECTED` | `8` | M6b: max number of selected context entries |
131
- | `PI_GHOST_CONTEXT_MAX_CHARS` | `4000` | M6b: total char budget for the selected context window |
132
- | `PI_GHOST_CONTEXT_BM25_WEIGHT` | `0.3` | M6b: weight of BM25 content score vs recency (0=pure recency, 1=pure relevance) |
133
- | `PI_GHOST_CONTEXT_TOOL_MAX` | `3` | M6c: max number of tool entries to include in selected context (0 disables tools entirely) |
134
- | `PI_GHOST_FIM_CONTEXT_ALL_INTENTS` | unset | Mercury FIM only: by default the conversation-context block is dropped from the FIM prompt for `prose`/`question` intents (a code-FIM edit model returns empty far more often when fed a large natural-language context — see session 51 investigation). Set to `1` to keep context for every intent (A/B against the empty-rate fix) |
135
- | `PI_GHOST_ALLOW_INSECURE` | unset | Set to `1` to allow `http://` provider base URLs on non-localhost hosts. Off by default — API keys + prompts would be sent in clear |
136
- | `PI_GHOST_PROFILE` | unset | Set to `off` to disable persistent profile capture + reranking. Default: enabled. Profile is local-only — see [Profile](#user-profile) below |
137
- | `PI_GHOST_PROFILE_DECAY_DAYS` | `90` | Days before an unused profile record's weight halves during compaction |
138
- | `PI_GHOST_PROVISIONAL` | unset | Set to `off` to disable the 150 ms provisional fallback. Default: enabled |
139
- | `PI_GHOST_PROVISIONAL_PREFER` | unset | Set to `1`/`true`/`on`/`yes` to bias swap decisions toward the already-displayed provisional. Raises the LLM-vs-provisional swap margin from 0.1 to 0.3, reducing perceived flicker at the cost of recall. Bench reports a `flicker=X%` rate so this knob can be evaluated |
140
- | `PI_GHOST_MIN_CONFIDENCE` | `0.55` | M3c.1: minimum reranker score for the top candidate to be shown. Range `[0, 1]`. Lower = more lenient, more ghosts shown |
141
- | `PI_GHOST_RERANK_WEIGHTS` | `0.45,0.25,0.15,0.15` | M3c: reranker weights as `logprob,profileBias,lengthPrior,agreement`. Negative values clamped to 0 |
142
- | `PI_GHOST_EVAL_CAPTURE` | unset | Set to `1` to capture committed prompts to `.pi/ghost-autocomplete/eval.jsonl` for offline replay (`pi-ghost-replay`). **Sensitive**: contains raw prompt text. Off by default |
143
- | `PI_GHOST_REGRET_WINDOW_MS` | `3000` | After accept, window in ms during which a >50% deletion of inserted text flags the record `acceptRegret: true` |
144
- | `PI_GHOST_SHORT_HOVER_MS` | `800` | Enhancement #3: a ghost dismissed within this many ms is flagged `shortHoverDismiss: true` and triggers a soft trie penalty (magnitude 0.25). Tunes the bench `short-hover` rate |
145
- | `PI_GHOST_METRICS` | unset | Set to `1` to write metrics to `.pi/ghost-autocomplete/metrics.jsonl` |
146
- | `PI_GHOST_DEBUG_LOG` | unset | Set to `1` to write debug records to `.pi/ghost-autocomplete/debug.jsonl` |
147
-
148
- ### Note on `temperature=0`
149
-
150
- `temperature=0` is the default and asks the provider for greedy decoding so
151
- the same prefix produces the same completion. This is **best-effort, not
152
- guaranteed**: vendors batch requests across users and kernel scheduling
153
- differences can produce different greedy paths from one call to the next.
154
- The session's in-memory completion cache is the real determinism guarantee
155
- — retyping the same prefix in the same context will reuse the previous
156
- ghost rather than re-querying. Raise temperature only if you want
157
- cache-miss diversity (e.g. `PI_GHOST_TEMPERATURE=0.5`).
158
-
159
- Run `/ghost` inside Pi to see the effective config; `/ghost on` and
160
- `/ghost off` toggle the feature for the current session.
161
-
162
- ## Register routing
163
-
164
- You are a developer typing a message **to** an AI coding agent — a request, an
165
- instruction, a question. A code fill-in-the-middle model (Mercury Edit), fed
166
- that message, tends to complete it like *code*: it writes the solution instead
167
- of finishing your sentence. That's the wrong register.
168
-
169
- Router mode fixes this by splitting the work between two models by **intent**:
170
-
171
- | Intent | Leg | Why |
172
- | --- | --- | --- |
173
- | `command`, `path-ref`, unclassified | **FIM** (`mercury-edit-2`, `/fim/completions`) | code/command/path drafts — what a code model is good at |
174
- | `instruction`, `question`, `prose` | **chat** (`mercury-2`, `/chat/completions`) | finishes *your message to the agent* with a persona prompt — "continue the developer's request, don't perform it" |
175
-
176
- Both legs default to the same vendor (Inception) and the same key, so a single
177
- `INCEPTION_API_KEY` is enough. If the chat leg is unreachable or unconfigured,
178
- the router degrades to FIM-only — identical to the default, so enabling it never
179
- regresses.
180
-
181
- ```bash
182
- # Single-vendor (Inception) — FIM for code, Mercury chat for instructions
183
- PI_GHOST_PROVIDER=router INCEPTION_API_KEY=your-key pi ...
184
-
185
- # Point the chat leg at a different model / endpoint / key
186
- PI_GHOST_PROVIDER=router \
187
- INCEPTION_API_KEY=your-key \
188
- PI_GHOST_CHAT_MODEL=mercury-3 pi ...
189
- ```
190
-
191
- See `PI_GHOST_CHAT_*` in the [Configuration](#configuration) table for the
192
- chat-leg overrides.
193
-
194
- If `PI_GHOST_CHAT_MODEL` is set to an id the endpoint doesn't recognise, the
195
- chat leg is preflighted at startup and Pi shows a one-shot warning naming the
196
- bad model — rather than silently falling back to the code model. In metrics,
197
- router events are tagged by the served register (`ghost-router:chat` vs
198
- `ghost-router:fim`), so `pi-ghost-bench` reports each leg's valid/bad-rate
199
- separately.
200
-
201
- ## Race mode
202
-
203
- Race mode fires requests to multiple providers simultaneously and uses the
204
- first valid response. This reduces perceived latency when one provider is slow
205
- or rate-limited.
206
-
207
- ```bash
208
- # Race Groq against Cerebras
209
- PI_GHOST_RACE_PROVIDERS=groq,cerebras pi ...
210
-
211
- # Race Groq, Cerebras, and Mercury Edit (Inception Labs)
212
- PI_GHOST_RACE_PROVIDERS=groq,cerebras,mercury-edit \
213
- INCEPTION_API_KEY=your-key pi ...
214
- ```
215
-
216
- Race-specific environment variables:
217
-
218
- | Variable | Default | Effect |
219
- | --- | --- | --- |
220
- | `PI_GHOST_RACE_PROVIDERS` | unset | Comma-separated list of providers to race (`groq`, `cerebras`, `local`, `mercury-edit`, `mercury-edit-2`, or any `KnownProvider`) |
221
- | `PI_GHOST_RACE_PAUSE_MS` | 400 | Milliseconds before the second+ provider is unblocked |
222
- | `PI_GHOST_RACE_COOLDOWN_MS` | 1500 | Quiet period after a winner before the next race starts |
223
- | `PI_GHOST_RACE_CUTOFF_MS` | 700 | Maximum time to wait for any provider before giving up |
224
- | `PI_GHOST_RACE_<NAME>_MODEL` | provider default | Override the model for a specific race member (e.g. `PI_GHOST_RACE_GROQ_MODEL`) |
225
- | `PI_GHOST_RACE_<NAME>_API_KEY` | env var lookup | Override the API key for a specific race member |
226
- | `PI_GHOST_RACE_<NAME>_BASE_URL` | provider default | Override the base URL for a specific race member |
227
- | `INCEPTION_API_KEY` | unset | API key for Mercury Edit; used when `mercury-edit` or `mercury-edit-2` appears in the race list |
228
- | `INCEPTION_BASE_URL` | `https://api.inceptionlabs.ai/v1` | Base URL override for Mercury Edit |
229
-
230
- `<NAME>` in the per-member variables is the provider name uppercased with
231
- non-alphanumeric characters replaced by `_` (e.g. `mercury-edit` →
232
- `MERCURY_EDIT`).
233
-
234
- ## Metrics and debug logging
235
-
236
- Enable structured JSONL logs for acceptance tracking and performance analysis:
237
-
238
- ```bash
239
- PI_GHOST_METRICS=1 pi ... # writes .pi/ghost-autocomplete/metrics.jsonl
240
- PI_GHOST_DEBUG_LOG=1 pi ... # writes .pi/ghost-autocomplete/debug.jsonl
241
- ```
242
-
243
- Metrics records include: timestamp, request id, provider latencies, completion
244
- mode (`deterministic` or `llm`), result (`produced`, `empty`, `rejected`,
245
- `aborted`, `error`), rejection reason, completion length, whether the ghost was
246
- shown, and acceptance outcome (`accepted`, `dismissed`, `stale`, `expired`).
247
- Raw prompt text and LLM output are never written to the metrics file. Both
248
- files rotate at 10 MB.
249
-
250
- ## User profile
251
-
252
- The extension keeps a long-lived JSONL store at
253
- `.pi/ghost-autocomplete/profile.jsonl` to learn slash-command, path, and
254
- trigram frequencies across sessions. The profile feeds three local-only
255
- signals:
256
-
257
- 1. **Slash fast-path.** When the active line starts with `/`, the editor
258
- serves the most-typed extension as a ghost before the LLM is queried
259
- (mode `trie`, provider `profile-slash`). Press Right to accept.
260
- 2. **Profile-bias rerank (M3c hook).** A `profileBias` score plugs into
261
- the multi-candidate reranker. Wired but only takes effect when the
262
- provider returns >1 candidate.
263
- 3. **Bounded path boost.** Persistent path frequency adds a small
264
- capped boost to deterministic path/fuzzy ranking, capped so it cannot
265
- beat a fresh exact path-prefix match.
266
-
267
- What's stored: paths (`src/foo.ts`), slash commands (`/review`),
268
- trigrams from committed prompts, and acceptance records keyed by the
269
- SHA-256 hash of the prefix. Raw prefixes and raw completions are
270
- **never** written. Token tuples are tokenized and capped at 4 for
271
- prefix tails / 8 for completions.
272
-
273
- Sanitization runs before persistence: messages with API keys (`sk-…`,
274
- `ghp_…`, JWT, etc.) or high-entropy tokens (Shannon > 4.0 bits/char,
275
- length ≥ 20) are dropped entirely. The check is shared with the session
276
- trie (`src/trie-sanitize.ts`).
277
-
278
- Rotation: when the JSONL exceeds 2 MB, the store compacts in place —
279
- sums weights per `(kind, key)`, halves records older than 90 days
280
- (configurable via `PI_GHOST_PROFILE_DECAY_DAYS`), drops weights below
281
- 0.1. The original file is preserved as `profile.jsonl.1`.
282
-
283
- Set `PI_GHOST_PROFILE=off` to disable capture + use entirely.
284
-
285
- ## Benchmark CLI
286
-
287
- The `pi-ghost-bench` binary reads a metrics JSONL file and reports p50/p95
288
- latency, acceptance rates, and whether PRD thresholds pass:
289
-
290
- ```bash
291
- pi-ghost-bench # reads .pi/ghost-autocomplete/metrics.jsonl
292
- pi-ghost-bench path/to/metrics.jsonl # explicit path
293
- pi-ghost-bench --json # machine-readable output
294
- ```
295
-
296
- Exit code is `0` when all thresholds pass, `1` when any fail, `2` on I/O error or when the file contains no records.
297
-
298
- ## How it works
299
-
300
- - The extension registers a `GhostEditor` via `ctx.ui.setEditorComponent`.
301
- `GhostEditor` extends Pi's `CustomEditor` so all built-in app keybindings
302
- (escape, ctrl+d, model switching, slash-command popup) keep working.
303
- - After every text change, a debounced request is sent through `pi-ai`'s
304
- `complete()` with an `AbortSignal` that is canceled on the next keystroke.
305
- - Before calling the LLM, the editor checks the path index for a
306
- deterministic match. If the current token is a file path prefix or a fuzzy
307
- filename with a strong enough score, the deterministic result is shown
308
- immediately and the LLM call is suppressed.
309
- - The path index is built asynchronously at session start via `git ls-files`
310
- from the repository root, capped at 20 000 files. High-confidence
311
- basename-prefix matches (score ≥85) preempt the LLM unconditionally; lower-
312
- confidence fuzzy matches only activate when an action word (`open`, `edit`,
313
- `read`, `show`, `view`, `find`, `check`, `look`, `load`, `import`, `include`,
314
- `see`, `run`) precedes the token.
315
- - Recent agent messages come from `ctx.sessionManager.getBranch()` — only
316
- user/assistant text, no tool calls. Both providers truncate to a small,
317
- fixed payload so cloud requests don't ship the whole file.
318
- - Path mentions are extracted from recent conversation messages (slashed
319
- paths, backticked path-like tokens, and bare filenames with known code
320
- extensions). These give a ranking boost (+10 exact, +7 basename) to
321
- deterministic path completions, so recently discussed files surface
322
- higher.
323
- - For fuzzy filename matches where the typed token is not a prefix of the
324
- match, the editor enters **replace mode**: accepting the ghost replaces
325
- the typed token with the full matched path. A `→ ` prefix is rendered
326
- before the ghost text to signal this behaviour. Prefix-suffix matches
327
- still append as usual.
328
- - `render()` searches for `CURSOR_MARKER` in the lines returned by
329
- `super.render()` and injects ANSI dim text directly after it. Pi's
330
- differential renderer redraws only the changed cells — no flicker, no
331
- full repaint.
332
- - Right Arrow is intercepted only if a ghost is shown, the popup
333
- autocomplete is **not** open, and the cursor sits at the end of the text.
334
- We use `matchesKey(data, "right")` from `@earendil-works/pi-tui`, which
335
- handles legacy CSI/SS3 sequences and the Kitty keyboard protocol uniformly,
336
- so kitty / WezTerm / iTerm2 (Kitty-mode) all work. Otherwise the input
337
- passes through to `super.handleInput`.
338
- - **Ctrl+Right** accepts the next *word* of the ghost (PRD §M3f). A "word"
339
- is a run of `\w+` characters, or a single non-word non-whitespace char
340
- with optional leading whitespace — VS Code's *Cursor Word Right* semantics.
341
- Repeated presses peel the ghost off one token at a time; the final press
342
- finalises the metric as `accepted-partial`.
343
- - **Alt+]** / **Alt+[** cycle through ranked candidates when the provider
344
- returns more than one (e.g. `openai-compat` with `PI_GHOST_MERCURY_PARALLEL_N>1`
345
- and `PI_GHOST_TEMPERATURE>0`). Single-candidate sources (cache, trie,
346
- speculative warm, deterministic path, default LLM fire) treat both keys
347
- as no-ops. The selected candidate's index is logged as `candidateRank`
348
- on accept along with `cycledCount`.
349
- - Visible-column math uses `string-width` + `strip-ansi`. Ghost text is
350
- clipped grapheme-aware to the remaining columns on the current visual line
351
- so wide CJK characters and emoji never overflow the row.
352
-
353
- ## Development
354
-
355
- ```bash
356
- npm install
357
- npm run typecheck # tsc --noEmit against pi-tui / pi-coding-agent v0.74
358
- npm test # vitest, ~970 unit tests (requires Rust: builds native crate)
359
- npm run build:dev # plain tsc -> dist/ WITH source maps (local debugging)
360
- ```
361
-
362
- The test suite and publish build delegate the crown-jewel logic (session
363
- trie, reranker, sanitize) to a native Rust binary, so **Rust is a dev-time
364
- dependency** ([install via rustup](https://rustup.rs)). End users do **not**
365
- need Rust — they receive a prebuilt, platform-matched `.node` via
366
- `optionalDependencies`.
367
-
368
- CI runs all three on Node 20 and 22 (`.github/workflows/ci.yml`).
369
-
370
- ## Source protection (publish build)
371
-
372
- This package ships **compiled + obfuscated** runtime, never raw TypeScript
373
- and never source maps. A private GitHub repo does **not** make an npm
374
- package private — anything in `files` is downloadable from the public
375
- registry — so the publish pipeline hardens the runtime against casual
376
- copying and reverse-engineering.
377
-
378
- `npm run build` (and the auto-running `prepack` hook on `npm pack` /
379
- `npm publish`) runs `scripts/build.mjs`, which:
380
-
381
- 1. builds the **native crown-jewel binary** (Rust, release, symbol-stripped)
382
- for the current platform and copies it to `dist/` as
383
- `pi-ghost-native.<triplet>.node` (skip with `PI_GHOST_SKIP_NATIVE=1`) — this
384
- is for **local dev runs only**; the main package tarball excludes it (see the
385
- `!dist/**/*.node` negation in `package.json` "files"), since end users get
386
- the binary from a per-platform optionalDependency, never bundled in `main`;
387
- 2. emits `.d.ts` declarations only (`tsconfig.build.json`) — no `.js`, no
388
- `.js.map`, no `.d.ts.map`;
389
- 3. **prunes internal declarations** — keeps only the `.d.ts` transitively
390
- reachable from the public entries (`index`, `bench`) and deletes the
391
- rest. Crown-jewel modules (`fast-path-engine`, `session-trie`, `reranker`,
392
- `provisional`, `regret-tracker`, `profile`, `edit-history-ring`, …) are
393
- internal and never re-exported, so their type declarations are removed
394
- from the tarball — they would otherwise be a symbol-level roadmap into
395
- the obfuscated bundle. A dangling-reference guard fails the build if
396
- pruning would break consumer type resolution;
397
- 4. bundles every entry point with **esbuild** into a single minified ESM
398
- chunk per entry (collapsing ~40 source modules into opaque bundles),
399
- keeping `@earendil-works/*` peers + `string-width`/`strip-ansi` external;
400
- 5. obfuscates each bundle with **javascript-obfuscator**;
401
- 6. re-adds exactly one `#!/usr/bin/env node` shebang to the two `bin` CLIs;
402
- 7. fails loudly if any `*.map` reaches `dist/`.
403
-
404
- Obfuscation is tuned to be **latency-safe**: this is a ghost-text engine
405
- where milliseconds matter, so control-flow flattening (the one transform
406
- with real runtime cost) is OFF. What's ON: hexadecimal identifier renaming,
407
- string-array encoding (base64) with two wrapper layers and hexadecimal
408
- indexes, string splitting (5-char chunks), numeric-literal-to-expression
409
- conversion (V8 constant-folds these at parse time, so zero runtime cost),
410
- object-key transformation, and light dead-code injection. Public export
411
- names are preserved so the pi extension loader and TS consumers keep working.
412
-
413
- Tune the trade-off in `OBFUSCATOR_OPTIONS` inside `scripts/build.mjs`:
414
-
415
- - **Smaller tarball / faster parse** → lower `deadCodeInjectionThreshold`
416
- (e.g. `0`) and `stringArrayThreshold` (e.g. `0.4`). `index.js` is ~575 kB
417
- with the hardened settings; dropping dead-code injection brings it back
418
- near the ~110 kB pre-obfuscation size.
419
- - **Stronger protection** (test thoroughly first) → enable
420
- `controlFlowFlattening`, but expect a measurable latency hit on the
421
- autocomplete fast path.
422
-
423
- What still ships readable: the `.d.ts` declarations for the **public API
424
- only** (internal module declarations are pruned in step 2), plus the
425
- JS wiring that calls into the native binary.
426
-
427
- ### Native crown jewels (tier-C hardening)
428
-
429
- The tuned algorithm values — the reranker weights (`0.40/0.25/0.15/0.10/0.10`),
430
- the prefix-aware `lengthPrior` curve, the confidence gates (`0.55/0.05`),
431
- the session-trie lookup gates + typeahead + decay, and the secret/volatile
432
- regex patterns — live **only** in a prebuilt, symbol-stripped native
433
- `.node` binary (`native/`, Rust + napi-rs). No tuned literal appears in the
434
- shipped JS: the wrapper fetches constants from the binary at load time, and
435
- the JS `trie-sanitize.ts` / `session-trie.ts` / `reranker.ts` are thin
436
- delegates. Feature extraction (merging, profile/edit-proximity closures)
437
- stays in JS because it needs JS closures; the scoring math crosses the
438
- boundary.
439
-
440
- A reverser with the obfuscated bundle alone sees only *calls* into a native
441
- binary, not the weights or the curves **as static literals**. Reading them out
442
- of the binary requires disassembling the stripped Rust (Ghidra/IDA over
443
- hours-to-days), not pasting into `obfuscator-io-deobfuscator` or asking an LLM.
444
- This raises the cost of *static* recovery versus obfuscation alone — but see
445
- the honest residual below: the scoring is recoverable *dynamically* regardless
446
- of where the constants live.
447
-
448
- **Distribution.** The binary ships per-platform as `optionalDependencies`
449
- (`pi-ghost-autocomplete-<os>-<cpu>[-libc>`), following the @napi-rs
450
- convention. End users on supported platforms need **no Rust toolchain** —
451
- npm installs only the matching prebuild. Unsupported platforms fail at load
452
- (by design: there is no recoverable JS fallback for the crown jewels). The
453
- trade-off is a per-platform CI/release matrix (`.github/workflows/release.yml`).
454
-
455
- **Honest residual.** This is not cryptographic protection, and two of the
456
- "hidden" pieces are weaker than the binary boundary implies:
457
-
458
- - **The reranker scoring is linear and black-box recoverable.** The score is a
459
- weighted sum of per-axis normalized features, and the public API returns the
460
- per-axis `scoreBreakdown` alongside the final `score`. Feeding ~5+ candidate
461
- sets and solving the resulting linear system recovers the weight vector and
462
- gate thresholds **without ever touching the binary** — no disassembly needed.
463
- Hiding the literals raises the bar against copy-paste, not against a
464
- competitor who probes the I/O.
465
- - **The secret/volatile regexes are public patterns.** `sk-ant`, `ghp_`,
466
- `AKIA`, `AIza`, JWT shape, etc. are the same prefixes published by gitleaks /
467
- trufflehog. Moving them into the binary hides nothing of value; it's done for
468
- consistency, not secrecy.
469
-
470
- What genuinely benefits from the boundary is the **session-trie lookup
471
- algorithm shape** (gates, typeahead-delta, decay, volatile verdict) — that
472
- structure is not trivially recoverable from I/O. Runtime instrumentation
473
- (`--inspect`, `LD_PRELOAD`, `process.dlopen` hooks) can still observe
474
- inputs/outputs of everything. What tier-C removes is the *automated,
475
- minutes-to-recover static* attack surface that pure JS obfuscation leaves. The
476
- real enforcement remains the **SSPL-1.0** license; the native boundary removes
477
- the temptation for the honest-but-curious and raises the cost for a casual
478
- competitor from "paste into a deobfuscator" to "learn Rust disassembly (or
479
- write a solver)."
480
-
481
- > **Platform support.** Prebuilt binaries ship for win32-x64-msvc,
482
- > darwin-arm64, linux-x64-gnu, linux-arm64-gnu. **Intel Mac (darwin-x64)
483
- > is not supported** — Apple stopped selling Intel Macs in 2020; Intel Mac
484
- > users can run via Rosetta 2. **musl (Alpine) is not supported** — there is
485
- > no JS fallback, so unsupported platforms fail at load by design.
486
-
487
- Verify what a release contains at any time:
488
-
489
- ```bash
490
- npm pack --dry-run
491
- # expect: obfuscated dist/*.js + public dist/*.d.ts + README only.
492
- # must show NO *.map, NO raw *.ts source (note: *.d.ts declarations are fine
493
- # and expected), and NO *.node (the binary ships via optionalDependencies).
494
- ```
495
-
496
- ## Limitations
497
-
498
- - No native Ollama provider in `pi-ai`; we configure an `openai-compat`
499
- model pointed at Ollama's `/v1` endpoint. Any OpenAI-compatible local
500
- server works the same way.
501
- - Pin to `^0.74` of `pi-tui` / `pi-coding-agent` / `pi-ai` /
502
- `pi-agent-core` (all under `@earendil-works`). The relevant interfaces are
503
- pre-1.0; expect occasional follow-up bumps.
504
- - Predictions are shown only at the end of the buffer to avoid splitting
505
- lines mid-stream.
506
- - The path index covers only tracked Git files. Untracked files are not
507
- indexed in v1.
508
- - Three-provider racing may create elevated cost or rate-limit pressure at
509
- high typing speeds.
510
-
511
- ## License
512
-
513
- SSPL-1.0 (Server Side Public License v1) — see [LICENSE](./LICENSE).
1
+ # pi-ghost-autocomplete
2
+
3
+ Inline ghost-text autocomplete for the [Pi coding agent](https://pi.dev/) (`@earendil-works/pi-coding-agent`).
4
+
5
+ While you type at the Pi prompt, an LLM predicts the most likely continuation
6
+ of the current line and renders it as dim grey "ghost text" right of the
7
+ cursor. Press **Right Arrow** to accept. The cursor never moves until you do,
8
+ the existing slash-command popup keeps **Tab**, and a slow or unreachable
9
+ backend silently disables itself rather than spamming the chat.
10
+
11
+ Four provider modes ship in the same package:
12
+
13
+ | Mode | Backend | Default debounce | Default model |
14
+ | --- | --- | --- | --- |
15
+ | `cloud` | Groq / Cerebras / OpenRouter via `@earendil-works/pi-ai` | 150 ms | `groq/llama-3.1-8b-instant` |
16
+ | `local` | Ollama / vLLM (OpenAI-compatible) | 400 ms | `qwen2.5-coder:1.5b` on `http://localhost:11434` |
17
+ | `openai-compat` | Any OpenAI-compatible REST API (e.g. Mercury Edit) — also used as the **default** provider | 150 ms | `mercury-edit-2` on `https://api.inceptionlabs.ai/v1` |
18
+ | `race` | Multiple providers in parallel; first valid response wins | 400 ms | per-member defaults |
19
+
20
+ ## Getting started
21
+
22
+ ### 1. Get a Mercury Edit 2 API key
23
+
24
+ The default provider is **Mercury Edit 2** by Inception Labs — a diffusion-based model optimised for low-latency code completions. Sign up at [platform.inceptionlabs.ai](https://platform.inceptionlabs.ai) to get an API key.
25
+
26
+ Set the `INCEPTION_API_KEY` environment variable so the extension can reach the API. Add it to your shell profile (e.g. `~/.bashrc`, `~/.zshrc`, or PowerShell `$PROFILE`) for persistence:
27
+
28
+ ```bash
29
+ # ~/.bashrc or ~/.zshrc
30
+ export INCEPTION_API_KEY="your-key"
31
+ ```
32
+
33
+ ```powershell
34
+ # PowerShell $PROFILE
35
+ $env:INCEPTION_API_KEY = "your-key"
36
+ ```
37
+
38
+ Or pass it inline for a single session:
39
+
40
+ ```bash
41
+ INCEPTION_API_KEY="your-key" pi
42
+ ```
43
+
44
+ ```powershell
45
+ $env:INCEPTION_API_KEY = "your-key"; pi
46
+ ```
47
+
48
+ To use a different provider instead (Groq, Cerebras, a local model, …), see [Configuration](#configuration).
49
+
50
+ ### 2. Install the extension
51
+
52
+ ```bash
53
+ pi install npm:pi-ghost-autocomplete
54
+ ```
55
+
56
+ This registers the extension globally. Pi picks it up automatically on the next launch. To install it only for a single project, add `-l`:
57
+
58
+ ```bash
59
+ pi install npm:pi-ghost-autocomplete -l # writes to .pi/ in the current directory
60
+ ```
61
+
62
+ ### 3. Launch Pi and try it
63
+
64
+ ```bash
65
+ pi
66
+ ```
67
+
68
+ Start typing at the prompt. A dim ghost text should appear to the right of your cursor within ~150 ms. Press **Right Arrow** to accept it, or keep typing to dismiss it.
69
+
70
+ > **The first session feels "cold."** The trie, cache, and profile all start
71
+ > empty, so every ghost is a fresh LLM call — slower and more generic. It gets
72
+ > noticeably faster and more accurate as you accept completions (the trie learns
73
+ > your patterns) and the cache warms up. Give it ~15–20 min of real use before
74
+ > judging quality.
75
+
76
+ Run `/ghost` to confirm the extension loaded and see the active configuration:
77
+
78
+ ```
79
+ Pi Ghost: enabled
80
+ minChars=3
81
+ maxLineLength=240
82
+ mode=openai-compat
83
+ provider=mercury-edit
84
+ baseUrl=https://api.inceptionlabs.ai/v1
85
+ model=mercury-edit-2
86
+ apiKey=4e24***
87
+ debounceMs=150
88
+ maxTokens=64
89
+ maxRecentMessages=4
90
+ ```
91
+
92
+ ### 4. (Optional) Load the extension without installing
93
+
94
+ Pass `-e` to load the package for a single session without a permanent install:
95
+
96
+ ```bash
97
+ pi -e npm:pi-ghost-autocomplete
98
+ ```
99
+
100
+ ## Install
101
+
102
+ ```bash
103
+ npm install pi-ghost-autocomplete
104
+ ```
105
+
106
+ ## Configuration
107
+
108
+ All knobs are environment variables — no extra config file needed:
109
+
110
+ | Variable | Default | Effect |
111
+ | --- | --- | --- |
112
+ | `PI_GHOST_DISABLED` | unset | Set to `1` to disable on launch |
113
+ | `PI_GHOST_PROVIDER` | unset (Mercury Edit 2) | `cloud`, `local`, or `router` — set explicitly to switch away from Mercury Edit 2 (ignored when `PI_GHOST_RACE_PROVIDERS` is set). See [Register routing](#register-routing) for `router` |
114
+ | `PI_GHOST_PROVIDER_NAME` | `groq` | Cloud only (`PI_GHOST_PROVIDER=cloud`): any `KnownProvider` from `pi-ai` (`groq`, `cerebras`, `openrouter`, …) |
115
+ | `PI_GHOST_MODEL` | provider-specific | Model id, e.g. `llama-3.1-8b-instant`, `qwen2.5-coder:1.5b` |
116
+ | `PI_GHOST_API_KEY` | env var lookup | Cloud only: explicit API key (else `pi-ai` env lookup is used) |
117
+ | `PI_GHOST_BASE_URL` | `http://localhost:11434` | Local only: Ollama / vLLM base URL |
118
+ | `PI_GHOST_DEBOUNCE_MS` | 150 (cloud) / 400 (local) | Inputs are debounced this long before a request is sent |
119
+ | `PI_GHOST_MAX_TOKENS` | 64 (cloud) / 48 (local) | Cap on completion tokens |
120
+ | `PI_GHOST_RECENT` | 4 (cloud) / 2 (local) | Number of recent user/assistant messages bundled as context |
121
+ | `PI_GHOST_MIN_CHARS` | 3 | Minimum draft length before a prediction fires |
122
+ | `PI_GHOST_MAX_LINE` | 240 | Predictions skipped if the current line exceeds this length |
123
+ | `PI_GHOST_TEMPERATURE` | `0` | Sampling temperature [0, 2]. `0` is best-effort deterministic; see note below |
124
+ | `PI_GHOST_MERCURY_PARALLEL_N` | `1` | Mercury FIM only: fire N parallel candidate completions for cycling. Requires `PI_GHOST_TEMPERATURE > 0` (downgrades silently to 1 at temp=0). Costs scale linearly with N |
125
+ | `PI_GHOST_CHAT_MODEL` | `mercury-2` | Router only: chat model id for the natural-language leg (`/chat/completions`) |
126
+ | `PI_GHOST_CHAT_BASE_URL` | = FIM base URL | Router only: override the chat leg's endpoint base (defaults to the same vendor as the FIM leg) |
127
+ | `PI_GHOST_CHAT_API_KEY` | = Inception/FIM key | Router only: override the chat leg's API key (defaults to the FIM leg's key) |
128
+ | `PI_GHOST_CHAT_MAX_TOKENS` | = `PI_GHOST_MAX_TOKENS` | Router only: cap on chat-leg completion tokens |
129
+ | `PI_GHOST_CONTEXT_MAX_SCAN` | `200` | M6b: cap on the candidate pool scanned for relevance ranking |
130
+ | `PI_GHOST_CONTEXT_MAX_SELECTED` | `8` | M6b: max number of selected context entries |
131
+ | `PI_GHOST_CONTEXT_MAX_CHARS` | `4000` | M6b: total char budget for the selected context window |
132
+ | `PI_GHOST_CONTEXT_BM25_WEIGHT` | `0.3` | M6b: weight of BM25 content score vs recency (0=pure recency, 1=pure relevance) |
133
+ | `PI_GHOST_CONTEXT_TOOL_MAX` | `3` | M6c: max number of tool entries to include in selected context (0 disables tools entirely) |
134
+ | `PI_GHOST_FIM_CONTEXT_ALL_INTENTS` | unset | Mercury FIM only: by default the conversation-context block is dropped from the FIM prompt for `prose`/`question` intents (a code-FIM edit model returns empty far more often when fed a large natural-language context — see session 51 investigation). Set to `1` to keep context for every intent (A/B against the empty-rate fix) |
135
+ | `PI_GHOST_ALLOW_INSECURE` | unset | Set to `1` to allow `http://` provider base URLs on non-localhost hosts. Off by default — API keys + prompts would be sent in clear |
136
+ | `PI_GHOST_PROFILE` | unset | Set to `off` to disable persistent profile capture + reranking. Default: enabled. Profile is local-only — see [Profile](#user-profile) below |
137
+ | `PI_GHOST_PROFILE_DECAY_DAYS` | `90` | Days before an unused profile record's weight halves during compaction |
138
+ | `PI_GHOST_PROVISIONAL` | unset | Set to `off` to disable the 150 ms provisional fallback. Default: enabled |
139
+ | `PI_GHOST_PROVISIONAL_PREFER` | unset | Set to `1`/`true`/`on`/`yes` to bias swap decisions toward the already-displayed provisional. Raises the LLM-vs-provisional swap margin from 0.1 to 0.3, reducing perceived flicker at the cost of recall. Bench reports a `flicker=X%` rate so this knob can be evaluated |
140
+ | `PI_GHOST_MIN_CONFIDENCE` | `0.55` | M3c.1: minimum reranker score for the top candidate to be shown. Range `[0, 1]`. Lower = more lenient, more ghosts shown |
141
+ | `PI_GHOST_RERANK_WEIGHTS` | `0.45,0.25,0.15,0.15` | M3c: reranker weights as `logprob,profileBias,lengthPrior,agreement`. Negative values clamped to 0 |
142
+ | `PI_GHOST_EVAL_CAPTURE` | unset | Set to `1` to capture committed prompts to `.pi/ghost-autocomplete/eval.jsonl` for offline replay (`pi-ghost-replay`). **Sensitive**: contains raw prompt text. Off by default |
143
+ | `PI_GHOST_REGRET_WINDOW_MS` | `3000` | After accept, window in ms during which a >50% deletion of inserted text flags the record `acceptRegret: true` |
144
+ | `PI_GHOST_SHORT_HOVER_MS` | `800` | Enhancement #3: a ghost dismissed within this many ms is flagged `shortHoverDismiss: true` and triggers a soft trie penalty (magnitude 0.25). Tunes the bench `short-hover` rate |
145
+ | `PI_GHOST_METRICS` | unset | Set to `1` to write metrics to `.pi/ghost-autocomplete/metrics.jsonl` |
146
+ | `PI_GHOST_DEBUG_LOG` | unset | Set to `1` to write debug records to `.pi/ghost-autocomplete/debug.jsonl` |
147
+
148
+ ### Note on `temperature=0`
149
+
150
+ `temperature=0` is the default and asks the provider for greedy decoding so
151
+ the same prefix produces the same completion. This is **best-effort, not
152
+ guaranteed**: vendors batch requests across users and kernel scheduling
153
+ differences can produce different greedy paths from one call to the next.
154
+ The session's in-memory completion cache is the real determinism guarantee
155
+ — retyping the same prefix in the same context will reuse the previous
156
+ ghost rather than re-querying. Raise temperature only if you want
157
+ cache-miss diversity (e.g. `PI_GHOST_TEMPERATURE=0.5`).
158
+
159
+ Run `/ghost` inside Pi to see the effective config; `/ghost on` and
160
+ `/ghost off` toggle the feature for the current session.
161
+
162
+ ## Register routing
163
+
164
+ You are a developer typing a message **to** an AI coding agent — a request, an
165
+ instruction, a question. A code fill-in-the-middle model (Mercury Edit), fed
166
+ that message, tends to complete it like *code*: it writes the solution instead
167
+ of finishing your sentence. That's the wrong register.
168
+
169
+ Router mode fixes this by splitting the work between two models by **intent**:
170
+
171
+ | Intent | Leg | Why |
172
+ | --- | --- | --- |
173
+ | `command`, `path-ref`, unclassified | **FIM** (`mercury-edit-2`, `/fim/completions`) | code/command/path drafts — what a code model is good at |
174
+ | `instruction`, `question`, `prose` | **chat** (`mercury-2`, `/chat/completions`) | finishes *your message to the agent* with a persona prompt — "continue the developer's request, don't perform it" |
175
+
176
+ Both legs default to the same vendor (Inception) and the same key, so a single
177
+ `INCEPTION_API_KEY` is enough. If the chat leg is unreachable or unconfigured,
178
+ the router degrades to FIM-only — identical to the default, so enabling it never
179
+ regresses.
180
+
181
+ ```bash
182
+ # Single-vendor (Inception) — FIM for code, Mercury chat for instructions
183
+ PI_GHOST_PROVIDER=router INCEPTION_API_KEY=your-key pi ...
184
+
185
+ # Point the chat leg at a different model / endpoint / key
186
+ PI_GHOST_PROVIDER=router \
187
+ INCEPTION_API_KEY=your-key \
188
+ PI_GHOST_CHAT_MODEL=mercury-3 pi ...
189
+ ```
190
+
191
+ See `PI_GHOST_CHAT_*` in the [Configuration](#configuration) table for the
192
+ chat-leg overrides.
193
+
194
+ If `PI_GHOST_CHAT_MODEL` is set to an id the endpoint doesn't recognise, the
195
+ chat leg is preflighted at startup and Pi shows a one-shot warning naming the
196
+ bad model — rather than silently falling back to the code model. In metrics,
197
+ router events are tagged by the served register (`ghost-router:chat` vs
198
+ `ghost-router:fim`), so `pi-ghost-bench` reports each leg's valid/bad-rate
199
+ separately.
200
+
201
+ ## Race mode
202
+
203
+ Race mode fires requests to multiple providers simultaneously and uses the
204
+ first valid response. This reduces perceived latency when one provider is slow
205
+ or rate-limited.
206
+
207
+ ```bash
208
+ # Race Groq against Cerebras
209
+ PI_GHOST_RACE_PROVIDERS=groq,cerebras pi ...
210
+
211
+ # Race Groq, Cerebras, and Mercury Edit (Inception Labs)
212
+ PI_GHOST_RACE_PROVIDERS=groq,cerebras,mercury-edit \
213
+ INCEPTION_API_KEY=your-key pi ...
214
+ ```
215
+
216
+ Race-specific environment variables:
217
+
218
+ | Variable | Default | Effect |
219
+ | --- | --- | --- |
220
+ | `PI_GHOST_RACE_PROVIDERS` | unset | Comma-separated list of providers to race (`groq`, `cerebras`, `local`, `mercury-edit`, `mercury-edit-2`, or any `KnownProvider`) |
221
+ | `PI_GHOST_RACE_PAUSE_MS` | 400 | Milliseconds before the second+ provider is unblocked |
222
+ | `PI_GHOST_RACE_COOLDOWN_MS` | 1500 | Quiet period after a winner before the next race starts |
223
+ | `PI_GHOST_RACE_CUTOFF_MS` | 700 | Maximum time to wait for any provider before giving up |
224
+ | `PI_GHOST_RACE_<NAME>_MODEL` | provider default | Override the model for a specific race member (e.g. `PI_GHOST_RACE_GROQ_MODEL`) |
225
+ | `PI_GHOST_RACE_<NAME>_API_KEY` | env var lookup | Override the API key for a specific race member |
226
+ | `PI_GHOST_RACE_<NAME>_BASE_URL` | provider default | Override the base URL for a specific race member |
227
+ | `INCEPTION_API_KEY` | unset | API key for Mercury Edit; used when `mercury-edit` or `mercury-edit-2` appears in the race list |
228
+ | `INCEPTION_BASE_URL` | `https://api.inceptionlabs.ai/v1` | Base URL override for Mercury Edit |
229
+
230
+ `<NAME>` in the per-member variables is the provider name uppercased with
231
+ non-alphanumeric characters replaced by `_` (e.g. `mercury-edit` →
232
+ `MERCURY_EDIT`).
233
+
234
+ ## Metrics and debug logging
235
+
236
+ Enable structured JSONL logs for acceptance tracking and performance analysis:
237
+
238
+ ```bash
239
+ PI_GHOST_METRICS=1 pi ... # writes .pi/ghost-autocomplete/metrics.jsonl
240
+ PI_GHOST_DEBUG_LOG=1 pi ... # writes .pi/ghost-autocomplete/debug.jsonl
241
+ ```
242
+
243
+ Metrics records include: timestamp, request id, provider latencies, completion
244
+ mode (`deterministic` or `llm`), result (`produced`, `empty`, `rejected`,
245
+ `aborted`, `error`), rejection reason, completion length, whether the ghost was
246
+ shown, and acceptance outcome (`accepted`, `dismissed`, `stale`, `expired`).
247
+ Raw prompt text and LLM output are never written to the metrics file. Both
248
+ files rotate at 10 MB.
249
+
250
+ ## User profile
251
+
252
+ The extension keeps a long-lived JSONL store at
253
+ `.pi/ghost-autocomplete/profile.jsonl` to learn slash-command, path, and
254
+ trigram frequencies across sessions. The profile feeds three local-only
255
+ signals:
256
+
257
+ 1. **Slash fast-path.** When the active line starts with `/`, the editor
258
+ serves the most-typed extension as a ghost before the LLM is queried
259
+ (mode `trie`, provider `profile-slash`). Press Right to accept.
260
+ 2. **Profile-bias rerank (M3c hook).** A `profileBias` score plugs into
261
+ the multi-candidate reranker. Wired but only takes effect when the
262
+ provider returns >1 candidate.
263
+ 3. **Bounded path boost.** Persistent path frequency adds a small
264
+ capped boost to deterministic path/fuzzy ranking, capped so it cannot
265
+ beat a fresh exact path-prefix match.
266
+
267
+ What's stored: paths (`src/foo.ts`), slash commands (`/review`),
268
+ trigrams from committed prompts, and acceptance records keyed by the
269
+ SHA-256 hash of the prefix. Raw prefixes and raw completions are
270
+ **never** written. Token tuples are tokenized and capped at 4 for
271
+ prefix tails / 8 for completions.
272
+
273
+ Sanitization runs before persistence: messages with API keys (`sk-…`,
274
+ `ghp_…`, JWT, etc.) or high-entropy tokens (Shannon > 4.0 bits/char,
275
+ length ≥ 20) are dropped entirely. The check is shared with the session
276
+ trie (`src/trie-sanitize.ts`).
277
+
278
+ Rotation: when the JSONL exceeds 2 MB, the store compacts in place —
279
+ sums weights per `(kind, key)`, halves records older than 90 days
280
+ (configurable via `PI_GHOST_PROFILE_DECAY_DAYS`), drops weights below
281
+ 0.1. The original file is preserved as `profile.jsonl.1`.
282
+
283
+ Set `PI_GHOST_PROFILE=off` to disable capture + use entirely.
284
+
285
+ ## Benchmark CLI
286
+
287
+ The `pi-ghost-bench` binary reads a metrics JSONL file and reports p50/p95
288
+ latency, acceptance rates, and whether PRD thresholds pass:
289
+
290
+ ```bash
291
+ pi-ghost-bench # reads .pi/ghost-autocomplete/metrics.jsonl
292
+ pi-ghost-bench path/to/metrics.jsonl # explicit path
293
+ pi-ghost-bench --json # machine-readable output
294
+ ```
295
+
296
+ Exit code is `0` when all thresholds pass, `1` when any fail, `2` on I/O error or when the file contains no records.
297
+
298
+ ## How it works
299
+
300
+ - The extension registers a `GhostEditor` via `ctx.ui.setEditorComponent`.
301
+ `GhostEditor` extends Pi's `CustomEditor` so all built-in app keybindings
302
+ (escape, ctrl+d, model switching, slash-command popup) keep working.
303
+ - After every text change, a debounced request is sent through `pi-ai`'s
304
+ `complete()` with an `AbortSignal` that is canceled on the next keystroke.
305
+ - Before calling the LLM, the editor checks the path index for a
306
+ deterministic match. If the current token is a file path prefix or a fuzzy
307
+ filename with a strong enough score, the deterministic result is shown
308
+ immediately and the LLM call is suppressed.
309
+ - The path index is built asynchronously at session start via `git ls-files`
310
+ from the repository root, capped at 20 000 files. High-confidence
311
+ basename-prefix matches (score ≥85) preempt the LLM unconditionally; lower-
312
+ confidence fuzzy matches only activate when an action word (`open`, `edit`,
313
+ `read`, `show`, `view`, `find`, `check`, `look`, `load`, `import`, `include`,
314
+ `see`, `run`) precedes the token.
315
+ - Recent agent messages come from `ctx.sessionManager.getBranch()` — only
316
+ user/assistant text, no tool calls. Both providers truncate to a small,
317
+ fixed payload so cloud requests don't ship the whole file.
318
+ - Path mentions are extracted from recent conversation messages (slashed
319
+ paths, backticked path-like tokens, and bare filenames with known code
320
+ extensions). These give a ranking boost (+10 exact, +7 basename) to
321
+ deterministic path completions, so recently discussed files surface
322
+ higher.
323
+ - For fuzzy filename matches where the typed token is not a prefix of the
324
+ match, the editor enters **replace mode**: accepting the ghost replaces
325
+ the typed token with the full matched path. A `→ ` prefix is rendered
326
+ before the ghost text to signal this behaviour. Prefix-suffix matches
327
+ still append as usual.
328
+ - `render()` searches for `CURSOR_MARKER` in the lines returned by
329
+ `super.render()` and injects ANSI dim text directly after it. Pi's
330
+ differential renderer redraws only the changed cells — no flicker, no
331
+ full repaint.
332
+ - Right Arrow is intercepted only if a ghost is shown, the popup
333
+ autocomplete is **not** open, and the cursor sits at the end of the text.
334
+ We use `matchesKey(data, "right")` from `@earendil-works/pi-tui`, which
335
+ handles legacy CSI/SS3 sequences and the Kitty keyboard protocol uniformly,
336
+ so kitty / WezTerm / iTerm2 (Kitty-mode) all work. Otherwise the input
337
+ passes through to `super.handleInput`.
338
+ - **Ctrl+Right** accepts the next *word* of the ghost (PRD §M3f). A "word"
339
+ is a run of `\w+` characters, or a single non-word non-whitespace char
340
+ with optional leading whitespace — VS Code's *Cursor Word Right* semantics.
341
+ Repeated presses peel the ghost off one token at a time; the final press
342
+ finalises the metric as `accepted-partial`.
343
+ - **Alt+]** / **Alt+[** cycle through ranked candidates when the provider
344
+ returns more than one (e.g. `openai-compat` with `PI_GHOST_MERCURY_PARALLEL_N>1`
345
+ and `PI_GHOST_TEMPERATURE>0`). Single-candidate sources (cache, trie,
346
+ speculative warm, deterministic path, default LLM fire) treat both keys
347
+ as no-ops. The selected candidate's index is logged as `candidateRank`
348
+ on accept along with `cycledCount`.
349
+ - Visible-column math uses `string-width` + `strip-ansi`. Ghost text is
350
+ clipped grapheme-aware to the remaining columns on the current visual line
351
+ so wide CJK characters and emoji never overflow the row.
352
+
353
+ ## Development
354
+
355
+ ```bash
356
+ npm install
357
+ npm run typecheck # tsc --noEmit against pi-tui / pi-coding-agent v0.74
358
+ npm test # vitest, ~970 unit tests (requires Rust: builds native crate)
359
+ npm run build:dev # plain tsc -> dist/ WITH source maps (local debugging)
360
+ ```
361
+
362
+ The test suite and publish build delegate the crown-jewel logic (session
363
+ trie, reranker, sanitize) to a native Rust binary, so **Rust is a dev-time
364
+ dependency** ([install via rustup](https://rustup.rs)). End users do **not**
365
+ need Rust — they receive a prebuilt, platform-matched `.node` via
366
+ `optionalDependencies`.
367
+
368
+ CI runs all three on Node 20 and 22 (`.github/workflows/ci.yml`).
369
+
370
+ ## Source protection (publish build)
371
+
372
+ This package ships **compiled + obfuscated** runtime, never raw TypeScript
373
+ and never source maps. A private GitHub repo does **not** make an npm
374
+ package private — anything in `files` is downloadable from the public
375
+ registry — so the publish pipeline hardens the runtime against casual
376
+ copying and reverse-engineering.
377
+
378
+ `npm run build` (and the auto-running `prepack` hook on `npm pack` /
379
+ `npm publish`) runs `scripts/build.mjs`, which:
380
+
381
+ 1. builds the **native crown-jewel binary** (Rust, release, symbol-stripped)
382
+ for the current platform and copies it to `dist/` as
383
+ `pi-ghost-native.<triplet>.node` (skip with `PI_GHOST_SKIP_NATIVE=1`) — this
384
+ is for **local dev runs only**; the main package tarball excludes it (see the
385
+ `!dist/**/*.node` negation in `package.json` "files"), since end users get
386
+ the binary from a per-platform optionalDependency, never bundled in `main`;
387
+ 2. emits `.d.ts` declarations only (`tsconfig.build.json`) — no `.js`, no
388
+ `.js.map`, no `.d.ts.map`;
389
+ 3. **prunes internal declarations** — keeps only the `.d.ts` transitively
390
+ reachable from the public entries (`index`, `bench`) and deletes the
391
+ rest. Crown-jewel modules (`fast-path-engine`, `session-trie`, `reranker`,
392
+ `provisional`, `regret-tracker`, `profile`, `edit-history-ring`, …) are
393
+ internal and never re-exported, so their type declarations are removed
394
+ from the tarball — they would otherwise be a symbol-level roadmap into
395
+ the obfuscated bundle. A dangling-reference guard fails the build if
396
+ pruning would break consumer type resolution;
397
+ 4. bundles every entry point with **esbuild** into a single minified ESM
398
+ chunk per entry (collapsing ~40 source modules into opaque bundles),
399
+ keeping `@earendil-works/*` peers + `string-width`/`strip-ansi` external;
400
+ 5. obfuscates each bundle with **javascript-obfuscator**;
401
+ 6. re-adds exactly one `#!/usr/bin/env node` shebang to the two `bin` CLIs;
402
+ 7. fails loudly if any `*.map` reaches `dist/`.
403
+
404
+ Obfuscation is tuned to be **latency-safe**: this is a ghost-text engine
405
+ where milliseconds matter, so control-flow flattening (the one transform
406
+ with real runtime cost) is OFF. What's ON: hexadecimal identifier renaming,
407
+ string-array encoding (base64) with two wrapper layers and hexadecimal
408
+ indexes, string splitting (5-char chunks), numeric-literal-to-expression
409
+ conversion (V8 constant-folds these at parse time, so zero runtime cost),
410
+ object-key transformation, and light dead-code injection. Public export
411
+ names are preserved so the pi extension loader and TS consumers keep working.
412
+
413
+ Tune the trade-off in `OBFUSCATOR_OPTIONS` inside `scripts/build.mjs`:
414
+
415
+ - **Smaller tarball / faster parse** → lower `deadCodeInjectionThreshold`
416
+ (e.g. `0`) and `stringArrayThreshold` (e.g. `0.4`). `index.js` is ~575 kB
417
+ with the hardened settings; dropping dead-code injection brings it back
418
+ near the ~110 kB pre-obfuscation size.
419
+ - **Stronger protection** (test thoroughly first) → enable
420
+ `controlFlowFlattening`, but expect a measurable latency hit on the
421
+ autocomplete fast path.
422
+
423
+ What still ships readable: the `.d.ts` declarations for the **public API
424
+ only** (internal module declarations are pruned in step 2), plus the
425
+ JS wiring that calls into the native binary.
426
+
427
+ ### Native crown jewels (tier-C hardening)
428
+
429
+ The tuned algorithm values — the reranker weights (`0.40/0.25/0.15/0.10/0.10`),
430
+ the prefix-aware `lengthPrior` curve, the confidence gates (`0.55/0.05`),
431
+ the session-trie lookup gates + typeahead + decay, and the secret/volatile
432
+ regex patterns — live **only** in a prebuilt, symbol-stripped native
433
+ `.node` binary (`native/`, Rust + napi-rs). No tuned literal appears in the
434
+ shipped JS: the wrapper fetches constants from the binary at load time, and
435
+ the JS `trie-sanitize.ts` / `session-trie.ts` / `reranker.ts` are thin
436
+ delegates. Feature extraction (merging, profile/edit-proximity closures)
437
+ stays in JS because it needs JS closures; the scoring math crosses the
438
+ boundary.
439
+
440
+ A reverser with the obfuscated bundle alone sees only *calls* into a native
441
+ binary, not the weights or the curves **as static literals**. Reading them out
442
+ of the binary requires disassembling the stripped Rust (Ghidra/IDA over
443
+ hours-to-days), not pasting into `obfuscator-io-deobfuscator` or asking an LLM.
444
+ This raises the cost of *static* recovery versus obfuscation alone — but see
445
+ the honest residual below: the scoring is recoverable *dynamically* regardless
446
+ of where the constants live.
447
+
448
+ **Distribution.** The binary ships per-platform as `optionalDependencies`
449
+ (`pi-ghost-autocomplete-<os>-<cpu>[-libc>`), following the @napi-rs
450
+ convention. End users on supported platforms need **no Rust toolchain** —
451
+ npm installs only the matching prebuild. Unsupported platforms fail at load
452
+ (by design: there is no recoverable JS fallback for the crown jewels). The
453
+ trade-off is a per-platform CI/release matrix (`.github/workflows/release.yml`).
454
+
455
+ **Honest residual.** This is not cryptographic protection, and two of the
456
+ "hidden" pieces are weaker than the binary boundary implies:
457
+
458
+ - **The reranker scoring is linear and black-box recoverable.** The score is a
459
+ weighted sum of per-axis normalized features, and the public API returns the
460
+ per-axis `scoreBreakdown` alongside the final `score`. Feeding ~5+ candidate
461
+ sets and solving the resulting linear system recovers the weight vector and
462
+ gate thresholds **without ever touching the binary** — no disassembly needed.
463
+ Hiding the literals raises the bar against copy-paste, not against a
464
+ competitor who probes the I/O.
465
+ - **The secret/volatile regexes are public patterns.** `sk-ant`, `ghp_`,
466
+ `AKIA`, `AIza`, JWT shape, etc. are the same prefixes published by gitleaks /
467
+ trufflehog. Moving them into the binary hides nothing of value; it's done for
468
+ consistency, not secrecy.
469
+
470
+ What genuinely benefits from the boundary is the **session-trie lookup
471
+ algorithm shape** (gates, typeahead-delta, decay, volatile verdict) — that
472
+ structure is not trivially recoverable from I/O. Runtime instrumentation
473
+ (`--inspect`, `LD_PRELOAD`, `process.dlopen` hooks) can still observe
474
+ inputs/outputs of everything. What tier-C removes is the *automated,
475
+ minutes-to-recover static* attack surface that pure JS obfuscation leaves. The
476
+ real enforcement remains the **SSPL-1.0** license; the native boundary removes
477
+ the temptation for the honest-but-curious and raises the cost for a casual
478
+ competitor from "paste into a deobfuscator" to "learn Rust disassembly (or
479
+ write a solver)."
480
+
481
+ > **Platform support.** Prebuilt binaries ship for win32-x64-msvc,
482
+ > darwin-arm64, linux-x64-gnu, linux-arm64-gnu. **Intel Mac (darwin-x64)
483
+ > is not supported** — Apple stopped selling Intel Macs in 2020; Intel Mac
484
+ > users can run via Rosetta 2. **musl (Alpine) is not supported** — there is
485
+ > no JS fallback, so unsupported platforms fail at load by design.
486
+
487
+ Verify what a release contains at any time:
488
+
489
+ ```bash
490
+ npm pack --dry-run
491
+ # expect: obfuscated dist/*.js + public dist/*.d.ts + README only.
492
+ # must show NO *.map, NO raw *.ts source (note: *.d.ts declarations are fine
493
+ # and expected), and NO *.node (the binary ships via optionalDependencies).
494
+ ```
495
+
496
+ ## Limitations
497
+
498
+ - No native Ollama provider in `pi-ai`; we configure an `openai-compat`
499
+ model pointed at Ollama's `/v1` endpoint. Any OpenAI-compatible local
500
+ server works the same way.
501
+ - Pin to `^0.74` of `pi-tui` / `pi-coding-agent` / `pi-ai` /
502
+ `pi-agent-core` (all under `@earendil-works`). The relevant interfaces are
503
+ pre-1.0; expect occasional follow-up bumps.
504
+ - Predictions are shown only at the end of the buffer to avoid splitting
505
+ lines mid-stream.
506
+ - The path index covers only tracked Git files. Untracked files are not
507
+ indexed in v1.
508
+ - Three-provider racing may create elevated cost or rate-limit pressure at
509
+ high typing speeds.
510
+
511
+ ## License
512
+
513
+ SSPL-1.0 (Server Side Public License v1) — see [LICENSE](./LICENSE).