@vectorize-io/hindsight-coding-agents 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,334 @@
1
+ # hindsight-coding-agents
2
+
3
+ Long-term project memory for **coding agents**, backed by [Hindsight](https://vectorize.io/hindsight).
4
+ One package, several agents: a shared reflect-and-inject core with a thin entry point per agent
5
+ (**opencode**, **Kilo CLI**, **Cline CLI**, **Claude Code**, **Codex CLI**, **Antigravity CLI**, **Cursor CLI**, **GitHub Copilot CLI**, **Grok Build**). Ingestion is fully
6
+ automatic — there is no setup command: a repo's git history and conversations flow into its memory
7
+ bank in the background as you work.
8
+
9
+ The premise: most of a real fix is derivable from the code, but the _last mile_ often hinges on a
10
+ project-specific decision that isn't in the code at all — a rounding rule, a retry allowlist, a
11
+ tie-break policy. Those decisions live in git history and past conversations. This package puts them
12
+ in front of the agent at the moment it starts working, and keeps a curated set of **knowledge pages**
13
+ (architecture, conventions, in-flight initiatives) that future sessions start from.
14
+
15
+ ## How it works
16
+
17
+ 1. **Seed a cold repo (automatic, once).** The first time an agent opens a repo whose bank is empty,
18
+ the entry point deterministically kicks off a background **seed**: it aggregates the last N commit
19
+ **messages** into a single cheap document (`gitlog` strategy) and spawns a short headless
20
+ **codebase survey** — run under the current agent's own CLI (claude/codex/antigravity/opencode),
21
+ read-only sandboxed — to map the structure. Both feed the knowledge pages.
22
+ 1. **Deepen progressively (automatic, every session).** Every session start also fires the
23
+ idempotent background **deepen engine**: it ingests any conversations not yet in the bank and the
24
+ next batch of recent commits **individually with their full diffs, newest first** — so full
25
+ decision-level precision arrives across sessions without a big-bang ingest. A per-bank lock and
26
+ document-id dedup make concurrent or repeated runs a no-op. Ask the agent
27
+ `hindsight_sync_status` to see where ingestion stands (`synced: true` = seeded memory fully
28
+ queryable).
29
+ 1. **Reflect once per session.** On the session's first prompt, the entry point runs Hindsight
30
+ `reflect` — an agentic synthesis over the bank that returns the past decision explaining the task
31
+ at hand, with its exact rule and literal values. The answer is cached for the session and
32
+ re-injected on every turn, wrapped in a visible-attribution directive so the agent surfaces a
33
+ `🧠 Using Hindsight Memories` header when memory informs its answer.
34
+ 1. **Knowledge-page sections every turn.** The repo's knowledge pages are fetched on a cadence and
35
+ matched **locally** against each prompt (a lexical section index — no server round-trip, no LLM
36
+ call, ~ms): the top-scoring page sections are injected with provenance and a pointer to the full
37
+ page. Fast like recall, organized like reflect; below a relevance floor nothing is injected.
38
+ 1. **Knowledge pages + tools.** At session start the agent is given the repo's page roster plus a
39
+ guide to the `hindsight_*` tools; it lists/reads pages to ground itself, searches raw memory for
40
+ specifics (`hindsight_search_memory` is where recall lives now), and calls
41
+ `hindsight_capture_initiative` right after a plan is approved to record a new feature as a tracked
42
+ page. On opencode these tools are registered natively; the hook harnesses get them through the
43
+ bundled **MCP server**.
44
+ 1. **Write back.** The live session is upserted into the bank as a JSON transcript — user/assistant
45
+ turns plus a compact `action` turn per tool call (tool name + primary target, e.g.
46
+ `Edit boltons/strutils.py`; no arguments or outputs) — so sessions compound into memory without
47
+ burying decisions in mechanical noise. On Stop for the hook harnesses; for the plugin harnesses
48
+ (opencode, Kilo) on the turn cadence **and** again when the session goes idle. The idle pass is
49
+ what captures the agent's reply: the per-turn hook fires while the host is building a request, so
50
+ the transcript it sees always stops short of the answer that request produces — without the idle
51
+ pass, write-back lagged a turn and a session's final exchange was never stored at all.
52
+ 1. **Never break the agent — never fail silently.** A failed reflect or page fetch degrades to
53
+ no-memory, but every outcome (`reflect_ok` / `reflect_empty` / `reflect_failed`, `pages_ok` /
54
+ `pages_failed`, with duration and error) is appended to a diagnostics file, so a memory-less
55
+ session can't masquerade as a memory session.
56
+
57
+ If the configured Hindsight server predates knowledge pages, the client detects that capability at
58
+ session start, skips page seeding and page lookups, and records `knowledge_pages_unavailable`.
59
+ Legacy bank configuration, git/session ingestion, reflection, and retention continue normally.
60
+
61
+ When memories **conflict** on the same rule, reflect prefers the latest/superseding decision — a rule
62
+ amended in a later conversation wins over the original.
63
+
64
+ ## Harnesses
65
+
66
+ Every harness runs the same surface (seed → session reflect → per-turn page sections → knowledge
67
+ tools → write-back); they differ only in how that surface is delivered.
68
+
69
+ | harness | kind | lifecycle wiring | install |
70
+ | ------------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
71
+ | `opencode` | persistent plugin | one process: load-time seed, session reflect + page sections, native tools, write-back | add the package dir to `opencode.json` → `"plugin": [...]` |
72
+ | `kilo` | persistent plugin | identical to `opencode` — Kilo CLI is an opencode fork, so it loads the same runtime (no hooks system) | `hindsight-coding-agents install kilo` → `kilo.json[c]` `"plugin"` |
73
+ | `claude-code` | per-prompt hooks | `SessionStart` (seed) + `UserPromptSubmit` (reflect + pages) + `Stop` (write-back) + MCP | `hindsight-coding-agents install claude-code` |
74
+ | `codex` | per-prompt hooks | same three hooks in `~/.codex/hooks.json` (+ `codex_hooks = true`, CLI ≥ 0.116) | `hindsight-coding-agents install codex` |
75
+ | `antigravity-cli` (`agy`) | lifecycle hooks | Antigravity CLI lifecycle hooks (`PreInvocation` + `Stop`) + MCP, plus a native colored `Hindsight · <bank>` status-line indicator | `hindsight-coding-agents install agy` |
76
+ | `cursor-cli` | lifecycle hooks | `sessionStart` (seed + pages) + `beforeSubmitPrompt` (reflect) + `stop` (write-back) | hooks in Cursor `hooks.json` |
77
+ | `copilot-cli` | lifecycle hooks | `sessionStart` (seed + pages) + `userPromptTransformed` (reflect) + `agentStop` (write-back) + MCP | `~/.copilot/hooks/hindsight-coding-agents.json` + `mcp-config.json` |
78
+ | `grok-build` | lifecycle hooks | `SessionStart` (seed) + `Stop` (write-back) + MCP | native `~/.grok/config.toml` — no Claude Code dependency |
79
+ | `cline-cli` | persistent plugin | native `beforeModel` (seed/reflect/pages) + `afterRun` (write-back) + MCP | `cline plugin install` (run by the installer) |
80
+
81
+ The hook-based harnesses share one runtime (`src/core/hook.ts`) plus their SessionStart/Stop
82
+ entrypoints. Persistent-plugin hosts (opencode/Kilo/Cline) delegate to the same `RuntimeCore`,
83
+ which in turn calls those shared session-start, prompt, and retain operations; only their host API
84
+ adapters differ. Opencode and Kilo can register the knowledge tools natively; Cline uses the shared
85
+ MCP server. All support opt-in **incremental git-sync** (retain commits new since the seed on load).
86
+
87
+ Antigravity renders the Hindsight indicator with its documented custom status-line command. It is
88
+ local and never calls the Hindsight API while the TUI redraws. If you already configured an
89
+ Antigravity custom status line, the installer leaves it untouched rather than replacing it.
90
+
91
+ ### Grok Build limitation
92
+
93
+ Grok Build executes `UserPromptSubmit` hooks but ignores their stdout. Hindsight can therefore not
94
+ inject a reflect result, memory block, or banner into Grok's model-visible conversation. Grok still
95
+ gets native bank setup and transcript retention plus Hindsight MCP tools and the companion skill;
96
+ ask it to call `hindsight_reflect` or `hindsight_search_knowledge_pages` when needed. Automatic
97
+ injection requires a future Grok prompt-transform API.
98
+
99
+ ### Cline CLI scope
100
+
101
+ Cline's external file hooks cannot alter a model request, so Hindsight uses Cline's native plugin
102
+ API instead. Its `beforeModel` hook injects the shared reflect/page context and
103
+ its `afterRun` hook upserts Cline's runtime transcript. `hindsight-coding-agents install cline-cli`
104
+ runs `cline plugin install --force <package-path>` and configures MCP plus the companion skill.
105
+ Cline CLI sandboxes plugin hooks with a three-second limit, so a slow first reflect finishes in the
106
+ background and is injected on a subsequent model call or turn rather than aborting the session.
107
+ Its write-back retains only user-visible user/assistant text; tool arguments, tool results and
108
+ command output, tool-role messages, reasoning parts, and Hindsight's injected context are excluded.
109
+ Cline IDE extensions are not currently supported by this CLI integration.
110
+
111
+ ### Install — one command, every agent
112
+
113
+ ```bash
114
+ npm install -g hindsight-coding-agents
115
+ hindsight-coding-agents install # detects your agents, wires each natively
116
+ hindsight-coding-agents install codex # or pick specific harnesses
117
+ hindsight-coding-agents uninstall # removes exactly what install added
118
+ ```
119
+
120
+ Install globally (not `npx`): the wiring points at this package's files, so it must live at a
121
+ stable path — the installer refuses to run from an npx cache. **Updating** is just
122
+ `npm update -g hindsight-coding-agents`: the wired paths stay valid, every new session runs the
123
+ new version; re-run `install` (idempotent) only when a release note says the wiring changed.
124
+
125
+ `install` merges the native wiring (hooks + MCP registration where the host wants them) into each
126
+ agent's own config, preserving everything already there; it is idempotent (re-run after moving the
127
+ package) and backs up any pre-existing file it touches as `<file>.hindsight-backup`. `uninstall`
128
+ removes only our entries. On Claude Code the install also ships a **companion skill**
129
+ (`~/.claude/skills/hindsight-coding-agent`) that teaches the agent how this memory works — what
130
+ "store this in hindsight" should do, the tool surface, per-repo configuration, debugging — so users
131
+ can ask the agent itself. Manual wiring per harness, if you prefer:
132
+
133
+ **opencode** installs directly — point `opencode.json` at the package dir:
134
+
135
+ ```json
136
+ { "plugin": ["/path/to/hindsight-coding-agents"] }
137
+ ```
138
+
139
+ **Claude Code** and **Codex** get their full three-hook + MCP wiring from this package's own
140
+ installer — `hindsight-coding-agents install claude-code` / `install codex`. This package's `bin`
141
+ entries (`hindsight-claude-hook`, `hindsight-codex-hook`,
142
+ `hindsight-cursor-hook`) are the individual injection-only `UserPromptSubmit` entrypoints for a
143
+ minimal, hand-wired setup.
144
+
145
+ Adding an agent: hook-based → write a `HookSpec` entry point (see `src/cursor-hook.ts`) and register
146
+ a `hookAdapter` in `src/harness/registry.ts`; persistent-plugin → implement `HarnessAdapter`
147
+ (`src/core/types.ts`) fully (see `src/harness/opencode.ts`).
148
+
149
+ ## Configuration
150
+
151
+ All configuration is **ONE JSON file, no environment variables** (exception: `HINDSIGHT_DIAG_FILE`
152
+ for the diagnostics path): `~/.hindsight/coding-agent.json`. Layering, later wins per field:
153
+
154
+ 1. built-in defaults
155
+ 2. the file's top level
156
+ 3. its `harnesses.<name>` section — per-agent override
157
+
158
+ There is deliberately no repo-carried config file — per-repo bank routing is `mapPathToBank`,
159
+ per-agent differences are `harnesses.<name>`.
160
+
161
+ Each entry point knows which harness it _is_ (the opencode plugin is loaded by opencode, the codex
162
+ hook by Codex...), so one shared config serves several agents side by side:
163
+
164
+ ```jsonc
165
+ {
166
+ "apiUrl": "https://api.hindsight.vectorize.io",
167
+ "harnesses": {
168
+ "opencode": { "reflectTimeoutMs": 60000 },
169
+ "claude-code": { "disabled": true }, // e.g. memory off for Claude only
170
+ },
171
+ }
172
+ ```
173
+
174
+ ### Reference
175
+
176
+ | field | default | meaning |
177
+ | ----------------------- | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
178
+ | `apiUrl` | `https://api.hindsight.vectorize.io` | Hindsight API base URL (set to `http://localhost:8888` for a local server) |
179
+ | `apiToken` | — | bearer token (Hindsight Cloud) |
180
+ | `bankId` | — | **explicit static bank**; unset ⇒ per-repo dynamic resolution (below) |
181
+ | `dynamicBankId` | dynamic iff no `bankId` | force dynamic (`true`) or static (`false`) resolution |
182
+ | `bankIdTemplate` | `"coding-agent::{gitProject}"` | dynamic bank id format; the default makes every agent share one bank per repo |
183
+ | `mapPathToBank` | — | absolute path → bank; **longest prefix wins**; overrides everything |
184
+ | `resolveWorktrees` | `true` | `{gitProject}`: linked worktrees share the main repo's bank |
185
+ | `disabled` | `false` | hard off-switch (inert plugin/hook — a no-memory baseline) |
186
+ | `reflectTimeoutMs` | `120000` | session-reflect timeout (hook harnesses additionally cap it at 25s to fit the host's hook window); on timeout the session runs without reflect (recorded) |
187
+ | `pageRefreshEveryTurns` | `10` | refetch the knowledge pages and re-inject the page roster + tool guide every N user turns |
188
+ | `autoSeed` | `true` | SessionStart: auto-seed a cold repo's bank from git history |
189
+ | `seedLimit` | `300` | auto-seed: most-recent-N-commits cap |
190
+ | `codebaseSurvey` | `true` | SessionStart: headless survey of a cold repo's structure, run under the current harness's own CLI (claude/codex/antigravity/opencode), falling back to any available agent |
191
+ | `surveyModel` | `haiku` | model for the survey — Claude recipe only (`claude -p --model`); other agents use their configured default |
192
+ | `surveyBudgetUsd` | `2` | survey spend cap — Claude recipe only (`claude -p --max-budget-usd`); other agents rely on their read-only sandbox |
193
+ | `retainSessions` | `true` | plugin-harness write-back (opencode, Kilo): async upsert of the session transcript every turn, plus an idle flush that captures the reply the per-turn pass can't see (set `false` to opt out; hook harnesses always write on Stop) |
194
+ | `retainEveryTurns` | `1` | opencode write-back cadence (user turns) |
195
+ | `logLevel` | `"info"` | plugin-log verbosity (`"debug"` \| `"info"` \| `"warn"` \| `"error"`); `HINDSIGHT_LOG_LEVEL` env overrides |
196
+ | `gitIngest` | `"message"` | git depth for seeding AND staying current (same engine): `"message"` = commit messages only (one doc, re-upserted when HEAD moves); `"full"` = messages + per-commit full diffs (progressive, newest first); `"none"` = git off |
197
+ | `harnesses.<name>` | — | per-harness override of any field above |
198
+ | `harness` | `opencode` | **deepen engine only**: which session format `--conversations` is read as |
199
+
200
+ ### Per-repo opt-in/out — `banks.<bankId>`
201
+
202
+ Per-repo control lives in the SAME file, keyed by the **resolved bank id** (shown in the session
203
+ banner) and applied AFTER bank resolution — so it works regardless of where the repo lives, and
204
+ survives directory moves:
205
+
206
+ ```jsonc
207
+ {
208
+ "banks": {
209
+ "coding-agent::secret-client": { "disabled": true }, // blacklist: no memory at all
210
+ "coding-agent::old-name": { "bank": "team::shared" }, // rename / converge banks
211
+ "coding-agent::big-mono": { "gitIngest": "full", "retainSessions": false },
212
+ },
213
+ }
214
+ ```
215
+
216
+ Any behavioral field can be overridden per bank, and `bank` **renames the destination** (single
217
+ hop: the section is selected by the resolved id, the target is literal — several ids may converge
218
+ on one shared bank, and the target's own section is not consulted). Other bank-resolution fields
219
+ are ignored inside a bank section.
220
+
221
+ #### Recipe: two repos, one shared bank
222
+
223
+ Two ways, by what the natural key is:
224
+
225
+ **By resolved id** — you know the repo names; works wherever the repos live (and keeps working if
226
+ they move). Both ids converge on one literal target:
227
+
228
+ ```jsonc
229
+ {
230
+ "banks": {
231
+ "coding-agent::backend": { "bank": "team::product" },
232
+ "coding-agent::frontend": { "bank": "team::product" },
233
+ },
234
+ }
235
+ ```
236
+
237
+ **By path prefix** — the repos live under one directory; a single `mapPathToBank` entry covers
238
+ every repo (present and future) beneath it:
239
+
240
+ ```jsonc
241
+ {
242
+ "mapPathToBank": { "/Users/me/work/client-x": "client-x-memory" },
243
+ }
244
+ ```
245
+
246
+ Rule of thumb: converge by **id** for a hand-picked set of repos; map by **path** when a folder is
247
+ the boundary ("everything I clone under `work/client-x` shares memory").
248
+
249
+ ### Bank resolution
250
+
251
+ Coding memory is **per repository**. Resolution order for the working directory:
252
+
253
+ 1. `mapPathToBank` — longest matching absolute-path prefix (mapping a repo root covers every
254
+ subdirectory; deeper mappings win; overrides even an explicit `bankId`).
255
+ 2. Static — `bankId` set (or `dynamicBankId: false`).
256
+ 3. Dynamic — `bankIdTemplate` with placeholders:
257
+ - `{gitProject}` — worktree-aware repo name: `git rev-parse --git-common-dir` resolves every
258
+ linked worktree to the **main** worktree's basename, so all worktrees of a repo share one bank
259
+ (bare repos use the bare dir name; non-git directories fall back to the dir basename)
260
+ - `{project}` — plain working-directory basename
261
+ - `{harness}` — the entry point asking (`opencode`, `claude-code`, `codex`, `antigravity-cli`, `cursor-cli`, `copilot-cli`)
262
+ - `{channel}` / `{user}` — `$HINDSIGHT_CHANNEL_ID` / `$HINDSIGHT_USER_ID`
263
+
264
+ The default `"coding-agent::{gitProject}"` is **harness-neutral**, so opencode, Claude Code, and Codex
265
+ all share one memory per repo — use `"{harness}-{gitProject}"` to split per agent instead.
266
+
267
+ ## Ingestion internals (no CLI)
268
+
269
+ There is no user-facing ingest command — the deepen engine (`dist/deepen.js`) is spawned by every
270
+ session start and does only the missing work: bank configuration, conversation import (dedup by
271
+ document id), the one-time gitlog seed, the next per-commit diff batch (newest first, bounded per
272
+ run), then knowledge pages once extraction has drained. Harnesses that need deterministic ingestion
273
+ (benchmarks, e2e suites) run the same engine directly and poll `dist/status.js` until
274
+ `"synced": true` — the exact readiness contract the `hindsight_sync_status` agent tool reports.
275
+
276
+ Past-conversation import accepts a normalized interchange file (engine `--conversations` flag):
277
+ `[{ "id": "s1", "turns": [{ "role": "user", "text": "...", "timestamp?": "ISO" }, ...] }, ...]`,
278
+ chronological (a later chat can amend an earlier one). Day-to-day, conversations simply accrue from
279
+ the live session write-back — no export step.
280
+
281
+ Local Hindsight for trying it out:
282
+
283
+ ```bash
284
+ docker run -d -p 8888:8888 -p 9999:9999 -e HINDSIGHT_API_LLM_PROVIDER=gemini \
285
+ -e HINDSIGHT_API_LLM_API_KEY=$GEMINI_API_KEY -e HINDSIGHT_API_LLM_MODEL=gemini-2.5-flash \
286
+ ghcr.io/vectorize-io/hindsight:latest
287
+ ```
288
+
289
+ ## Diagnostics & logging
290
+
291
+ Two files, two audiences:
292
+
293
+ **Leveled plugin log** (humans debugging): `$TMPDIR/hindsight-coding-agent/plugin.log` (override
294
+ `HINDSIGHT_LOG_FILE`) — timestamped `LEVEL [scope] message` lines from every component, including
295
+ the ingestion engine. Level defaults to `info`; set `"logLevel": "debug"` in config or
296
+ `HINDSIGHT_LOG_LEVEL=debug` for ad-hoc debugging (at `debug`, every diag event below is mirrored
297
+ here too, so one file tells the whole story).
298
+
299
+ **Structured diag events** (machines/harnesses): every reflect and page-fetch outcome is appended
300
+ as a JSON line to `/tmp/hindsight-plugin.log` (override with `HINDSIGHT_DIAG_FILE`):
301
+
302
+ ```json
303
+ {
304
+ "ts": "2026-07-27T07:05:52Z",
305
+ "harness": "claude-code",
306
+ "event": "reflect_ok",
307
+ "ms": 14210,
308
+ "chars": 792,
309
+ "query": "..."
310
+ }
311
+ ```
312
+
313
+ `reflect_failed` / `pages_failed` record the error; if you're comparing memory-on vs memory-off,
314
+ check this file — a run whose reflects failed is a no-memory run. Seed starts are logged as
315
+ `seed_started`.
316
+
317
+ ## Layout
318
+
319
+ ```
320
+ src/
321
+ core/ # harness-agnostic: config (layered), bank resolution, hindsight client, missions,
322
+ # git + chat ingest, git-sync, seed + survey, pages-index (local section matching),
323
+ # knowledge-injection, knowledge-tools, session-start, transcript readers,
324
+ # hook runtime, RuntimeCore
325
+ harness/ # per-agent adapters + registry (opencode persistent; claude/codex/antigravity/cursor/copilot as hooks)
326
+ index.ts # opencode plugin entrypoint
327
+ claude-hook.ts / claude-sessionstart-hook.ts / claude-stop-hook.ts # Claude Code entrypoints
328
+ codex-hook.ts / codex-sessionstart-hook.ts / codex-stop-hook.ts # Codex CLI entrypoints
329
+ antigravity-hook.ts / antigravity-stop-hook.ts # Antigravity CLI entrypoints
330
+ cursor-hook.ts # Cursor CLI hook entrypoint (bin: hindsight-cursor-hook)
331
+ copilot-*.ts # GitHub Copilot CLI SessionStart/prompt/agentStop entrypoints
332
+ mcp-server.ts # hindsight_* knowledge/recall tools for the hook harnesses (MCP)
333
+ backfill.ts # backfill CLI (bin: hindsight-coding-backfill)
334
+ ```