hippocamp 0.2.5 → 0.2.7

package/docs/how-it-works.md

@@ -0,0 +1,119 @@
+# How It Works
+
+[← Back to README](../README.md)
+
+## What It Stores
+
+- durable preferences and working style
+- current project state and open threads
+- meaningful events and decisions
+- references to commits, PRs, issues, reviews, and CI instead of duplicated GitHub facts
+
+## Why Git
+
+Git is a practical default for agent memory:
+
+- Auditable: every memory change has a diff, author, timestamp, and commit history.
+- Readable: memory stays as Markdown files that humans and agents can inspect without a special UI.
+- Collaborative: multiple agents and humans can review, branch, merge, and roll back the same memory repo.
+- Portable: a private remote lets the same memory follow you across machines and agent environments.
+- Agent-native: coding agents are already connected to GitHub through MCP, CLIs, or local Git credentials, so Hippocamp does not need an external database, hosted dependency, or separate token setup.
+
+## Memory Layout
+
+Global memory lives in:
+
+```text
+~/.lagoon/
+```
+
+Project memory lives in:
+
+```text
+~/.lagoon/projects/<project-slug>/
+```
+
+Suggested files:
+
+```text
+identity.md
+how_i_work.md
+preferences.md
+open_loops.md
+events/YYYY-MM-DD.md
+events/YYYY-MM-DD.index.json
+projects/<project-slug>/
+  project.md
+  current_state.md
+  open_threads.md
+  events/YYYY-MM-DD.md
+  events/YYYY-MM-DD.index.json
+```
+
+## MCP Tools
+
+The local MCP server exposes:
+
+- `wake_up`
+- `read_memory_file`
+- `write_memory_file`
+- `append_event`
+- `list_memory_files`
+- `search_memory`
+- `sync_memory`
+
+Typical agent flow:
+
+1. Call `wake_up` at the start of a top-level task.
+2. Read the returned global and project memory.
+3. Use `search_memory` for task-specific recall after wake-up.
+4. Use `append_event` for meaningful milestones, with a title and 1-8 concise keywords.
+5. Update curated files like `current_state.md` and `open_threads.md` before finishing.
+
+Writes sync by default. If sync fails or is skipped, call `sync_memory`.
+
+Event writes update a sibling `events/YYYY-MM-DD.index.json` file. The Markdown event remains the canonical memory; the sidecar is rebuildable. Search ranks indexed events and curated files with BM25 and typo tolerance ([MiniSearch](https://github.com/lucaong/minisearch)), weighting keywords above titles and titles above bodies. It returns small coherent evidence blocks from indexed events; an event found only by its keywords or title returns its first paragraphs. If no indexed result is strong enough, it returns no event result instead of scanning whole event logs.
+
+## Memory Rules
+
+- Keep memory concise.
+- Prefer curated summaries over raw event history.
+- Give events a short `Keywords:` section; keywords weigh the most in search ranking. Older events with a `Cues:` section still search the same way.
+- Do not duplicate GitHub-owned facts such as commits, PRs, issues, reviews, or CI results.
+- Store artifact references plus the missing rationale, preference, assumption, or follow-up context.
+- Use project scope for project-specific state.
+- Use global scope only for durable context that should follow you across projects.
+
+Events are stamped automatically with agent/session provenance when available (`Agent:` / `Session:` lines). Installers set `HIPPOCAMP_AGENT`; session defaults to a process-lifetime id. No manual config is required.
+
+Example event content:
+
+```md
+Agent: claude
+Session: mcp-a1b2c3d4
+
+Keywords:
+- local-first
+- token-free
+- install-story
+
+References:
+- commit: abc1234
+- pr: #12
+
+Decision:
+Keep the local MCP path token-free and rely on normal git credentials.
+
+Why:
+This reduces onboarding friction for open-source users and avoids cloud auth concerns in the MVP.
+```
+
+## What This Is Not
+
+- Not a hosted cloud service
+- Not a database
+- Not a notes app
+- Not a vector store
+- Not a queue
+
+The cloud/API version can be redesigned later. The MVP is intentionally local-first and Git-native.

package/docs/results.md

@@ -0,0 +1,59 @@
+# Results
+
+[← Back to README](../README.md)
+
+Every number below comes from a script in this repo, so you can rerun it. We report where Hippocamp falls short too.
+
+## Finding the right memory
+
+LongMemEval-S, all 470 answerable questions, top 5 results, no model involved (`npm run eval:retrieval`).
+
+| Measure | Hippocamp | BM25 reference |
+| --- | ---: | ---: |
+| Right session in top 5 (Recall@5) | 90.5% | 91.4% |
+| At least one right session in top 5 (Hit@5) | 96.0% | 96.8% |
+| Returned text contains the answer turn | 87.9% | n/a |
+| Text returned per query | 5.9k chars | 65.5k chars |
+| Search latency, cold index | 42 ms | n/a |
+
+BM25 returns whole sessions; Hippocamp returns the matching paragraphs of each result.
+
+## Answering with it
+
+100 LongMemEval-S questions, reader Claude Sonnet 5, grader gpt-4o with the official LongMemEval grading prompts (`npm run eval:qa`).
+
+| What the reader gets | Text per question | Answer accuracy |
+| --- | ---: | ---: |
+| Hippocamp, top 5 | 6.7k chars | 71% |
+| Hippocamp, top 10 | 12.9k chars | 77% |
+| The correct sessions, given directly | 28.8k chars | 90% |
+
+The same reader reaches 90% with the correct sessions, so the remaining gap is retrieval, not the reader.
+
+## Real coding questions
+
+53 questions about real work in a personal Lagoon across 13 projects (`npm run eval:lagoon`). This is our own test set, kept in the private Lagoon.
+
+| Measure | Result |
+| --- | ---: |
+| Right event in top 5, 43 answerable questions | 42 / 43 |
+| No results for 10 questions no memory is about | 10 / 10 |
+
+## Why less text matters
+
+Memory results share the agent's context with code, diffs, and tool output, and agents search several times per task. About 6k characters per search is roughly 1.5k tokens; whole sessions would be about 16k. Fewer tokens make each turn cheaper and faster, and keep the relevant lines from being buried. Less text is only useful when the answer is still in it, which is why we report answer accuracy next to text size.
+
+## Limits
+
+- Recall@5 is just under the BM25 reference, by less than 1 point, with 11 times less text.
+- No embeddings or semantic search: a memory written in other words can be missed. On LongMemEval-S, 9 of 470 questions miss with no word in common with the answer.
+- Answer accuracy is measured on 100 questions, about plus or minus 4 points.
+- Self-reported scores from other memory systems use their own readers and graders, so they are not directly comparable with these.
+
+## Reproduce
+
+`npm run eval:retrieval` measures search on LongMemEval-S without any model: Recall@5, Hit@5, MRR, snippet evidence (the returned text comes from an answer turn), returned context size, and latency, next to an in-process BM25 reference. Run it with `--help` for the one-time data download.
+
+`npm run eval:lagoon` runs the same kind of check, read-only, against your real Lagoon. It reads cases from `<Lagoon root>/evals/retrieval-cases.json`: a project, a query, and the event ids that answer it. An empty list means no memory should be returned. Keep the cases in Lagoon, not in this repo, because they quote private work.
+
+`npm run eval:qa` measures answer accuracy on the same LongMemEval-S data with the official LongMemEval reader and grader prompts. It compares Hippocamp top 5 (`hippocamp`), top 10 (`hippocamp-k10`), and the labeled answer sessions (`oracle`). It calls an OpenAI-compatible endpoint from `MANIFEST_BASE_URL` and `MANIFEST_API_KEY`; set `QA_READER_MODEL` and `QA_GRADER_MODEL` to change models. Results are cached in `.context/qa-results.jsonl`, and `--dry-run` shows the jobs and token estimates without any API call.

package/docs/setup.md

@@ -0,0 +1,66 @@
+# Setup
+
+[← Back to README](../README.md)
+
+## Install
+
+```bash
+npx hippocamp@latest install
+```
+
+To install into one agent only:
+
+```bash
+npx hippocamp@latest install-claude
+npx hippocamp@latest install-codex
+npx hippocamp@latest install-grok
+```
+
+Upgrade with the same command: `npx hippocamp@latest upgrade`. Installers default to `~/.lagoon` as the memory repo; pass `--global-root /absolute/path/to/lagoon` to use another clone.
+
+Installed through npx, the installer copies the package to `~/.hippocamp/<version>/` and registers that copy, so the server starts without npx or network and keeps working if npm cleans its cache. From a source checkout, `npm install` then `npm run install:claude` (or `install:codex`, `install:grok`) registers the checkout directly.
+
+The Codex installer refreshes `~/.codex/AGENTS.md`. The Claude installer refreshes `~/.claude/CLAUDE.md`. The Grok installer refreshes `~/.grok/rules/hippocamp.md` and registers the MCP server with `HIPPOCAMP_AGENT=grok`.
+Managed instruction blocks tell the agent to call `wake_up` at the start of new top-level coding tasks before repo exploration or edits.
+
+## Lagoon Repo
+
+Hippocamp keeps memory in a local Git repo. The installer creates `~/.lagoon` if it is missing. To use memory you already have on another machine, clone it first:
+
+```bash
+git clone git@github.com:YOUR_USER/lagoon.git ~/.lagoon
+```
+
+The repo should usually be private. Pushing memory to a private remote keeps it available across machines and agent environments while still using normal Git access controls. Hippocamp does not need a GitHub token for local MCP mode; it uses your normal local Git credentials.
+
+Memory works locally without a remote. To back it up privately and share it across machines, create a private remote:
+
+```bash
+gh repo create lagoon --private --source ~/.lagoon --remote origin --push
+```
+
+If push auth is not configured yet, use your preferred GitHub setup. With GitHub CLI:
+
+```bash
+gh auth login
+gh auth setup-git
+cd ~/.lagoon
+git push --dry-run
+```
+
+Once `git push --dry-run` works from `~/.lagoon`, Hippocamp can sync memory.
+
+## Configuration
+
+Environment variables are optional for local use:
+
+- `HIPPOCAMP_GLOBAL_ROOT`: local Lagoon clone path. Default: `~/.lagoon`
+- `HIPPOCAMP_PROJECT_ROOT`: project root used to infer the current project slug. Default: current working directory
+- Git worktrees, including Conductor workspaces, resolve to the main repository name so every workspace shares the same project memory.
+- An empty folder (nothing but `.git`, like a fresh agent session folder) has no project: project memory falls back to global memory, so these sessions start clean and stay searchable from everywhere.
+- `MANIFEST_BASE_URL`: OpenAI-compatible base URL used by `hippocamp dream --write`
+- `MANIFEST_API_KEY`: API key used by `hippocamp dream --write`
+- `HIPPOCAMP_DREAM_MODEL`: model used by Dream. Default: `auto`
+- `HIPPOCAMP_DREAM_THRESHOLD_CHARS`: wake-up size required before Dream proposes compaction. Default: `20000`
+- `HIPPOCAMP_DREAM_TARGET_CHARS`: target combined size for `current_state.md` and `open_threads.md`. Default: `15000`
+- `HIPPOCAMP_DREAM_AUTO_MERGE`: set to `true` for a guarded squash merge in a repository named `lagoon`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "hippocamp",
-  "version": "0.2.5",
+  "version": "0.2.7",
   "private": false,
   "description": "Local Git-backed memory for AI agents.",
   "license": "MIT",

package/scripts/eval-lagoon.cjs

@@ -61,8 +61,9 @@ async function run(options) {
   }
 
   const { cases } = JSON.parse(await fs.readFile(options.cases, "utf8"));
-  // searchMemory derives the project slug from the folder name, so a bare temp
-  // folder per slug reads that project's memory without touching any real repo.
+  // searchMemory derives the project slug from the folder name, so a temp path per
+  // slug reads that project's memory without touching any real repo. The folder is
+  // not created: an existing empty folder has no project and reads global memory.
   const sandboxRoot = await fs.mkdtemp(path.join(os.tmpdir(), "hippocamp-eval-lagoon-"));
   const answerable = { count: 0, hit: 0, mrr: 0, chars: 0, topScores: [] };
   const empty = { count: 0, correct: 0, topScores: [] };
@@ -74,7 +75,6 @@ async function run(options) {
       }
 
       const projectRoot = path.join(sandboxRoot, item.project);
-      await fs.mkdir(projectRoot, { recursive: true });
 
       const search = await memory.searchMemory({
         query: item.query,

package/scripts/hippocamp-dream.cjs

@@ -272,7 +272,7 @@ async function buildThreadEvidence(report, { budgetChars }) {
     const item = {
       path: match.path,
       heading: match.heading,
-      cues: match.cues || [],
+      keywords: match.keywords || [],
       score: match.score,
       snippet: match.snippet,
     };
@@ -348,7 +348,7 @@ function createPrompt({ report, targetChars, threadEvidence }) {
         report.files["open_threads.md"].content.trim() || "# Open Threads",
         "",
         "## Thread Evidence",
-        "Evidence is retrieved from cue-indexed project events. It is intentionally small and may be incomplete.",
+        "Evidence is retrieved from keyword-indexed project events. It is intentionally small and may be incomplete.",
         `Evidence budget: ${threadEvidence.evidenceChars}/${threadEvidence.evidenceBudgetChars} chars`,
         JSON.stringify(threadEvidence.items, null, 2),
       ].join("\n"),

package/scripts/hippocamp-mcp.cjs

@@ -114,23 +114,23 @@ async function main() {
     "append_event",
     {
       description:
-        "Append a dated event entry under events/YYYY-MM-DD.md and update the sibling cue index. Requires a title and 1-8 short cues for recall. Automatically stamps Agent/Session provenance when available. Store references to GitHub artifacts instead of copying their content.",
+        "Append a dated event entry under events/YYYY-MM-DD.md and update the sibling keyword index. Requires a title and 1-8 short keywords for recall. Automatically stamps Agent/Session provenance when available. Store references to GitHub artifacts instead of copying their content.",
       inputSchema: {
         scope: z.enum(["global", "project"]).default("project"),
         title: z.string().trim().min(1),
-        cues: z.array(z.string().trim().min(1)).min(1).max(8),
+        keywords: z.array(z.string().trim().min(1)).min(1).max(8),
         content: z.string(),
         sync: z.boolean().optional(),
         projectRoot: projectRootSchema,
       },
     },
-    async ({ scope, title, cues, content, sync, projectRoot }) => {
+    async ({ scope, title, keywords, content, sync, projectRoot }) => {
       const clientVersion = server.server?.getClientVersion?.();
       return toTextResult(
         await memory.appendEvent({
           scope,
           title,
-          cues,
+          keywords,
           content,
           projectRoot,
           sync,
@@ -162,7 +162,7 @@ async function main() {
     "search_memory",
     {
       description:
-        "Search task-specific memory. Returns no results when the question's distinctive words appear in no memory, unless the question names a date. Event search uses cue indexes and returns only bounded coherent evidence; it does not scan unindexed event logs as a fallback. Event results carry their ISO timestamp as id; use it for when-questions. A month name in the query favors events from that month. A YYYY-MM-DD date, today, yesterday, this/last week, or this/last month keeps only events from those dates (UTC); undated files still match. Default 5 results suits a single fact; ask for 10-20 when the question combines several past sessions (counts, totals, comparisons, timelines), and search again if results look incomplete.",
+        "Search task-specific memory. Returns no results when the question's distinctive words appear in no memory, unless the question names a date. Event search uses keyword indexes and returns only bounded coherent evidence; it does not scan unindexed event logs as a fallback. Event results carry their ISO timestamp as id; use it for when-questions. A month name in the query favors events from that month. A YYYY-MM-DD date, today, yesterday, this/last week, or this/last month keeps only events from those dates (UTC); undated files still match. Default 5 results suits a single fact; ask for 10-20 when the question combines several past sessions (counts, totals, comparisons, timelines), and search again if results look incomplete.",
       inputSchema: {
         query: z.string(),
         scope: z.enum(["global", "project", "both"]).default("both"),