kimetsu-pi 0.1.3 → 0.1.4

package/README.md

@@ -11,50 +11,55 @@
 **kimetsu-pi** brings [Kimetsu](https://kimetsu.dev) — a local-first memory brain
 for coding agents — to the [Pi](https://pi.dev) coding agent.
 
-Coding agents are brilliant and forgetful: every session starts from zero. Kimetsu
-is a sidecar brain that captures the lessons your agent earns, learns which ones
-actually help, and hands them back before the next task. The memory pipeline makes
-**no LLM calls** — storage and retrieval are 100% local, free, and offline-capable.
+Kimetsu carries useful lessons between sessions and retrieves relevant project
+knowledge before a task. Storage and retrieval run locally without LLM API calls.
+Automatic lesson extraction can use a configured model; its cost and connectivity
+depend on that configuration.
 
 Learn more at **[kimetsu.dev](https://kimetsu.dev)**.
 
 ## What it is
 
-Pi has no MCP layer, so this package brings Kimetsu to Pi through Pi's own
-extension points:
+This package connects Kimetsu through Pi's native extension and skill interfaces:
 
 - **Extension** (`extensions/kimetsu.ts`) — a TypeScript Pi extension that hooks
-  Pi lifecycle events (`session_start`, `before_agent_start`, `agent_end`,
-  `session_shutdown`) and shells
-  out to the `kimetsu` binary to warm, load, and persist brain context around each
-  session. Each call is capped by a timeout, so a slow or hung binary never stalls
-  Pi. If the binary is not on `PATH`, every hook silently no-ops and Pi is
+  Pi lifecycle events and invokes the `kimetsu` binary to warm, load, and persist
+  brain context around each session. Interactive hooks have a 10-second timeout;
+  session saving has a
+  separate five-minute limit for model-based distillation. If the binary is not
+  on `PATH`, every hook silently no-ops and Pi is
   completely unaffected.
 - **Skill** (`skills/kimetsu-brain/SKILL.md`) — a Pi skill that teaches the agent
-  when and how to consult and record memories during a task.
+  to reuse injected context, correct outdated memories, and cite memories that
+  actually helped during a task.
+
+## What's new in 0.1.4
+
+- Fresh memory per task: older raw injections are excluded from model requests
+  and new summaries, including after corrections, queued user messages, and
+  branch changes.
+- Session saving gets up to five minutes, while interactive hooks retain their
+  10-second limits.
+- Updated guidance for memory reuse, corrections, and usefulness feedback.
+- Verified with Pi 0.85.1 and Kimetsu 2.8.0 on Linux, macOS, and Windows; updated
+  dependencies, isolated integration tests, and protected CI and release gates.
 
 ## Why Kimetsu
 
-- **Remembers what matters** — conventions, failure patterns, the exact command
-  that regenerates your schema. Captured once, retrieved by meaning.
+- **Remembers project knowledge** — conventions, failure patterns, and useful
+  commands stay available across sessions.
 - **Speaks first** — proactive session-start digests, episodic resumes, and
   pre-task context, so the agent's first turn already knows your repo.
-- **Learns what helps** — cited memories get promoted; stale advice decays and is
-  pruned.
-- **Model-free retrieval** — FTS5 + local embeddings + local reranking. Zero API
-  cost, works offline.
-- **Stays yours** — one SQLite file per project. No cloud, no vector DB, no
-  telemetry. Back it up with `cp`.
-
-## Benchmarks
-
-| | |
-|---:|---|
-| **89.4%** | LoCoMo, the long-conversation memory benchmark |
-| **83.0%** | LongMemEval, the public long-term-memory benchmark |
-| **73.3%** | BEAM 100K memory benchmark |
-| **13×** | cheaper per solved task ($0.19 vs $2.47 on a Terminal-Bench slice) |
-| **~1M** | memories in ~3 GB RAM with sub-2s retrieval, one SQLite file |
+- **Learns what helps** — citations provide usefulness feedback; maintenance
+  identifies memories that may need pruning.
+- **Local retrieval** — lexical search in the default lean build, with optional
+  local embeddings and reranking. No LLM API calls for retrieval.
+- **Stays yours** — a local SQLite brain per project, with optional user-wide
+  memory. No hosted vector database is required.
+
+For Kimetsu's benchmark results, datasets, and evaluation limits, see the
+[core project's benchmark notes](https://github.com/RodCor/kimetsu#benchmarks-vs-other-memory-systems).
+Those evaluate the memory system; this package's CI verifies its Pi integration.
 
 ## Prerequisites
 
@@ -64,7 +69,8 @@ Install current Pi (Node.js 22.19 or newer):
 npm install -g @earendil-works/pi-coding-agent
 ```
 
-The `kimetsu` v2.7.0 or newer binary must be on `PATH`. Install it with:
+The `kimetsu` binary must be on `PATH`. Kimetsu 2.8.0 is the tested release for
+this package; the CLI integration requires at least 2.7.0. Install it with:
 
 ```sh
 npm install -g kimetsu-ai
@@ -80,12 +86,44 @@ absent, the extension silently no-ops and Pi is unaffected.
 pi install npm:kimetsu-pi
 ```
 
+From the Git project whose memory you want to use, initialize Kimetsu if that
+project does not already have a brain:
+
+```sh
+cd /path/to/your/project
+kimetsu init
+kimetsu brain status
+```
+
+Start Pi in that project, or run `/reload` in an existing Pi session to load the
+extension and skill. Confirm installation with `pi list`.
+
+## Upgrade
+
+For an installation that follows the latest npm version:
+
+```sh
+pi update npm:kimetsu-pi
+```
+
+For a pinned installation, select this version explicitly:
+
+```sh
+pi install npm:kimetsu-pi@0.1.4
+```
+
+Then restart Pi or run `/reload`. Pi package updates and Kimetsu CLI updates are
+separate; use `npm install -g kimetsu-ai@latest` if the CLI also needs updating.
+
 ## What it does on each event
 
 | Pi lifecycle event | Kimetsu command run |
 | --- | --- |
 | `session_start` | `kimetsu brain warm` |
 | `before_agent_start` | `kimetsu brain context-hook --warm-on-first-prompt` |
+| `context` | Filters earlier Kimetsu injections out of each model request |
+| `session_before_compact`, `session_before_tree` | Excludes Kimetsu injections from new summaries |
+| `session_tree` | Expires the previous branch's active injection |
 | `agent_end` | `kimetsu brain stop-hook` |
 | `session_shutdown` | `kimetsu brain session-end-hook` |
 
@@ -94,14 +132,52 @@ The extension uses Pi's `SessionManager` identity, so `/new`, `/resume`, and
 Pi's persisted JSONL transcript to the stop and session-end hooks, enabling
 Kimetsu's configured distiller and automatic work-episode capture.
 
+Only the current task's Kimetsu injection is sent to the model. Older injections
+remain in the saved transcript but are filtered from subsequent model requests,
+including when retrieval returns no context. Ordinary conversation and other
+extensions' messages are preserved. Queued user messages expire the previous
+injection; tool continuations keep it. New compaction and branch summaries omit
+raw Kimetsu injections. This does not rewrite earlier assistant answers or
+existing summaries.
+
+Session saving can make two model calls, so it may take longer than retrieval.
+Pi awaits it when closing or switching a session, for up to five minutes; it
+normally finishes sooner. The longer deadline applies only to
+`session-end-hook`. Prompt retrieval, warm-up, and `stop-hook` retain their
+10-second limits. Saving remains best-effort and needs a clean shutdown.
+
+The bundled skill teaches the agent to reuse relevant injected context, fetch
+missing evidence, correct existing memories, and cite only memories that
+actually helped. Run `kimetsu brain status` from the project directory to inspect
+the brain. Automatic model-based distillation requires configuration.
+
 ## Development
 
 ```sh
-npm install
+npm ci
 npm test
 npm run typecheck
 ```
 
+To run the optional CLI integration test, set `KIMETSU_TEST_BINARY` to the
+absolute path of a Kimetsu 2.8.0+ executable named `kimetsu` (`kimetsu.exe` on
+Windows), then run `npm run test:integration`. This command fails if the binary
+is missing. It creates an isolated temporary brain, uses
+lexical retrieval without model calls, and verifies correction, context
+filtering, citation, and session persistence.
+
+The fixture anchors a new Git repository and gives subprocesses temporary homes,
+caches, and user-brain paths. It removes inherited Git/Kimetsu overrides and
+credentials before initialization. Never run integration commands against your
+normal project or home brain.
+
+CI requires the real CLI test on Linux, macOS, and Windows using checksum-pinned
+Kimetsu 2.8.0 binaries and the minimum supported Node version. The `required checks`
+gate also requires typechecking, tests, the dependency audit, and package dry-run.
+Release tags must match the package version and point to a commit on `main` with
+successful CI. Wait for that commit's CI before tagging; a release started too
+early must be rerun after CI passes.
+
 ## Links
 
 - **Website & docs:** [kimetsu.dev](https://kimetsu.dev)

package/extensions/kimetsu.ts

@@ -14,11 +14,16 @@
 // crash, unparseable output. Kimetsu is a sidecar — it must never break Pi.
 
 import { spawn } from "node:child_process";
+import { randomUUID } from "node:crypto";
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 
-/** Hard cap on any single kimetsu invocation. A hung binary must not stall a turn. */
+/** Interactive hooks must not leave a turn waiting on a hung binary. */
 const EXEC_TIMEOUT_MS = 10000;
 
+/** Session saving may distill lessons and an episode in two model calls
+ *  (120s each by default). Leave time for both plus local persistence. */
+const SESSION_SAVE_TIMEOUT_MS = 300000;
+
 /** Fallback session id when Pi's context does not expose one. Stable per process,
  *  which is what the brain's per-session dedupe and refractory windows need. */
 const FALLBACK_SESSION_ID = `pi-${process.pid}`;
@@ -30,7 +35,7 @@ const FALLBACK_SESSION_ID = `pi-${process.pid}`;
  * stdout is PIPED, not ignored: the context hook communicates entirely through
  * it. stderr stays ignored so diagnostics never mix into the parsed payload.
  */
-function kimetsuRun(args: string[], input?: string): Promise<string> {
+function kimetsuRun(args: string[], input?: string, timeoutMs = EXEC_TIMEOUT_MS): Promise<string> {
   return new Promise((resolve) => {
     let settled = false;
     let timer: ReturnType<typeof setTimeout> | undefined;
@@ -52,7 +57,7 @@ function kimetsuRun(args: string[], input?: string): Promise<string> {
       timer = setTimeout(() => {
         child.kill();
         done();
-      }, EXEC_TIMEOUT_MS);
+      }, timeoutMs);
       if (typeof timer.unref === "function") timer.unref();
 
       child.stdout?.setEncoding("utf8");
@@ -147,11 +152,17 @@ function workspaceArgs(ctx: any): string[] {
 }
 
 export default function (pi: ExtensionAPI) {
+  // Pi persists injected messages, including display:false messages. Only the
+  // current task's injection belongs in future model calls. A unique marker
+  // survives Pi's message cloning without trusting content or timestamps.
+  let activeContext: { id: string; sessionId: string } | undefined;
+
   // session_start fires once when Pi starts up or a new session begins.
   // Warming spawns the embedder daemon so the first real retrieval is semantic
   // rather than falling back to lexical FTS.
   // (`brain warm` takes no --workspace: it resolves the project from its cwd.)
   pi.on("session_start", async (_event, _ctx) => {
+    activeContext = undefined;
     await kimetsuRun(["brain", "warm"]);
   });
 
@@ -161,14 +172,19 @@ export default function (pi: ExtensionAPI) {
   // --warm-on-first-prompt folds the repo digest and episodic resume into the
   // first turn of each session.
   pi.on("before_agent_start", async (event, ctx) => {
+    const request = { id: randomUUID(), sessionId: sessionIdOf(ctx) };
+    // Expire the last task immediately, even if retrieval is empty or fails.
+    activeContext = request;
     const payload = JSON.stringify({
-      session_id: sessionIdOf(ctx),
+      session_id: request.sessionId,
       prompt: typeof event?.prompt === "string" ? event.prompt : "",
     });
     const stdout = await kimetsuRun(
       ["brain", "context-hook", "--warm-on-first-prompt", ...workspaceArgs(ctx)],
       payload,
     );
+    // A session switch or a newer prompt can supersede an in-flight request.
+    if (activeContext !== request) return;
     const content = parseAdditionalContext(stdout);
     if (content === undefined) return; // nothing relevant — zero tokens
     return {
@@ -176,10 +192,64 @@ export default function (pi: ExtensionAPI) {
         customType: "kimetsu-brain",
         content,
         display: false,
+        details: { kimetsuContextId: request.id },
       },
     };
   });
 
+  pi.on("context", async (event, ctx) => {
+    const context = activeContext;
+    const currentIndex = context && context.sessionId === sessionIdOf(ctx)
+      ? event.messages.findIndex((message) => {
+        if (message.role !== "custom" || message.customType !== "kimetsu-brain") return false;
+        const details = message.details as { kimetsuContextId?: unknown } | undefined;
+        return details?.kimetsuContextId === context.id;
+      })
+      : -1;
+    // Queued steering/follow-up messages bypass before_agent_start. Expire
+    // the old injection when a newer user message arrives; tool results alone
+    // do not end the current task's context.
+    if (currentIndex >= 0 && event.messages.some((message, index) =>
+      index > currentIndex && message.role === "user"
+    )) activeContext = undefined;
+
+    // Filter the model's copy only; preserve the persisted session history.
+    return {
+      messages: event.messages.filter((message, index) =>
+        message.role !== "custom" || message.customType !== "kimetsu-brain"
+        || (activeContext !== undefined && index === currentIndex)
+      ),
+    };
+  });
+
+  pi.on("session_before_compact", async (event) => {
+    // Summarization bypasses the context event. Do not turn retrieved evidence
+    // into a durable summary that could outlive a correction or invalidation.
+    event.preparation.messagesToSummarize = event.preparation.messagesToSummarize.filter(
+      (message) => message.role !== "custom" || message.customType !== "kimetsu-brain",
+    );
+    event.preparation.turnPrefixMessages = event.preparation.turnPrefixMessages.filter(
+      (message) => message.role !== "custom" || message.customType !== "kimetsu-brain",
+    );
+  });
+
+  pi.on("session_before_tree", async (event) => {
+    // Pi retains a reference to this temporary summary input array, so filter
+    // in place. The persisted session entries themselves are left untouched.
+    const entries = event.preparation.entriesToSummarize;
+    let kept = 0;
+    for (const entry of entries) {
+      if (entry.type !== "custom_message" || entry.customType !== "kimetsu-brain") {
+        entries[kept++] = entry;
+      }
+    }
+    entries.length = kept;
+  });
+
+  pi.on("session_tree", async () => {
+    activeContext = undefined;
+  });
+
   // agent_end fires after the LLM turn completes (maps to Kimetsu stop-hook).
   pi.on("agent_end", async (event, ctx) => {
     await kimetsuRun(
@@ -190,9 +260,11 @@ export default function (pi: ExtensionAPI) {
 
   // session_shutdown fires on clean session close (maps to session-end-hook).
   pi.on("session_shutdown", async (_event, ctx) => {
+    activeContext = undefined;
     await kimetsuRun(
       ["brain", "session-end-hook", ...workspaceArgs(ctx)],
       lifecyclePayload(ctx),
+      SESSION_SAVE_TIMEOUT_MS,
     );
   });
 }

package/package.json

@@ -1,28 +1,53 @@
 {
   "name": "kimetsu-pi",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Kimetsu brain as a Pi.dev package — local-first, cross-session memory for the Pi coding agent that gets sharper every run.",
-  "keywords": ["pi-package", "pi", "kimetsu", "memory", "brain", "rag", "mcp", "extension", "skill"],
+  "keywords": [
+    "pi-package",
+    "pi",
+    "kimetsu",
+    "memory",
+    "brain",
+    "rag",
+    "mcp",
+    "extension",
+    "skill"
+  ],
   "homepage": "https://kimetsu.dev",
-  "repository": { "type": "git", "url": "git+https://github.com/RodCor/kimetsu-pi.git" },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/RodCor/kimetsu-pi.git"
+  },
   "license": "MIT OR Apache-2.0",
   "type": "module",
   "pi": {
-    "extensions": ["./extensions"],
-    "skills": ["./skills"],
+    "extensions": [
+      "./extensions"
+    ],
+    "skills": [
+      "./skills"
+    ],
     "image": "https://raw.githubusercontent.com/RodCor/kimetsu/main/docs/assets/kimetsu-logo.png"
   },
-  "files": ["extensions/", "skills/", "README.md", "LICENSE"],
-  "engines": { "node": ">=22.19.0" },
+  "files": [
+    "extensions/",
+    "skills/",
+    "README.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=22.19.0"
+  },
   "scripts": {
     "typecheck": "tsc --noEmit",
-    "test": "vitest run"
+    "test": "vitest run",
+    "test:integration": "node scripts/test-integration.mjs"
   },
   "devDependencies": {
-    "@earendil-works/pi-coding-agent": "^0.84.2",
+    "@earendil-works/pi-coding-agent": "^0.85.1",
     "@types/node": "^26.0.0",
     "typescript": "^7.0.0",
-    "vitest": "^4.1.10"
+    "vitest": "5.0.0"
   },
   "peerDependencies": {
     "@earendil-works/pi-coding-agent": "*"

package/skills/kimetsu-brain/SKILL.md

@@ -1,21 +1,64 @@
 ---
 name: kimetsu-brain
-description: Use Kimetsu brain shell commands as a persistent memory sidecar across Pi sessions.
+description: Use when Pi tasks benefit from prior session knowledge, durable lessons, memory corrections, or feedback on helpful memories.
 ---
-Kimetsu is a persistent brain sidecar accessible via the `kimetsu` CLI. Use it
-when the task may benefit from prior session knowledge, workflow memory, or
-durable cross-session context.
-
-Brain-first workflow:
-1. Before planning or editing broad coding, review, debugging, or setup tasks,
-   run `kimetsu brain context <query>` and read the returned capsules as working
-   context before deciding on a plan.
-2. After solving a non-obvious problem, run
-   `kimetsu brain memory add --scope project --kind <kind> "<lesson>"` with a
-   concrete, actionable lesson. Choose `fact`, `preference`, `convention`,
-   `command`, or `failure_pattern` for `<kind>`.
-3. Run `kimetsu brain status` when you need to know whether the brain is
-   initialized, has accepted memories, or has pending proposals.
-
-Optional mode: Kimetsu brain context is a preferred first step for non-trivial
-work. If the binary is unavailable, note the absence and continue normally.
+Kimetsu is a persistent memory sidecar accessed through the `kimetsu` CLI.
+Run commands from the relevant project directory. If the binary is unavailable,
+note the absence and continue normally.
+
+## Use the context already provided
+
+The Pi extension retrieves context before each task. Read that injection first.
+When it covers the current question, proceed without repeating the same lookup.
+Use `kimetsu brain context "<specific question>"` when no useful context was
+injected, the task changes, or a missing detail or correction needs fresh evidence.
+An empty result is a reason to inspect the repository, not repeat the same query.
+
+Memory is evidence from earlier work. Check conflicts against current files and
+the user's instructions. Respect project, environment, and version boundaries;
+partial or conflicting evidence does not justify filling gaps with assumptions.
+
+## Record and correct durable lessons
+
+After verifying a reusable lesson, record it with
+`kimetsu brain memory add --scope project --kind <kind> "<lesson>"`.
+Choose `fact`, `preference`, `convention`, `command`, or `failure_pattern`.
+Include the subject and any environment/version limits in the text.
+
+For a correction to an existing claim, update its actual memory ID instead of
+adding a contradictory duplicate:
+
+```sh
+kimetsu brain memory edit <memory-id> --text "Production gateway port is 4000. Development gateway port remains 3000."
+```
+
+Preserve still-valid parts of the claim. Different environments or historical
+versions can both be valid; a production correction does not retire development
+guidance. When the entire memory is obsolete or false, use
+`kimetsu brain memory invalidate <memory-id> --reason "<verified reason>"`.
+
+Find actual IDs with `kimetsu brain context "<specific question>" --json` or
+`kimetsu brain memory list --json`; match the text and scope before editing,
+invalidating, or citing. For a `memory:<id>` expansion handle, use only `<id>`.
+Never invent an ID or treat a file capsule as a memory.
+The edit/invalidate commands above operate on the current workspace brain.
+Listings may also include portable user-brain memories; those commands cannot
+correct them. For a user-brain claim, report the correction and this limitation
+instead of claiming the portable memory was updated.
+
+## Credit explicit usefulness
+
+When a particular memory materially helped, record that reliance:
+
+```sh
+kimetsu brain cite --memory-id <memory-id> --query "<task it helped with>" --note "<how it helped>"
+```
+
+Cite only memories actually used. Being injected, or having tests pass, is not
+enough to credit a memory. A citation records usefulness, not proof of truth;
+correct wrong claims using the commands above. Do not cite unused memories or
+repeat credit for the same use.
+
+`kimetsu brain status` reports initialization, accepted memories, and pending
+proposals. Automatic session saving is separate from these deliberate actions;
+model-based lesson distillation requires a configured distiller.