pi-hermes-memory 0.7.23 → 0.8.1

package/README.md

@@ -92,6 +92,18 @@ Every write — memory and skills — passes through a scanner before being acce
 
 ![Security: Content Scanning](docs/images/security-flow.svg)
 
+## Development
+
+`npm run check` and `npm test` only work from a **full git checkout** after `npm install`. The published npm package intentionally omits `tests/`, TypeScript, and `tsconfig.json` (production install for Pi). Validate from source or rely on CI before publish.
+
+```bash
+git clone https://github.com/chandra447/pi-hermes-memory.git
+cd pi-hermes-memory
+npm install
+npm run check
+npm test
+```
+
 ## Installation
 
 ```bash
@@ -339,18 +351,18 @@ Background review triggers based on **activity level**, not just turn count:
 
 Both counters reset after each review.
 
-### Background Review Transport
+### Direct-Transport LLM Calls (Review, Flush, Correction, Consolidation)
 
-By default, background review uses an in-process `completeSimple()` side-channel: a small JSON-only prompt, no child `pi` process, and memory writes applied directly by the extension. This keeps the main session's system prompt, tools, and LLM prefix cache intact.
+By default, background review, session flush, correction save, and the manual `/memory-consolidate` command use an in-process `completeSimple()` side-channel: a small JSON-only prompt, no child `pi` process, and memory writes applied directly by the extension. This keeps the main session's system prompt, tools, and LLM prefix cache intact, and avoids the subprocess path's argv/`--no-extensions` concerns entirely on the common path.
 
-If direct review fails (no model, no auth, provider error, unparseable response), it automatically falls back to the legacy `pi -p --no-session` subprocess path.
+If direct mode fails (no model, no auth, provider error, unparseable response, or — for consolidation only — a result that didn't actually free any space), it automatically falls back to the legacy `pi -p --no-session` subprocess path. The automatic over-capacity consolidator triggered from `MemoryStore` itself always uses the subprocess path, since it runs without extension-runtime access.
 
 Set `reviewTransport` in config only when you need to override this:
 
 | Value | Behavior |
 |---|---|
 | `direct` (default) | Try in-process `completeSimple()` first; fall back to subprocess on failure |
-| `subprocess` | Always use `pi -p` subprocess (pre-PR #92 behavior) |
+| `subprocess` | Always use `pi -p` subprocess for every LLM-driven memory operation (pre-PR #92 behavior) |
 
 ### Skill Auto-Extraction
 
@@ -471,11 +483,12 @@ Create `~/.pi/agent/hermes-memory-config.json`:
 | `sessionSearch` | `{ "variant": "legacy" }` | Session search implementation: `legacy` keeps the existing SQLite/FTS snippet search; `anchors` uses the opt-in Markdown request surface and returns compact JSONL line-range anchors from `~/.pi/agent/sessions/` |
 | `llmModelOverride` | unset | Optional model override for background review (direct and subprocess), correction save, session flush, and consolidation |
 | `llmThinkingOverride` | unset | Optional thinking override for those LLM calls; valid values are `off`, `minimal`, `low`, `medium`, `high`, and `xhigh`. If `llmModelOverride` is set and this is omitted, review/child calls default to `off` |
+| `childExtensionPaths` | unset | Trusted provider/auth adapter entry paths explicitly allowed in isolated child Pi processes; sibling packages matching the `*-oauth-adapter`/`*-auth-adapter` naming convention (including scoped packages, via their `package.json` `pi.extensions` manifest) are detected automatically — this setting is only needed for adapters that don't match that convention. In-process direct transport (the default for review/flush/correction/consolidation) doesn't need this at all, since it reads whatever provider auth is already registered |
 | `nudgeInterval` | `10` | Turns between auto-reviews |
 | `nudgeToolCalls` | `15` | Tool calls between auto-reviews (OR with turns) |
 | `reviewRecentMessages` | `0` | Recent messages included in background review (`0` = all) |
 | `reviewEnabled` | `true` | Enable/disable background learning loop |
-| `reviewTransport` | `direct` | Background review LLM transport: `direct` uses in-process `completeSimple()` with subprocess fallback; `subprocess` forces legacy `pi -p` only |
+| `reviewTransport` | `direct` | LLM transport for background review, session flush, correction save, and manual consolidation: `direct` uses in-process `completeSimple()` with subprocess fallback; `subprocess` forces legacy `pi -p` only |
 | `memoryOverflowStrategy` | `auto-consolidate` | Behavior when MEMORY.md, USER.md, failures.md, or project-scoped memory reaches its character limit: `auto-consolidate` runs the existing consolidation flow; `reject` returns an error; `fifo-evict` rotates older entries in file order until the new entry fits |
 | `autoConsolidate` | `true` | Legacy alias for `memoryOverflowStrategy` when `memoryOverflowStrategy` is not set (`true` = `auto-consolidate`, `false` = `reject`) |
 | `consolidationTimeoutMs` | `60000` | Maximum time in milliseconds for auto-consolidation to complete |

package/package.json

@@ -1,11 +1,12 @@
 {
   "name": "pi-hermes-memory",
-  "version": "0.7.23",
-  "description": "🧠 Persistent memory + 🔍 session search + 🛡️ secret scanning for Pi. Token-aware policy-only memory by default, SQLite FTS5 search, auto-consolidation, procedural skills. 368 tests. Ported from Hermes agent.",
+  "version": "0.8.1",
+  "description": "🧠 Persistent memory + 🔍 session search + 🛡️ secret scanning for Pi. Token-aware policy-only memory by default, SQLite FTS5 search, auto-consolidation, procedural skills. 693 tests. Ported from Hermes agent.",
   "type": "module",
   "main": "src/index.ts",
   "files": [
     "src",
+    "scripts",
     "README.md",
     "LICENSE",
     "docs"
@@ -16,8 +17,8 @@
     ]
   },
   "scripts": {
-    "check": "tsc --noEmit",
-    "test": "tests/run-all.sh"
+    "check": "node scripts/ensure-dev.mjs check && tsc --noEmit",
+    "test": "node scripts/ensure-dev.mjs test && tests/run-all.sh"
   },
   "keywords": [
     "pi-package",

package/scripts/ensure-dev.mjs

@@ -0,0 +1,57 @@
+#!/usr/bin/env node
+/**
+ * Guard for check/test npm scripts.
+ *
+ * The published tarball intentionally omits tests/ and TypeScript (devDependency).
+ * When those scripts are run from an installed package (e.g. after `pi install`),
+ * fail with a clear message instead of `tsc: not found` / `tests/run-all.sh: not found`.
+ *
+ * @see https://github.com/chandra447/pi-hermes-memory/issues/108
+ */
+import { existsSync } from "node:fs";
+import { createRequire } from "node:module";
+import { dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const root = join(dirname(fileURLToPath(import.meta.url)), "..");
+const mode = process.argv[2];
+const REPO = "https://github.com/chandra447/pi-hermes-memory";
+
+function fail(message) {
+  console.error(`\n[pi-hermes-memory] ${message}`);
+  console.error("");
+  console.error("These scripts only work from a full source checkout:");
+  console.error(`  git clone ${REPO}.git`);
+  console.error("  cd pi-hermes-memory && npm install");
+  console.error("  npm run check   # or: npm test");
+  console.error("");
+  console.error(
+    "The published package is for runtime use via Pi; CI validates check/test before publish.",
+  );
+  console.error("");
+  process.exit(1);
+}
+
+if (mode === "check") {
+  const require = createRequire(join(root, "package.json"));
+  try {
+    require.resolve("typescript/package.json");
+  } catch {
+    fail(
+      "`npm run check` requires TypeScript (a devDependency). The published npm package does not include it.",
+    );
+  }
+  if (!existsSync(join(root, "tsconfig.json"))) {
+    fail(
+      "`npm run check` requires tsconfig.json. The published npm package does not include it.",
+    );
+  }
+} else if (mode === "test") {
+  if (!existsSync(join(root, "tests", "run-all.sh"))) {
+    fail(
+      "`npm test` requires the tests/ directory. The published npm package intentionally omits it.",
+    );
+  }
+} else {
+  fail(`Unknown mode "${mode ?? ""}". Expected "check" or "test".`);
+}

package/src/config.ts

@@ -141,6 +141,12 @@ export function loadConfig(configPath = DEFAULT_CONFIG_PATH): MemoryConfig {
         if (trimmed.length > 0) config.llmModelOverride = trimmed;
       }
       if (isThinkingLevel(parsed.llmThinkingOverride)) config.llmThinkingOverride = parsed.llmThinkingOverride;
+      if (isStringArray(parsed.childExtensionPaths)) {
+        const childExtensionPaths = [...new Set<string>(
+          (parsed.childExtensionPaths as string[]).map((item) => item.trim()).filter(Boolean),
+        )];
+        if (childExtensionPaths.length > 0) config.childExtensionPaths = childExtensionPaths;
+      }
       if (hasMemoryOverflowStrategy) {
         config.autoConsolidate = config.memoryOverflowStrategy === "auto-consolidate";
       } else if (hasLegacyAutoConsolidate) {

package/src/constants.ts

@@ -145,16 +145,11 @@ For failures, include: what was tried, why it failed, what error occurred, and w
 
 Only act if there's something genuinely worth saving. If nothing stands out, just say 'Nothing to save.' and stop.`;
 
-// ─── Direct (in-process) background review prompts ───
-export const DIRECT_REVIEW_SYSTEM_PROMPT = `You review coding conversations and extract durable memories worth saving across sessions.
-
-Review these aspects:
-- **Memory**: User persona, preferences, expectations about how the agent should behave, work style.
-- **Failures & Corrections**: What failed, user corrections, insights, conventions, tool quirks.
-
-Do NOT create or modify skills. Only save genuinely durable facts — not task progress, session outcomes, or temporary state.
-
-Respond with JSON only (no markdown fences):
+// ─── Shared JSON operations schema for direct (in-process) completions ───
+// (review/flush/consolidation/correction all ask the model to respond with
+// this same {"operations":[...]} shape instead of calling the memory tool,
+// since direct mode is a single completeSimple() call with no tool loop).
+const DIRECT_MEMORY_OPERATIONS_SCHEMA = `Respond with JSON only (no markdown fences):
 {
   "operations": [
     {
@@ -171,10 +166,57 @@ Operation fields:
 - content: required for add/replace
 - old_text: required for replace/remove (substring match)
 - category: for failure target — failure | correction | insight | convention | tool-quirk | preference
-- failure_reason: optional context for failure entries
+- failure_reason: optional context for failure entries`;
+
+export const DIRECT_REVIEW_SYSTEM_PROMPT = `You review coding conversations and extract durable memories worth saving across sessions.
+
+Review these aspects:
+- **Memory**: User persona, preferences, expectations about how the agent should behave, work style.
+- **Failures & Corrections**: What failed, user corrections, insights, conventions, tool quirks.
+
+Do NOT create or modify skills. Only save genuinely durable facts — not task progress, session outcomes, or temporary state.
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}
 
 If nothing is worth saving, return {"operations":[]}.`;
 
+// ─── Direct (in-process) flush prompt — used by session-flush.ts when the
+// session is about to lose context (compaction/shutdown). ───
+export const DIRECT_FLUSH_SYSTEM_PROMPT = `The session is being compressed and about to lose context. Save anything worth remembering from the conversation — prioritize user preferences, corrections, and recurring patterns over task-specific details.
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}
+
+If nothing is worth saving, return {"operations":[]}.`;
+
+// ─── Direct (in-process) consolidation prompt — used by auto-consolidate.ts.
+// Unlike review/flush/correction, a single triggerConsolidation() call is
+// always scoped to exactly one target/store, passed via the user prompt. ───
+export const DIRECT_CONSOLIDATION_SYSTEM_PROMPT = `The memory store you're given is at capacity. Consolidate its current entries:
+- Merge related entries into a single, concise entry
+- Remove outdated or superseded entries (entries older than 30 days without recent references are candidates for removal)
+- Keep the most important and frequently-referenced facts
+- Preserve user preferences and corrections (highest priority)
+
+Each entry shows when it was created and last referenced in HTML comments (<!-- created=..., last=... -->). Use this to identify stale entries.
+
+Express a merge as "remove" operations for the entries being dropped plus one "add" operation for the new merged entry. Be aggressive about merging — less is more. Every operation MUST use the exact target given to you in the user message; do not touch any other target.
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}`;
+
+// ─── Direct (in-process) correction-save prompt — used by correction-detector.ts. ───
+export const DIRECT_CORRECTION_SYSTEM_PROMPT = `The user just corrected the agent. Review what went wrong and decide what durable memory to save.
+
+Priority:
+1. User preference ("don't do X", "always use Y instead")
+2. Wrong assumption the agent made
+3. Environment fact the agent got wrong
+
+If this contradicts an existing entry, use a "replace" operation to update it instead of "add".
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}
+
+If nothing is worth saving beyond the automatic failure-memory capture, return {"operations":[]}.`;
+
 // ─── Flush prompt (ported from flush_memories() in run_agent.py ~L7379) ───
 export const FLUSH_PROMPT = `[System: The session is being compressed. Save anything worth remembering — prioritize user preferences, corrections, and recurring patterns over task-specific details.]`;