pi-hermes-memory 0.7.22 → 0.8.0

package/README.md

@@ -339,6 +339,19 @@ Background review triggers based on **activity level**, not just turn count:
 
 Both counters reset after each review.
 
+### Direct-Transport LLM Calls (Review, Flush, Correction, Consolidation)
+
+By default, background review, session flush, correction save, and the manual `/memory-consolidate` command use an in-process `completeSimple()` side-channel: a small JSON-only prompt, no child `pi` process, and memory writes applied directly by the extension. This keeps the main session's system prompt, tools, and LLM prefix cache intact, and avoids the subprocess path's argv/`--no-extensions` concerns entirely on the common path.
+
+If direct mode fails (no model, no auth, provider error, unparseable response, or — for consolidation only — a result that didn't actually free any space), it automatically falls back to the legacy `pi -p --no-session` subprocess path. The automatic over-capacity consolidator triggered from `MemoryStore` itself always uses the subprocess path, since it runs without extension-runtime access.
+
+Set `reviewTransport` in config only when you need to override this:
+
+| Value | Behavior |
+|---|---|
+| `direct` (default) | Try in-process `completeSimple()` first; fall back to subprocess on failure |
+| `subprocess` | Always use `pi -p` subprocess for every LLM-driven memory operation (pre-PR #92 behavior) |
+
 ### Skill Auto-Extraction
 
 After a complex task (8+ tool calls using 2+ different tools in a single turn), the extension automatically asks the agent:
@@ -430,6 +443,7 @@ Create `~/.pi/agent/hermes-memory-config.json`:
   "nudgeToolCalls": 15,
   "reviewRecentMessages": 0,
   "reviewEnabled": true,
+  "reviewTransport": "direct",
   "memoryOverflowStrategy": "auto-consolidate",
   "autoConsolidate": true,
   "correctionDetection": true,
@@ -455,12 +469,14 @@ Create `~/.pi/agent/hermes-memory-config.json`:
 | `memoryDir` | `~/.pi/agent/pi-hermes-memory` | Custom directory for extension storage files |
 | `projectsMemoryDir` | `projects-memory` | Subdirectory under `~/.pi/agent/` for project-scoped memory |
 | `sessionSearch` | `{ "variant": "legacy" }` | Session search implementation: `legacy` keeps the existing SQLite/FTS snippet search; `anchors` uses the opt-in Markdown request surface and returns compact JSONL line-range anchors from `~/.pi/agent/sessions/` |
-| `llmModelOverride` | unset | Optional model override for child `pi -p` subprocess calls used by background review, correction save, session flush, and consolidation |
-| `llmThinkingOverride` | unset | Optional thinking override for those child subprocess calls; valid values are `off`, `minimal`, `low`, `medium`, `high`, and `xhigh`. If `llmModelOverride` is set and this is omitted, child calls default to `off` |
+| `llmModelOverride` | unset | Optional model override for background review (direct and subprocess), correction save, session flush, and consolidation |
+| `llmThinkingOverride` | unset | Optional thinking override for those LLM calls; valid values are `off`, `minimal`, `low`, `medium`, `high`, and `xhigh`. If `llmModelOverride` is set and this is omitted, review/child calls default to `off` |
+| `childExtensionPaths` | unset | Trusted provider/auth adapter entry paths explicitly allowed in isolated child Pi processes; sibling packages matching the `*-oauth-adapter`/`*-auth-adapter` naming convention (including scoped packages, via their `package.json` `pi.extensions` manifest) are detected automatically — this setting is only needed for adapters that don't match that convention. In-process direct transport (the default for review/flush/correction/consolidation) doesn't need this at all, since it reads whatever provider auth is already registered |
 | `nudgeInterval` | `10` | Turns between auto-reviews |
 | `nudgeToolCalls` | `15` | Tool calls between auto-reviews (OR with turns) |
 | `reviewRecentMessages` | `0` | Recent messages included in background review (`0` = all) |
 | `reviewEnabled` | `true` | Enable/disable background learning loop |
+| `reviewTransport` | `direct` | LLM transport for background review, session flush, correction save, and manual consolidation: `direct` uses in-process `completeSimple()` with subprocess fallback; `subprocess` forces legacy `pi -p` only |
 | `memoryOverflowStrategy` | `auto-consolidate` | Behavior when MEMORY.md, USER.md, failures.md, or project-scoped memory reaches its character limit: `auto-consolidate` runs the existing consolidation flow; `reject` returns an error; `fifo-evict` rotates older entries in file order until the new entry fits |
 | `autoConsolidate` | `true` | Legacy alias for `memoryOverflowStrategy` when `memoryOverflowStrategy` is not set (`true` = `auto-consolidate`, `false` = `reject`) |
 | `consolidationTimeoutMs` | `60000` | Maximum time in milliseconds for auto-consolidation to complete |

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "pi-hermes-memory",
-  "version": "0.7.22",
-  "description": "🧠 Persistent memory + 🔍 session search + 🛡️ secret scanning for Pi. Token-aware policy-only memory by default, SQLite FTS5 search, auto-consolidation, procedural skills. 368 tests. Ported from Hermes agent.",
+  "version": "0.8.0",
+  "description": "🧠 Persistent memory + 🔍 session search + 🛡️ secret scanning for Pi. Token-aware policy-only memory by default, SQLite FTS5 search, auto-consolidation, procedural skills. 693 tests. Ported from Hermes agent.",
   "type": "module",
   "main": "src/index.ts",
   "files": [
@@ -45,6 +45,7 @@
     "url": "https://github.com/chandra447/pi-hermes-memory"
   },
   "peerDependencies": {
+    "@earendil-works/pi-ai": "*",
     "@earendil-works/pi-coding-agent": ">=0.74.0"
   },
   "devDependencies": {

package/src/config.ts

@@ -1,6 +1,6 @@
 import * as fs from "node:fs";
 import * as path from "node:path";
-import type { MemoryConfig, MemoryOverflowStrategy, SessionSearchVariant, ThinkingLevel } from "./types.js";
+import type { MemoryConfig, MemoryOverflowStrategy, ReviewTransport, SessionSearchVariant, ThinkingLevel } from "./types.js";
 import {
   DEFAULT_MEMORY_CHAR_LIMIT,
   DEFAULT_USER_CHAR_LIMIT,
@@ -19,8 +19,13 @@ import { AGENT_ROOT, normalizeConfiguredMemoryDir, normalizeProjectsMemoryDir }
 
 const MEMORY_OVERFLOW_STRATEGIES: readonly MemoryOverflowStrategy[] = ["auto-consolidate", "reject", "fifo-evict"];
 const SESSION_SEARCH_VARIANTS: readonly SessionSearchVariant[] = ["legacy", "anchors"];
+const REVIEW_TRANSPORTS: readonly ReviewTransport[] = ["direct", "subprocess"];
 const THINKING_LEVELS: readonly ThinkingLevel[] = ["off", "minimal", "low", "medium", "high", "xhigh"];
 
+function isReviewTransport(value: unknown): value is ReviewTransport {
+  return typeof value === "string" && REVIEW_TRANSPORTS.includes(value as ReviewTransport);
+}
+
 function isMemoryOverflowStrategy(value: unknown): value is MemoryOverflowStrategy {
   return typeof value === "string" && MEMORY_OVERFLOW_STRATEGIES.includes(value as MemoryOverflowStrategy);
 }
@@ -42,6 +47,7 @@ const DEFAULT_CONFIG: MemoryConfig = {
   nudgeInterval: DEFAULT_NUDGE_INTERVAL,
   reviewRecentMessages: DEFAULT_REVIEW_RECENT_MESSAGES,
   reviewEnabled: true,
+  reviewTransport: "direct",
   flushOnCompact: true,
   flushOnShutdown: true,
   flushMinTurns: DEFAULT_FLUSH_MIN_TURNS,
@@ -91,6 +97,7 @@ export function loadConfig(configPath = DEFAULT_CONFIG_PATH): MemoryConfig {
       if (typeof parsed.nudgeInterval === "number") config.nudgeInterval = parsed.nudgeInterval;
       if (isNonNegativeNumber(parsed.reviewRecentMessages)) config.reviewRecentMessages = parsed.reviewRecentMessages;
       if (typeof parsed.reviewEnabled === "boolean") config.reviewEnabled = parsed.reviewEnabled;
+      if (isReviewTransport(parsed.reviewTransport)) config.reviewTransport = parsed.reviewTransport;
       if (typeof parsed.flushOnCompact === "boolean") config.flushOnCompact = parsed.flushOnCompact;
       if (typeof parsed.flushOnShutdown === "boolean") config.flushOnShutdown = parsed.flushOnShutdown;
       if (typeof parsed.flushMinTurns === "number") config.flushMinTurns = parsed.flushMinTurns;
@@ -134,6 +141,12 @@ export function loadConfig(configPath = DEFAULT_CONFIG_PATH): MemoryConfig {
         if (trimmed.length > 0) config.llmModelOverride = trimmed;
       }
       if (isThinkingLevel(parsed.llmThinkingOverride)) config.llmThinkingOverride = parsed.llmThinkingOverride;
+      if (isStringArray(parsed.childExtensionPaths)) {
+        const childExtensionPaths = [...new Set<string>(
+          (parsed.childExtensionPaths as string[]).map((item) => item.trim()).filter(Boolean),
+        )];
+        if (childExtensionPaths.length > 0) config.childExtensionPaths = childExtensionPaths;
+      }
       if (hasMemoryOverflowStrategy) {
         config.autoConsolidate = config.memoryOverflowStrategy === "auto-consolidate";
       } else if (hasLegacyAutoConsolidate) {

package/src/constants.ts

@@ -145,6 +145,78 @@ For failures, include: what was tried, why it failed, what error occurred, and w
 
 Only act if there's something genuinely worth saving. If nothing stands out, just say 'Nothing to save.' and stop.`;
 
+// ─── Shared JSON operations schema for direct (in-process) completions ───
+// (review/flush/consolidation/correction all ask the model to respond with
+// this same {"operations":[...]} shape instead of calling the memory tool,
+// since direct mode is a single completeSimple() call with no tool loop).
+const DIRECT_MEMORY_OPERATIONS_SCHEMA = `Respond with JSON only (no markdown fences):
+{
+  "operations": [
+    {
+      "action": "add",
+      "target": "memory",
+      "content": "entry text"
+    }
+  ]
+}
+
+Operation fields:
+- action: "add" | "replace" | "remove"
+- target: "memory" | "user" | "project" | "failure"
+- content: required for add/replace
+- old_text: required for replace/remove (substring match)
+- category: for failure target — failure | correction | insight | convention | tool-quirk | preference
+- failure_reason: optional context for failure entries`;
+
+export const DIRECT_REVIEW_SYSTEM_PROMPT = `You review coding conversations and extract durable memories worth saving across sessions.
+
+Review these aspects:
+- **Memory**: User persona, preferences, expectations about how the agent should behave, work style.
+- **Failures & Corrections**: What failed, user corrections, insights, conventions, tool quirks.
+
+Do NOT create or modify skills. Only save genuinely durable facts — not task progress, session outcomes, or temporary state.
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}
+
+If nothing is worth saving, return {"operations":[]}.`;
+
+// ─── Direct (in-process) flush prompt — used by session-flush.ts when the
+// session is about to lose context (compaction/shutdown). ───
+export const DIRECT_FLUSH_SYSTEM_PROMPT = `The session is being compressed and about to lose context. Save anything worth remembering from the conversation — prioritize user preferences, corrections, and recurring patterns over task-specific details.
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}
+
+If nothing is worth saving, return {"operations":[]}.`;
+
+// ─── Direct (in-process) consolidation prompt — used by auto-consolidate.ts.
+// Unlike review/flush/correction, a single triggerConsolidation() call is
+// always scoped to exactly one target/store, passed via the user prompt. ───
+export const DIRECT_CONSOLIDATION_SYSTEM_PROMPT = `The memory store you're given is at capacity. Consolidate its current entries:
+- Merge related entries into a single, concise entry
+- Remove outdated or superseded entries (entries older than 30 days without recent references are candidates for removal)
+- Keep the most important and frequently-referenced facts
+- Preserve user preferences and corrections (highest priority)
+
+Each entry shows when it was created and last referenced in HTML comments (<!-- created=..., last=... -->). Use this to identify stale entries.
+
+Express a merge as "remove" operations for the entries being dropped plus one "add" operation for the new merged entry. Be aggressive about merging — less is more. Every operation MUST use the exact target given to you in the user message; do not touch any other target.
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}`;
+
+// ─── Direct (in-process) correction-save prompt — used by correction-detector.ts. ───
+export const DIRECT_CORRECTION_SYSTEM_PROMPT = `The user just corrected the agent. Review what went wrong and decide what durable memory to save.
+
+Priority:
+1. User preference ("don't do X", "always use Y instead")
+2. Wrong assumption the agent made
+3. Environment fact the agent got wrong
+
+If this contradicts an existing entry, use a "replace" operation to update it instead of "add".
+
+${DIRECT_MEMORY_OPERATIONS_SCHEMA}
+
+If nothing is worth saving beyond the automatic failure-memory capture, return {"operations":[]}.`;
+
 // ─── Flush prompt (ported from flush_memories() in run_agent.py ~L7379) ───
 export const FLUSH_PROMPT = `[System: The session is being compressed. Save anything worth remembering — prioritize user preferences, corrections, and recurring patterns over task-specific details.]`;