@equationalapplications/core-llm-wiki 4.7.0 → 4.9.0

package/README.md

@@ -2,6 +2,10 @@
 
 Pure TypeScript business logic for LLM Wiki Memory.
 
+[![npm version](https://img.shields.io/npm/v/%40equationalapplications%2Fcore-llm-wiki?label=core)](https://www.npmjs.com/package/@equationalapplications/core-llm-wiki) [![npm downloads](https://img.shields.io/npm/dm/%40equationalapplications%2Fcore-llm-wiki?label=downloads)](https://www.npmjs.com/package/@equationalapplications/core-llm-wiki)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.x-3178C6?logo=typescript&logoColor=white)](https://www.typescriptlang.org/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+
 > Inspired by [Andrej Karpathy's LLM Wiki memory spec](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).
 
 ## Features
@@ -92,10 +96,87 @@ const wikiMemory = new WikiMemory(db, {
     staleInferredAfterDays: 60,        // default: 60 (days before runHeal downgrades inferred facts; null to disable)
     preFilterLimit: 50,                // default: undefined — MiniSearch pre-filter before cosine scan; recommended for >500 facts
     hybridWeight: 0.7,                 // default: undefined — blend semantic (1.0) ↔ keyword (0.0); pure semantic when unset
+    enableOutbox: false,               // default: false — when true, entry/task mutations write to an internal SQLite outbox table for external sync (e.g. via @equationalapplications/prisma-outbox)
+
+    // Global prompt overrides — librarianSystemPrompt and healSystemPrompt apply to write() auto-runs;
+    // ingestSystemPrompt applies only to explicit ingestDocument() calls.
+    // ⚠ Overrides replace the entire default prompt, including the JSON output contract.
+    // See "JSON Output Contracts" in the Prompt Management & Overrides section below.
+    prompts: {
+      ingestSystemPrompt: `Extract core facts from this document: {{documentChunk}}\n\nReturn ONLY valid JSON: { "facts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }] }. No markdown.`,
+      librarianSystemPrompt: `You are an expert curator. Synthesize these thoughts:\n{{events}}\n\nCurrent Facts:\n{{currentFacts}}\n\nReturn ONLY valid JSON: { "facts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }], "tasks": [{ "description": "string", "priority": 0 }] }. No markdown.`,
+      healSystemPrompt: `Fix the memory graph based on these candidates: {{healCandidates}}\n\nReturn ONLY valid JSON: { "downgraded": ["factId"], "deleted": ["factId"], "newFacts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }] }. No markdown.`,
+    },
+  },
+});
+```
+
+## Prompt Management & Overrides
+
+Core maintenance tasks (`ingestDocument`, `runLibrarian`, `runHeal`) use system prompts to instruct the LLM. You can customize these prompts using `{{mustache}}` style variables to inject context dynamically.
+
+> **JSON Output Contracts:** Prompt overrides replace the *entire* default system prompt, including the JSON response schema the parser depends on. Your override **must** instruct the LLM to return raw JSON — no markdown. Required shapes:
+>
+> | Operation | Required JSON shape |
+> |-----------|-------------------|
+> | `ingestDocument` | `{ "facts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain\|inferred\|tentative" }] }` |
+> | `runLibrarian` | `{ "facts": [...], "tasks": [{ "description": "string", "priority": 5 }] }` — `priority` is an integer 0–10 |
+> | `runHeal` | `{ "downgraded": ["factId"], "deleted": ["factId"], "newFacts": [...] }` |
+
+### Global Overrides (Auto-Runs)
+
+If your application relies on `write()` to automatically maintain the memory graph in the background (via `autoLibrarianThreshold` and `autoHealThreshold`), configure custom prompts globally at instantiation. This ensures the internal `WriteService` uses your domain-specific instructions when it triggers an auto-run.
+
+```typescript
+const wikiMemory = new WikiMemory(db, {
+  llmProvider,
+  config: {
+    prompts: {
+      // Override must include the JSON output contract — it replaces the entire default prompt.
+      librarianSystemPrompt: `You are an expert curator. Synthesize these thoughts:\n{{events}}\n\nCurrent Facts:\n{{currentFacts}}\n\nReturn ONLY a valid JSON object: { "facts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }], "tasks": [{ "description": "string", "priority": 0 }] }. No markdown.`,
+    },
   },
 });
+
+// WriteService uses the global prompt whenever autoLibrarianThreshold is hit
+await wikiMemory.write('user-123', { event_type: 'observation', summary: '...' });
 ```
 
+Available `{{variables}}` per prompt type:
+
+| Prompt | Variables |
+|--------|-----------|
+| `ingestSystemPrompt` | `{{documentChunk}}` |
+| `librarianSystemPrompt` | `{{events}}`, `{{currentFacts}}` |
+| `healSystemPrompt` | `{{healCandidates}}`, `{{documentAnchors}}`, `{{allTasks}}`, `{{recentEvents}}` |
+
+When a template contains `{{variable}}` tags, the matching data is hydrated directly into `systemPrompt` and a short fixed string is used as `userPrompt`. When a template has no `{{}}` tags, the raw data is appended as `userPrompt` — backward compatible with plain-string overrides.
+
+### Runtime Overrides (Manual Execution)
+
+Pass `promptOverride` per-call for one-off instructions. **Runtime overrides apply only to that single call — they do not persist for future auto-runs triggered by `write()`.**
+
+```typescript
+// Override the base default AND global config for this single execution.
+// Each override must include the JSON output contract (replaces the entire default prompt).
+await wikiMemory.runLibrarian('user-123', {
+  promptOverride: `One-off extraction task:\n{{events}}\n\nReturn ONLY valid JSON: { "facts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }], "tasks": [{ "description": "string", "priority": 0 }] }. No markdown.`,
+});
+
+await wikiMemory.runHeal('user-123', {
+  promptOverride: `Domain-specific healing: {{healCandidates}}\n\nReturn ONLY valid JSON: { "downgraded": ["factId"], "deleted": ["factId"], "newFacts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }] }. No markdown.`,
+});
+
+await wikiMemory.ingestDocument('user-123', {
+  sourceRef: 'doc-1',
+  sourceHash: sha256(content),
+  documentChunk: content,
+  promptOverride: `Strict technical extraction: {{documentChunk}}\n\nReturn ONLY valid JSON: { "facts": [{ "title": "string", "body": "string", "tags": ["string"], "confidence": "certain|inferred|tentative" }] }. No markdown.`,
+});
+```
+
+> **Important:** If your app relies on `write()` auto-runs and needs custom prompts for those runs, use `config.prompts` at construction time. Runtime `promptOverride` values are never forwarded to `WriteService`-triggered internal runs.
+
 ## Retrieval Tuning
 
 Optimize `read()` performance and blend retrieval strategies: