persyst-mcp 2.2.5 → 2.2.7

package/README.md

@@ -1,203 +1,192 @@
 # Persyst
 
-**Local-first MCP memory server for coding agents.**
+**Local-first, compliance-grade MCP memory layer for regulated enterprise coding teams using AI assistants.**
 
-Persyst gives AI coding agents (Claude Code, Cursor, VS Code, Aider, Windsurf, Antigravity) persistent memory across sessions. It stores memories in a local SQLite database with hybrid keyword + semantic search — no cloud, no API keys, works offline.
+Persyst gives AI coding agents (Claude Code, Cursor, VS Code, Aider, Windsurf, Antigravity) persistent memory across sessions. It stores memories in a local SQLite database with hybrid keyword + semantic search — operating 100% offline with zero cloud egress.
 
-## How It Works
+---
 
-```
-Your AI Agent ←→ MCP (stdio) ←→ Persyst ←→ SQLite (local)
-```
+## Compliance-Grade Security Features
+
+Persyst is built from the ground up for highly regulated enterprise environments (finance, healthcare, defense) subject to **SOC 2**, **HIPAA**, and the **EU AI Act**:
 
-1. **Agent stores a memory** → Persyst saves it + generates a search embedding
-2. **Agent searches memories** → Persyst finds matches by both keywords AND meaning
-3. **"dark mode" ↔ "night theme"** → Semantic search understands synonyms
+* **100% Data Residency (Zero-Egress)**: All vector calculations, full-text searches, and model inferences run locally on the developer's workstation. No database records or context data ever leave the local machine. Bypasses Business Associate Agreement (BAA) complexity for HIPAA.
+* **Cryptographic Chain of Custody**: Every context retrieval generates an Ed25519 cryptographic signature sealing the query and retrieved memory hashes. Each attestation is chained to the previous one via SHA-256 hash chains, creating a tamper-evident audit ledger verifiable by security teams.
+* **Automatic Secret Redaction**: Scans incoming log files and text writes to redact high-entropy secrets (API keys, JWTs, database strings, private keys) before they reach the persistent database.
+* **Event-Driven File Watching**: Integrates `chokidar` for instant scanning of agent transcript folders, guaranteeing that your memories are synchronized immediately after each agent interaction.
+* **Workspace Project Isolation**: Supports `PERSYST_PROJECT` environment partitioning, preventing cross-project context leaks while allowing shared enterprise compliance rules.
 
-> 🚨 **First-Run Note**: On the first start, Persyst will automatically download the local embedding model (`all-MiniLM-L6-v2` ~50MB). This can take 30-60 seconds depending on your connection. The server will log `Loading embedding model...` and then proceed normally.
+*Read more in our compliance mapping guides:*
+- [SOC 2 Type II Controls](compliance/SOC2-controls.md)
+- [HIPAA Mapping & PHI Boundaries](compliance/HIPAA-mapping.md)
+- [EU AI Act Article 13 Transparency](compliance/EU-AI-Act-Article13.md)
+- [Compliance Audit Trail Sample](compliance/audit-trail-sample.md)
 
 ---
 
-## Quick Start
+## Quick Start & Automatic IDE Setup
 
-You don't need to install anything globally. You can run it instantly using `npx`:
+You don't need to configure MCP files manually. Persyst includes an automated setup CLI that detects installed editors and configures rule wrappers and global settings in seconds.
 
-### 1. Add to Claude Code or Claude Desktop
+### Automatic One-Command Setup
 
-#### Claude Code (CLI)
-Add this to your global configuration file located at `~/.claude.json`:
-```json
-{
-  "mcpServers": {
-    "persyst": {
-      "command": "npx",
-      "args": ["-y", "persyst-mcp"]
-    }
-  }
-}
+Run the setup wizard in your target project directory:
+
+```bash
+npx persyst-mcp init
 ```
 
-#### Claude Desktop
-Add this to your Claude Desktop configuration file:
-* **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
-* **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
-* **Linux**: `~/.config/Claude/claude_desktop_config.json`
+This command automatically:
+1. Generates local cryptographic Ed25519 keypairs in `~/.persyst`.
+2. Creates workspace rule files (`.cursorrules`, `.windsurfrules`, `.clinerules`, `.persystrules.md`) to instruct agents on memory retrieval.
+3. Automatically writes global MCP server configurations for **Cursor**, **Claude Code**, **Aider**, and **Continue.dev** with project-scoped environment parameters (`PERSYST_PROJECT`).
+
+---
+
+## Manual MCP Configuration
 
+If you prefer to configure your agent manually, add the MCP server definition to your editor:
+
+### Claude Code (`~/.claude.json`) & Claude Desktop
 ```json
 {
   "mcpServers": {
     "persyst": {
       "command": "npx",
-      "args": ["-y", "persyst-mcp"]
+      "args": ["-y", "persyst-mcp"],
+      "env": {
+        "PERSYST_PROJECT": "my-project"
+      }
     }
   }
 }
 ```
 
----
-
-## Setup for Other Agents
-
 ### VS Code (Cline / Roo Code)
-Add this configuration to your user settings under the MCP settings file (`cline_mcp_settings.json`):
+Add to your user settings under `cline_mcp_settings.json`:
 ```json
 {
   "mcpServers": {
     "persyst": {
       "command": "npx",
-      "args": ["-y", "persyst-mcp"]
+      "args": ["-y", "persyst-mcp"],
+      "env": {
+        "PERSYST_PROJECT": "my-project"
+      }
     }
   }
 }
 ```
 
 ### Cursor
-Add Persyst in Cursor under **Settings → Features → MCP**:
+Under **Settings → Features → MCP**:
 1. Click **+ Add New MCP Server**
 2. Name: `persyst`
 3. Type: `stdio`
 4. Command: `npx -y persyst-mcp`
 
 ### Aider
-Start Aider from the command line passing the server command:
-```bash
-aider --mcp-server persyst:npx -y persyst-mcp
-```
-Or append this to your `.aider.conf.yml` project file:
+Append to your `.aider.conf.yml` project file:
 ```yaml
 mcp-server:
   - name: persyst
     command: npx -y persyst-mcp
-```
-
-### Antigravity
-Add Persyst to your Antigravity agent configuration file at `~/.gemini/antigravity/mcp_config.json`:
-```json
-{
-  "mcpServers": {
-    "persyst": {
-      "command": "npx",
-      "args": ["-y", "persyst-mcp"]
-    }
-  }
-}
+    env:
+      PERSYST_PROJECT: my-project
 ```
 
 ---
 
-## Available Tools
-
-| Tool | Description | Parameters |
-|------|-------------|------------|
-| `add_memory` | Store a new memory | `content` (string), `importance` (0-1, optional) |
-| `search_memories` | Hybrid keyword + semantic search | `query` (string), `limit` (number) |
-| `get_memory` | Get memory by ID | `id` (number) |
-| `update_memory` | Update memory content | `id` (number), `content` (string) |
-| `delete_memory` | Delete a memory and clean up edges | `id` (number) |
-| `get_recent_memories` | Get latest memories | `limit` (number) |
-| `get_important_memories` | Get by importance score | `limit` (number) |
-| `get_optimized_context` | Get compressed, ranked context block | `query` (string), `max_tokens` (number) |
-| `ingest_git_commits` | Import recent git commits as memories | `repo_path` (string), `count` (number) |
-| `consolidate_memories` | Merge highly similar duplicate memories | — |
-| `get_memory_history` | Retrieve all versions of a memory | `query` (string) |
-| `get_agent_stats` | Agent reputation stats | — |
-| `export_audit_log` | Export attestation audit log | `start_date`, `end_date` (ISO8601) |
-| `verify_attestation` | Verify Ed25519 signature chain | `attestation_id` (string) |
+## Passive Recording vs. Active Retrieval
+
+> **Note on Agent Integration**: Persyst operates in two complementary modes:
+> 1. **Passive Recording**: The file watcher automatically extracts and saves memories from your agent conversation transcripts in the background.
+> 2. **Active Retrieval**: The AI agent calls `search_memories` or `get_optimized_context` to fetch relevant context.
+>
+> The IDE itself does not automatically inject retrieved memories into prompt inputs unless configured to do so via workspace rules (e.g. `.cursorrules`, `.windsurfrules`, `.clinerules`) or custom system prompt builders.
 
 ---
 
-## How Search Works
+## Available Tools (19 MCP Endpoints)
+
+| Tool | Description | Key Parameters |
+|------|-------------|----------------|
+| `add_memory` | Store a new memory with secret redaction & contradiction check | `content`, `importance` (0-1), `agent_id`, `shared` |
+| `search_memories` | Hybrid keyword + semantic search with attestation | `query`, `limit`, `agent_id` |
+| `get_memory` | Retrieve a specific memory by ID (boosts importance) | `id`, `agent_id` |
+| `update_memory` | Update content & archive previous version | `id`, `content`, `agent_id` |
+| `delete_memory` | Permanently delete a memory & clean knowledge graph edges | `id` |
+| `get_recent_memories` | Fetch latest memories ordered by creation date | `limit`, `agent_id` |
+| `get_important_memories` | Fetch memories ranked by importance score | `limit`, `agent_id` |
+| `get_optimized_context` | Graph-hopped context prompt compiled within token budget | `query`, `max_tokens`, `agent_id`, `intent` |
+| `ingest_git_commits` | Parse & import recent git commits as structured memories | `repo_path`, `count` |
+| `watch_git_repo` | Poll repository for changes and auto-ingest new commits | `repo_path` |
+| `consolidate_memories` | Semantic deduplication sweep merging similar memories | — |
+| `get_memory_history` | Retrieve complete version history and semantic diffs | `query` |
+| `get_agent_stats` | View agent reputation scores & contradiction metrics | — |
+| `export_audit_log` | Export cryptographic attestation audit log (JSON/Markdown) | `start_date`, `end_date` |
+| `verify_attestation` | Verify Ed25519 signature & SHA-256 chain integrity | `attestation_id` |
+| `add_entity` | Add named entity to knowledge graph | `name`, `type` |
+| `link_entity_memory` | Create edge between knowledge graph entity and memory | `entity_id`, `memory_id`, `relation` |
+| `search_by_entity` | Query linked memories via knowledge graph traversal | `entity_name` |
+
+---
 
-Persyst uses **hybrid search** — combining two strategies:
+## Local HTTP Gateway & Swarm Integration
 
-1. **Keyword Search (FTS5)** — Exact word matches using BM25 ranking
-2. **Semantic Search (sqlite-vec)** — Meaning-based using local embeddings
+In addition to STDIO transport, Persyst automatically launches a high-throughput local HTTP Gateway on port `4321` (`http://127.0.0.1:4321`).
 
-Results from both are merged. Keyword matches get a score boost so exact matches rank higher, but semantic matches still surface related memories.
+- **`/health`**: Health check and database status
+- **`/stats`**: Global memory & agent reputation statistics
+- **`/system-prompt`**: Formatted prompt context injection
+- **`/compliance/export`**: Cryptographic compliance audit report export (supports `format=markdown`)
+- **`/events`**: Real-time Server-Sent Events (SSE) stream for agent swarms
 
 ---
 
-## Tech Stack
+## How Hybrid Search Works
 
-- **Runtime:** Node.js 18+
-- **Database:** SQLite via better-sqlite3
-- **Vector Search:** sqlite-vec (local, no cloud)
-- **Full-Text Search:** SQLite FTS5
-- **Embeddings:** @huggingface/transformers + all-MiniLM-L6-v2 (384-dim, ~50MB)
-- **Protocol:** MCP over stdio
+Persyst combines two complementary search strategies:
 
----
-
-## Troubleshooting
+1. **Keyword Search (SQLite FTS5)** — Fast, exact string matching using BM25 ranking.
+2. **Semantic Search (sqlite-vec)** — Deep meaning-based matching using local `all-MiniLM-L6-v2` embeddings.
 
-#### `better-sqlite3` installation fails
-`better-sqlite3` compiles native C++ code on installation. Make sure you have python and C++ build tools installed on your system:
-* **Windows:** Run `npm install --global windows-build-tools` or install Visual Studio Build Tools.
-* **macOS/Linux:** Run `xcode-select --install` or install `build-essential`.
+Results are merged dynamically. Keyword matches receive a score boost so exact matches rank at the top, while semantic similarity surfaces conceptually relevant memories even when different phrasing is used.
 
-#### The agent is stuck or loading forever on startup
-This is normal on the **very first run** because Persyst is downloading the ~50MB embedding model. Wait 30-60 seconds for it to complete. The next runs will be instant.
+---
 
-#### Command not found: `persyst-mcp`
-Instead of running it globally, prefer using the `npx -y persyst-mcp` command in your agent configurations. It automatically installs and updates the server non-interactively.
+## Tech Stack
 
-#### Permission Denied
-Do not run `npx` with `sudo`. If you run into permission issues, ensure your npm global prefix is owned by your user account.
+- **Runtime:** Node.js 18+
+- **Database:** SQLite via `better-sqlite3` (synchronous, WAL mode)
+- **Vector Search:** `sqlite-vec` (in-process, zero cloud egress)
+- **Full-Text Search:** SQLite FTS5
+- **Embeddings:** `@huggingface/transformers` + `all-MiniLM-L6-v2` (384-dim, local ONNX)
+- **Watcher:** `chokidar` event-driven file monitoring
+- **Protocol:** MCP over stdio + HTTP Gateway
 
 ---
 
 ## Backup & Migration
 
-Persyst includes built-in JSONL export/import commands for portable memory backup and cross-machine migration.
+Persyst includes built-in JSONL export/import commands for portable memory backup and cross-machine migration:
 
 ```bash
-# Export all memories to a file
+# Export all memories to a JSONL file
 npx persyst-mcp export
-# → persyst-export-<timestamp>.jsonl
 
 # Export to a specific file
 npx persyst-mcp export my-backup.jsonl
 
-# Preview what would be imported (dry run)
+# Preview import (dry run)
 npx persyst-mcp import my-backup.jsonl --dry-run
 
-# Import memories (skips exact & semantic duplicates automatically)
+# Import memories (deduplicates automatically)
 npx persyst-mcp import my-backup.jsonl
 ```
 
 ---
 
-## Roadmap & Future Directions
-
-Persyst is built for the privacy-focused solo developer. We are actively hardening the local-first experience before introducing network dependencies.
-
-* **File-Based Sync** ✅ **Done**: `persyst-export` / `persyst-import` JSONL commands for backup and migration.
-* **IDE Integrations**: First-class extensions for Cursor, VS Code, and Aider configuration helper commands.
-* **True P2P Sync (Roadmap)**: Peer-to-peer secure sync between developer devices without relying on central cloud servers.
-
----
-
 ## License
 
 MIT License. See [LICENSE](LICENSE) for details.
-

package/bin/export.js

@@ -100,16 +100,16 @@ try {
     });
   });
 
-  console.log(`✅ Exported ${count} memories to: ${outputFile}`);
+  console.log(`[OK] Exported ${count} memories to: ${outputFile}`);
   if (namespace) {
-    console.log(`   Namespace filter: "${namespace}" + shared`);
+    console.log(`     Namespace filter: "${namespace}" + shared`);
   }
   if (includeArchived) {
-    console.log('   Includes archived (superseded) memories.');
+    console.log('     Includes archived (superseded) memories.');
   }
 
 } catch (err) {
-  console.error(`❌ Export failed: ${err.message}`);
+  console.error(`[ERROR] Export failed: ${err.message}`);
   process.exit(1);
 } finally {
   closeDatabase();

package/bin/extract.js

@@ -114,9 +114,9 @@ async function run() {
   }
 
   if (!jsonOutput) {
-    console.log(`\n📋 Heuristic fact(s) extracted: ${heuristicFacts.length}`);
+    console.log(`\n[INFO] Heuristic fact(s) extracted: ${heuristicFacts.length}`);
     for (const f of heuristicFacts) {
-      console.log(`  ✓ [${f.category}] (conf: ${f.confidence}) ${f.content}`);
+      console.log(`  [OK] [${f.category}] (conf: ${f.confidence}) ${f.content}`);
     }
   }
 
@@ -128,7 +128,7 @@ async function run() {
   // --- Store to database (unless dry-run) ---
   if (!dryRun && allFacts.length > 0) {
     if (!jsonOutput) {
-      console.log(`\n💾 Storing to database...`);
+      console.log(`\n[INFO] Storing to database...`);
     }
 
     const { insertMemory, insertVector, memoryExists } = await import('../src/database.js');
@@ -142,7 +142,7 @@ async function run() {
       if (memoryExists(fact.content)) {
         dupes++;
         if (!jsonOutput) {
-          console.log(`  ⏭ Duplicate: "${fact.content.slice(0, 50)}..."`);
+          console.log(`  [SKIP] Duplicate: "${fact.content.slice(0, 50)}..."`);
         }
         continue;
       }
@@ -158,15 +158,15 @@ async function run() {
 
       stored++;
       if (!jsonOutput) {
-        console.log(`  ✅ Stored memory #${id}: "${fact.content.slice(0, 60)}..."`);
+        console.log(`  [OK] Stored memory #${id}: "${fact.content.slice(0, 60)}..."`);
       }
     }
 
     if (!jsonOutput) {
-      console.log(`\n📊 Result: ${stored} stored, ${dupes} duplicates skipped`);
+      console.log(`\n[INFO] Result: ${stored} stored, ${dupes} duplicates skipped`);
     }
   } else if (dryRun && !jsonOutput) {
-    console.log(`\n🔍 Dry run — no facts stored.`);
+    console.log(`\n[INFO] Dry run — no facts stored.`);
   }
 
   // --- JSON output ---
@@ -180,6 +180,6 @@ async function run() {
 }
 
 run().catch(err => {
-  console.error(`\n❌ Extraction failed: ${err.message}`);
+  console.error(`\n[ERROR] Extraction failed: ${err.message}`);
   process.exit(1);
 });

package/bin/import.js

@@ -40,7 +40,7 @@ const skipEmbeddings = args.includes('--skip-embeddings');
 const DEDUP_THRESHOLD = 0.85;
 
 if (!inputFile) {
-  console.error('❌ Usage: persyst-import <file.jsonl> [--dry-run] [--namespace=<ns>] [--skip-embeddings]');
+  console.error('[ERROR] Usage: persyst-import <file.jsonl> [--dry-run] [--namespace=<ns>] [--skip-embeddings]');
   process.exit(1);
 }
 
@@ -49,10 +49,10 @@ if (!inputFile) {
 // ============================================================
 
 async function main() {
-  console.log(`📥 Persyst Import${isDryRun ? ' (DRY RUN — nothing will be written)' : ''}`);
-  console.log(`   Source: ${inputFile}`);
-  if (forceNamespace) console.log(`   Forcing namespace: "${forceNamespace}"`);
-  if (skipEmbeddings) console.log('   Skipping embedding regeneration.');
+  console.log(`[IMPORT] Persyst Import${isDryRun ? ' (DRY RUN — nothing will be written)' : ''}`);
+  console.log(`         Source: ${inputFile}`);
+  if (forceNamespace) console.log(`         Forcing namespace: "${forceNamespace}"`);
+  if (skipEmbeddings) console.log('         Skipping embedding regeneration.');
   console.log('');
 
   const rl = createInterface({
@@ -74,7 +74,7 @@ async function main() {
     try {
       record = JSON.parse(trimmed);
     } catch (err) {
-      console.error(`  ⚠️  Line ${lineNum}: Invalid JSON — skipping`);
+      console.error(`  [WARN] Line ${lineNum}: Invalid JSON — skipping`);
       errors++;
       continue;
     }
@@ -82,7 +82,7 @@ async function main() {
     const { content, importance_score = 1.0, namespace, provenance, valid_until } = record;
 
     if (!content || typeof content !== 'string' || content.trim().length === 0) {
-      console.error(`  ⚠️  Line ${lineNum}: Empty content — skipping`);
+      console.error(`  [WARN] Line ${lineNum}: Empty content — skipping`);
       errors++;
       continue;
     }
@@ -97,7 +97,7 @@ async function main() {
 
     // --- Dedup: exact content match ---
     if (memoryExists(content, targetNamespace)) {
-      console.log(`  ⏭️  Line ${lineNum}: Already exists — skipping "${content.slice(0, 60)}..."`);
+      console.log(`  [SKIP] Line ${lineNum}: Already exists — skipping "${content.slice(0, 60)}..."`);
       skipped++;
       continue;
     }
@@ -107,7 +107,7 @@ async function main() {
       try {
         const similar = await searchHybrid(content, 1, null, null, targetNamespace);
         if (similar.length > 0 && parseFloat(similar[0].similarity) >= DEDUP_THRESHOLD) {
-          console.log(`  ⏭️  Line ${lineNum}: Semantically similar to #${similar[0].id} (sim=${similar[0].similarity}) — skipping`);
+          console.log(`  [SKIP] Line ${lineNum}: Semantically similar to #${similar[0].id} (sim=${similar[0].similarity}) — skipping`);
           skipped++;
           continue;
         }
@@ -117,7 +117,7 @@ async function main() {
     }
 
     if (isDryRun) {
-      console.log(`  ✅ Would import: "${content.slice(0, 80)}${content.length > 80 ? '...' : ''}" → ns="${targetNamespace}"`);
+      console.log(`  [OK] Would import: "${content.slice(0, 80)}${content.length > 80 ? '...' : ''}" → ns="${targetNamespace}"`);
       imported++;
       continue;
     }
@@ -132,10 +132,10 @@ async function main() {
         insertVector(id, embedding);
       }
 
-      console.log(`  ✅ Imported #${id}: "${content.slice(0, 70)}${content.length > 70 ? '...' : ''}"`);
+      console.log(`  [OK] Imported #${id}: "${content.slice(0, 70)}${content.length > 70 ? '...' : ''}"`);
       imported++;
     } catch (err) {
-      console.error(`  ❌ Line ${lineNum}: Failed to insert — ${err.message}`);
+      console.error(`  [ERROR] Line ${lineNum}: Failed to insert — ${err.message}`);
       errors++;
     }
   }
@@ -143,16 +143,16 @@ async function main() {
   console.log('');
   console.log('═'.repeat(50));
   if (isDryRun) {
-    console.log(`📊 Dry run complete: ${imported} would import, ${skipped} skipped, ${errors} errors`);
+    console.log(`[INFO] Dry run complete: ${imported} would import, ${skipped} skipped, ${errors} errors`);
   } else {
-    console.log(`📊 Import complete: ${imported} imported, ${skipped} skipped, ${errors} errors`);
+    console.log(`[INFO] Import complete: ${imported} imported, ${skipped} skipped, ${errors} errors`);
   }
   console.log('═'.repeat(50));
 }
 
 main()
   .catch(err => {
-    console.error(`❌ Import crashed: ${err.message}`);
+    console.error(`[ERROR] Import crashed: ${err.message}`);
     process.exit(1);
   })
   .finally(() => {