mcp-memory-keeper 0.10.0 → 0.10.1

package/CHANGELOG.md

@@ -7,8 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.10.1] - 2025-01-11
+
 ### Fixed
 
+- **Token Limit Enforcement** - Fixed MCP protocol token limit errors
+
+  - Added automatic response truncation when approaching 25,000 token limit
+  - Implemented `calculateSafeItemCount()` helper to determine safe result size
+  - Enhanced pagination metadata with `truncated` and `truncatedCount` fields
+  - Improved warning messages with specific pagination instructions
+  - Prevents "response exceeds maximum allowed tokens" errors from MCP clients
+
 - **Pagination Defaults in context_get** - Improved consistency
   - Added proper validation of pagination parameters at handler level
   - Default limit of 100 items now properly applied when not specified

package/README.md

@@ -1,11 +1,106 @@
 # MCP Memory Keeper - Claude Code Context Management
 
+[![npm version](https://img.shields.io/npm/v/mcp-memory-keeper.svg)](https://www.npmjs.com/package/mcp-memory-keeper)
+[![npm downloads](https://img.shields.io/npm/dm/mcp-memory-keeper.svg)](https://www.npmjs.com/package/mcp-memory-keeper)
 [![CI](https://github.com/mkreyman/mcp-memory-keeper/actions/workflows/ci.yml/badge.svg)](https://github.com/mkreyman/mcp-memory-keeper/actions/workflows/ci.yml)
 [![codecov](https://codecov.io/gh/mkreyman/mcp-memory-keeper/branch/main/graph/badge.svg)](https://codecov.io/gh/mkreyman/mcp-memory-keeper)
-[![npm version](https://badge.fury.io/js/mcp-memory-keeper.svg)](https://badge.fury.io/js/mcp-memory-keeper)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 A Model Context Protocol (MCP) server that provides persistent context management for Claude AI coding assistants. Never lose context during compaction again! This MCP server helps Claude Code maintain context across sessions, preserving your work history, decisions, and progress.
 
+## 🚀 Quick Start
+
+Get started in under 30 seconds:
+
+```bash
+# Add memory-keeper to Claude
+claude mcp add memory-keeper npx mcp-memory-keeper
+
+# Start a new Claude session and use it!
+# Try: Analyze the current repo and save your analysis in memory-keeper
+```
+
+That's it! Memory Keeper is now available in all your Claude sessions. Your context is stored in `~/mcp-data/memory-keeper/` and persists across sessions.
+
+## 🚀 Practical Memory Keeper Workflow Example
+
+### **Custom Command + CLAUDE.md = Automatic Context Management**
+
+#### **CLAUDE.md** (condensed example)
+
+```markdown
+# Project Configuration
+
+## Development Rules
+
+- Always use memory-keeper to track progress
+- Save architectural decisions and test results
+- Create checkpoints before context limits
+
+## Quality Standards
+
+- All tests must pass before marking complete
+- Document actual vs claimed results
+```
+
+#### **Custom Command Example: `/my-dev-workflow`**
+
+```markdown
+# My Development Workflow
+
+When working on the provided project:
+
+- Use memory-keeper with channel: <project_name>
+- Save progress at every major milestone
+- Document all decisions with category: "decision"
+- Track implementation status with category: "progress"
+- Before claiming anything is complete, save test results
+
+## Workflow Steps
+
+1. Initialize session with project name as channel
+2. Save findings during investigation
+3. Create checkpoint before major changes
+4. Document what actually works vs what should work
+```
+
+#### **Usage Example**
+
+```
+User: /my-dev-workflow authentication-service
+
+AI: Setting up workflow for authentication-service.
+[Uses memory-keeper with channel "authentication-service"]
+
+[... AI works, automatically saving context ...]
+
+User: "Getting close to context limit. Create checkpoint and give me a key"
+
+AI: "Checkpoint created: authentication-service-checkpoint-20250126-143026"
+
+[Continue working until context reset or compact manually]
+
+User: "Restore from key: authentication-service-checkpoint-20250126-143026"
+
+AI: "Restored! Continuing OAuth implementation. We completed the token validation, working on refresh logic..."
+```
+
+**The Pattern:**
+
+1. Custom command includes instructions to use memory-keeper
+2. AI follows those instructions automatically
+3. **When you notice the conversation getting long, YOU ask Claude to save a checkpoint** (like saving your game before a boss fight!)
+4. **When Claude runs out of space and starts fresh, YOU tell it to restore using the checkpoint key**
+
+**🎯 Key Feature:** Memory Keeper is a shared board! You can:
+
+- Continue in the same session after reset
+- Start a completely new session and restore
+- Have multiple Claude sessions running in parallel, all sharing the same memory
+- One session can save context that another session retrieves
+
+This enables powerful workflows like having one Claude session doing research while another implements code, both sharing discoveries through Memory Keeper!
+
 ## Why MCP Memory Keeper?
 
 Claude Code users often face context loss when the conversation window fills up. This MCP server solves that problem by providing a persistent memory layer for Claude AI. Whether you're working on complex refactoring, multi-file changes, or long debugging sessions, Memory Keeper ensures your Claude assistant remembers important context, decisions, and progress.
@@ -39,31 +134,33 @@ Claude Code users often face context loss when the conversation window fills up.
 
 ## Installation
 
-### Method 1: NPX (Recommended)
-
-The simplest way to use memory-keeper with Claude:
+### Recommended: NPX Installation
 
 ```bash
 claude mcp add memory-keeper npx mcp-memory-keeper
 ```
 
-That's it! This will:
+This single command:
 
-- Always use the latest version
-- Handle all dependencies automatically
-- Create the data directory at `~/mcp-data/memory-keeper/`
-- Work across different platforms
+- ✅ Always uses the latest version
+- ✅ Handles all dependencies automatically
+- ✅ Works across macOS, Linux, and Windows
+- ✅ No manual building or native module issues
 
-### Method 2: Global Installation
+### Alternative Installation Methods
 
-If you prefer a global installation:
+<details>
+<summary>Global Installation</summary>
 
 ```bash
 npm install -g mcp-memory-keeper
 claude mcp add memory-keeper mcp-memory-keeper
 ```
 
-### Method 3: From Source
+</details>
+
+<details>
+<summary>From Source (for development)</summary>
 
 ```bash
 # 1. Clone the repository
@@ -80,6 +177,8 @@ npm run build
 claude mcp add memory-keeper node /absolute/path/to/mcp-memory-keeper/dist/index.js
 ```
 
+</details>
+
 ## Configuration
 
 ### Environment Variables
@@ -96,13 +195,13 @@ Choose where to save the configuration:
 
 ```bash
 # Project-specific (default) - only for you in this project
-claude mcp add memory-keeper node /path/to/mcp-memory-keeper/dist/index.js
+claude mcp add memory-keeper npx mcp-memory-keeper
 
 # Shared with team via .mcp.json
-claude mcp add --scope project memory-keeper node /path/to/mcp-memory-keeper/dist/index.js
+claude mcp add --scope project memory-keeper npx mcp-memory-keeper
 
 # Available across all your projects
-claude mcp add --scope user memory-keeper node /path/to/mcp-memory-keeper/dist/index.js
+claude mcp add --scope user memory-keeper npx mcp-memory-keeper
 ```
 
 #### Verify Configuration
@@ -126,20 +225,14 @@ claude mcp get memory-keeper
 {
   "mcpServers": {
     "memory-keeper": {
-      "command": "node",
-      "args": ["/absolute/path/to/mcp-memory-keeper/dist/index.js"]
+      "command": "npx",
+      "args": ["mcp-memory-keeper"]
     }
   }
 }
 ```
 
-**Important**: Replace `/absolute/path/to/mcp-memory-keeper` with the actual path where you cloned/installed the project.
-
-### Example paths:
-
-- macOS: `/Users/username/projects/mcp-memory-keeper/dist/index.js`
-- Windows: `C:\\Users\\username\\projects\\mcp-memory-keeper\\dist\\index.js`
-- Linux: `/home/username/projects/mcp-memory-keeper/dist/index.js`
+That's it! No paths needed - npx automatically handles everything.
 
 ### Verify Installation
 
@@ -166,7 +259,7 @@ If Memory Keeper isn't working:
 ```bash
 # Remove and re-add the server
 claude mcp remove memory-keeper
-claude mcp add memory-keeper node /absolute/path/to/mcp-memory-keeper/dist/index.js
+claude mcp add memory-keeper npx mcp-memory-keeper
 
 # Check logs for errors
 # The server output will appear in Claude Code's output panel
@@ -174,26 +267,19 @@ claude mcp add memory-keeper node /absolute/path/to/mcp-memory-keeper/dist/index
 
 ### Updating to Latest Version
 
-To get the latest features and bug fixes:
-
-```bash
-# 1. Navigate to your Memory Keeper directory
-cd /path/to/mcp-memory-keeper
+With the npx installation method, you automatically get the latest version every time! No manual updates needed.
 
-# 2. Pull the latest changes
-git pull
+If you're using the global installation method:
 
-# 3. Install any new dependencies (if package.json changed)
-npm install
-
-# 4. Rebuild the project
-npm run build
+```bash
+# Update to latest version
+npm update -g mcp-memory-keeper
 
-# 5. Start a new Claude session
+# Start a new Claude session
 # The updated features will be available immediately
 ```
 
-**Note**: You don't need to reconfigure the MCP server in Claude after updating. Just pull, build, and start a new session!
+**Note**: You don't need to reconfigure the MCP server in Claude after updating. Just start a new session!
 
 ## Usage
 

package/bin/mcp-memory-keeper

@@ -27,7 +27,9 @@ const serverPath = path.join(__dirname, '..', 'dist', 'index.js');
 // Check if the server is built
 if (!fs.existsSync(serverPath)) {
   console.error('Error: Server not built. This should not happen with the npm package.');
-  console.error('Please report this issue at: https://github.com/mkreyman/mcp-memory-keeper/issues');
+  console.error(
+    'Please report this issue at: https://github.com/mkreyman/mcp-memory-keeper/issues'
+  );
   process.exit(1);
 }
 
@@ -37,16 +39,16 @@ process.chdir(DATA_DIR);
 // Spawn the server
 const child = spawn(process.execPath, [serverPath, ...process.argv.slice(2)], {
   stdio: 'inherit',
-  env: process.env
+  env: process.env,
 });
 
 // Handle exit
-child.on('exit', (code) => {
+child.on('exit', code => {
   process.exit(code);
 });
 
 // Handle errors
-child.on('error', (err) => {
+child.on('error', err => {
   console.error('Failed to start memory-keeper server:', err);
   process.exit(1);
-});
+});

package/dist/__tests__/integration/tokenLimitEnforcement.test.js

@@ -0,0 +1,134 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+const globals_1 = require("@jest/globals");
+// Helper functions from the main index.ts file
+function estimateTokens(text) {
+    return Math.ceil(text.length / 4);
+}
+function calculateSafeItemCount(items, tokenLimit) {
+    if (items.length === 0)
+        return 0;
+    let safeCount = 0;
+    let currentTokens = 0;
+    // Include base response structure in token calculation
+    const baseResponse = {
+        items: [],
+        pagination: {
+            total: 0,
+            returned: 0,
+            offset: 0,
+            hasMore: false,
+            nextOffset: null,
+            totalCount: 0,
+            page: 1,
+            pageSize: 0,
+            totalPages: 1,
+            hasNextPage: false,
+            hasPreviousPage: false,
+            previousOffset: null,
+            totalSize: 0,
+            averageSize: 0,
+            defaultsApplied: {},
+            truncated: false,
+            truncatedCount: 0,
+        },
+    };
+    // Estimate tokens for base response structure
+    const baseTokens = estimateTokens(JSON.stringify(baseResponse, null, 2));
+    currentTokens = baseTokens;
+    // Add items one by one until we approach the token limit
+    for (let i = 0; i < items.length; i++) {
+        const itemTokens = estimateTokens(JSON.stringify(items[i], null, 2));
+        // Leave some buffer (10%) to account for formatting and additional metadata
+        if (currentTokens + itemTokens > tokenLimit * 0.9) {
+            break;
+        }
+        currentTokens += itemTokens;
+        safeCount++;
+    }
+    // Always return at least 1 item if any exist, even if it exceeds limit
+    // This prevents infinite loops and ensures progress
+    return Math.max(safeCount, items.length > 0 ? 1 : 0);
+}
+(0, globals_1.describe)('Token Limit Enforcement Unit Tests', () => {
+    (0, globals_1.describe)('calculateSafeItemCount', () => {
+        (0, globals_1.it)('should return 0 for empty items array', () => {
+            const result = calculateSafeItemCount([], 20000);
+            (0, globals_1.expect)(result).toBe(0);
+        });
+        (0, globals_1.it)('should return at least 1 item if any exist', () => {
+            const largeItem = {
+                key: 'large.item',
+                value: 'X'.repeat(100000), // Very large item
+                category: 'test',
+                priority: 'high',
+            };
+            const result = calculateSafeItemCount([largeItem], 20000);
+            (0, globals_1.expect)(result).toBe(1);
+        });
+        (0, globals_1.it)('should truncate items when approaching token limit', () => {
+            // Create multiple medium-sized items
+            const items = [];
+            for (let i = 0; i < 50; i++) {
+                items.push({
+                    key: `item.${i}`,
+                    value: 'This is a medium-sized test value that contains enough text to trigger token limit enforcement when many items are returned together. '.repeat(20),
+                    category: 'test',
+                    priority: 'high',
+                });
+            }
+            const result = calculateSafeItemCount(items, 20000);
+            (0, globals_1.expect)(result).toBeLessThan(50);
+            (0, globals_1.expect)(result).toBeGreaterThan(0);
+        });
+        (0, globals_1.it)('should handle small items that all fit within limit', () => {
+            const items = [];
+            for (let i = 0; i < 10; i++) {
+                items.push({
+                    key: `small.item.${i}`,
+                    value: 'Small value',
+                    category: 'test',
+                    priority: 'high',
+                });
+            }
+            const result = calculateSafeItemCount(items, 20000);
+            (0, globals_1.expect)(result).toBe(10);
+        });
+        (0, globals_1.it)('should respect token limit with buffer', () => {
+            // Create items that would exceed token limit
+            const items = [];
+            const itemValue = 'X'.repeat(2000); // 2KB item that will definitely cause truncation
+            for (let i = 0; i < 100; i++) {
+                items.push({
+                    key: `large.buffer.item.${i}`,
+                    value: itemValue,
+                    category: 'test',
+                    priority: 'high',
+                });
+            }
+            const result = calculateSafeItemCount(items, 20000);
+            // Should be significantly less than all items due to token limits
+            (0, globals_1.expect)(result).toBeLessThan(100);
+            (0, globals_1.expect)(result).toBeGreaterThan(0);
+            // Verify that the result respects the buffer by checking actual tokens
+            const actualTokens = result * estimateTokens(JSON.stringify(items[0], null, 2));
+            (0, globals_1.expect)(actualTokens).toBeLessThan(20000 * 0.9); // Should be under 90% of limit
+        });
+    });
+    (0, globals_1.describe)('estimateTokens', () => {
+        (0, globals_1.it)('should estimate tokens correctly', () => {
+            const text = 'This is a test string';
+            const tokens = estimateTokens(text);
+            (0, globals_1.expect)(tokens).toBe(Math.ceil(text.length / 4));
+        });
+        (0, globals_1.it)('should handle empty strings', () => {
+            const tokens = estimateTokens('');
+            (0, globals_1.expect)(tokens).toBe(0);
+        });
+        (0, globals_1.it)('should handle large strings', () => {
+            const largeText = 'X'.repeat(10000);
+            const tokens = estimateTokens(largeText);
+            (0, globals_1.expect)(tokens).toBe(2500); // 10000 / 4
+        });
+    });
+});

package/dist/index.js

@@ -185,6 +185,52 @@ function calculateResponseMetrics(items) {
     const averageSize = items.length > 0 ? Math.round(totalSize / items.length) : 0;
     return { totalSize, estimatedTokens, averageSize };
 }
+// Helper to calculate how many items can fit within token limit
+function calculateSafeItemCount(items, tokenLimit) {
+    if (items.length === 0)
+        return 0;
+    let safeCount = 0;
+    let currentTokens = 0;
+    // Include base response structure in token calculation
+    const baseResponse = {
+        items: [],
+        pagination: {
+            total: 0,
+            returned: 0,
+            offset: 0,
+            hasMore: false,
+            nextOffset: null,
+            totalCount: 0,
+            page: 1,
+            pageSize: 0,
+            totalPages: 1,
+            hasNextPage: false,
+            hasPreviousPage: false,
+            previousOffset: null,
+            totalSize: 0,
+            averageSize: 0,
+            defaultsApplied: {},
+            truncated: false,
+            truncatedCount: 0,
+        },
+    };
+    // Estimate tokens for base response structure
+    const baseTokens = estimateTokens(JSON.stringify(baseResponse, null, 2));
+    currentTokens = baseTokens;
+    // Add items one by one until we approach the token limit
+    for (let i = 0; i < items.length; i++) {
+        const itemTokens = estimateTokens(JSON.stringify(items[i], null, 2));
+        // Leave some buffer (10%) to account for formatting and additional metadata
+        if (currentTokens + itemTokens > tokenLimit * 0.9) {
+            break;
+        }
+        currentTokens += itemTokens;
+        safeCount++;
+    }
+    // Always return at least 1 item if any exist, even if it exceeds limit
+    // This prevents infinite loops and ensures progress
+    return Math.max(safeCount, items.length > 0 ? 1 : 0);
+}
 // Helper to parse relative time strings
 function parseRelativeTime(relativeTime) {
     const now = new Date();
@@ -625,16 +671,35 @@ server.setRequestHandler(types_js_1.CallToolRequestSchema, async (request) => {
                 // Calculate response metrics
                 const metrics = calculateResponseMetrics(result.items);
                 const TOKEN_LIMIT = 20000; // Conservative limit to stay well under MCP's 25k limit
-                // Check if we're approaching token limits
+                // Check if we're approaching token limits and enforce truncation
                 const isApproachingLimit = metrics.estimatedTokens > TOKEN_LIMIT;
+                let actualItems = result.items;
+                let wasTruncated = false;
+                let truncatedCount = 0;
+                if (isApproachingLimit) {
+                    // Calculate how many items we can safely return
+                    const safeItemCount = calculateSafeItemCount(result.items, TOKEN_LIMIT);
+                    if (safeItemCount < result.items.length) {
+                        actualItems = result.items.slice(0, safeItemCount);
+                        wasTruncated = true;
+                        truncatedCount = result.items.length - safeItemCount;
+                    }
+                }
                 // Calculate pagination metadata
                 // Use the validated limit and offset from paginationValidation
                 const effectiveLimit = limit; // Already validated and defaulted
                 const effectiveOffset = offset; // Already validated and defaulted
                 const currentPage = effectiveLimit > 0 ? Math.floor(effectiveOffset / effectiveLimit) + 1 : 1;
                 const totalPages = effectiveLimit > 0 ? Math.ceil(result.totalCount / effectiveLimit) : 1;
-                const hasNextPage = currentPage < totalPages;
+                // Update pagination to account for truncation
+                const hasNextPage = wasTruncated || currentPage < totalPages;
                 const hasPreviousPage = currentPage > 1;
+                // Calculate next offset accounting for truncation
+                const nextOffset = hasNextPage
+                    ? wasTruncated
+                        ? effectiveOffset + actualItems.length
+                        : effectiveOffset + effectiveLimit
+                    : null;
                 // Track whether defaults were applied
                 const defaultsApplied = {
                     limit: rawLimit === undefined,
@@ -642,7 +707,7 @@ server.setRequestHandler(types_js_1.CallToolRequestSchema, async (request) => {
                 };
                 // Enhanced response format
                 if (includeMetadata) {
-                    const itemsWithMetadata = result.items.map(item => ({
+                    const itemsWithMetadata = actualItems.map(item => ({
                         key: item.key,
                         value: item.value,
                         category: item.category,
@@ -657,10 +722,10 @@ server.setRequestHandler(types_js_1.CallToolRequestSchema, async (request) => {
                         items: itemsWithMetadata,
                         pagination: {
                             total: result.totalCount,
-                            returned: result.items.length,
+                            returned: actualItems.length,
                             offset: effectiveOffset,
                             hasMore: hasNextPage,
-                            nextOffset: hasNextPage ? effectiveOffset + effectiveLimit : null,
+                            nextOffset: nextOffset,
                             // Extended pagination metadata
                             totalCount: result.totalCount,
                             page: currentPage,
@@ -676,12 +741,20 @@ server.setRequestHandler(types_js_1.CallToolRequestSchema, async (request) => {
                             averageSize: metrics.averageSize,
                             // Defaults applied
                             defaultsApplied: defaultsApplied,
+                            // Truncation information
+                            truncated: wasTruncated,
+                            truncatedCount: truncatedCount,
                         },
                     };
                     // Add warning if approaching token limits
                     if (isApproachingLimit) {
-                        response.pagination.warning =
-                            'Large result set. Consider using smaller limit or more specific filters.';
+                        if (wasTruncated) {
+                            response.pagination.warning = `Response truncated due to token limits. ${truncatedCount} items omitted. Use pagination with offset=${nextOffset} to retrieve remaining items.`;
+                        }
+                        else {
+                            response.pagination.warning =
+                                'Large result set. Consider using smaller limit or more specific filters.';
+                        }
                     }
                     return {
                         content: [
@@ -694,19 +767,27 @@ server.setRequestHandler(types_js_1.CallToolRequestSchema, async (request) => {
                 }
                 // Return enhanced format for all queries to support pagination
                 const response = {
-                    items: result.items,
+                    items: actualItems,
                     pagination: {
                         total: result.totalCount,
-                        returned: result.items.length,
+                        returned: actualItems.length,
                         offset: effectiveOffset,
                         hasMore: hasNextPage,
-                        nextOffset: hasNextPage ? effectiveOffset + effectiveLimit : null,
+                        nextOffset: nextOffset,
+                        // Truncation information
+                        truncated: wasTruncated,
+                        truncatedCount: truncatedCount,
                     },
                 };
                 // Add warning if approaching token limits
                 if (isApproachingLimit) {
-                    response.pagination.warning =
-                        'Large result set. Consider using smaller limit or more specific filters.';
+                    if (wasTruncated) {
+                        response.pagination.warning = `Response truncated due to token limits. ${truncatedCount} items omitted. Use pagination with offset=${nextOffset} to retrieve remaining items.`;
+                    }
+                    else {
+                        response.pagination.warning =
+                            'Large result set. Consider using smaller limit or more specific filters.';
+                    }
                 }
                 return {
                     content: [

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcp-memory-keeper",
-  "version": "0.10.0",
+  "version": "0.10.1",
   "description": "MCP server for persistent context management in AI coding assistants",
   "main": "dist/index.js",
   "bin": {