@softerist/heuristic-mcp 2.1.46 → 3.0.0

package/.agent/workflows/code-review.md

@@ -0,0 +1,60 @@
+---
+description: Perform a comprehensive, senior-level code review following strict quality, security, and architectural guidelines.
+version: 2.1
+last_updated: 2026-01-29
+---
+
+You are a senior software engineer and code reviewer. Review the code provided according to the following strict guidelines.
+
+## 1. Review Scope & Context
+- **For code <100 lines**: Quick review focusing on correctness and obvious bugs.
+- **For code 100-500 lines**: Standard review following all sections below.
+- **For code >500 lines or critical systems**: Deep review with extra emphasis on architecture, security, and edge cases.
+- If reviewing multiple files, prioritize by risk (security > data handling > business logic > UI).
+- **Domain-Specific Checks**: If the code domain is identifiable (e.g., web, mobile, embedded, ML, healthcare), apply relevant domain-specific standards (e.g., WCAG for web, PII handling for healthcare, real-time constraints for embedded).
+
+## 2. Line-by-Line Review
+- Review every function and critical code path. For files >200 lines, focus on functions with issues rather than every line.
+- Group line-by-line findings by severity (**Critical** first), then by file/function within each severity tier.
+- For each issue, quote the relevant line(s).
+- **Confidence Ratings**: For ambiguous issues where context is missing, explicitly note `[Confidence: Low/Medium/High]` and explain what additional information would clarify the finding.
+
+## 3. High-Level Review
+- Focus on system-level patterns not visible at function scope: module organization, layer violations, missing abstraction layers, deployment concerns.
+- Do not check for issues already covered in line-by-line (avoid redundancy).
+
+## 4. Follow-up Reviews
+- For follow-up reviews, focus on: regression verification, new code introduced by fixes, and cross-cutting impact of changes.
+
+## 5. Stalled/Incomplete Logic
+- Check for "forgotten error paths": places where exceptions/errors could occur but aren't caught, or catch blocks that are empty/only log without recovery strategy.
+- Verify TODOs or placeholders.
+
+## 6. Consistency
+- Identify the dominant pattern in the codebase and flag deviations. If no clear majority, note the inconsistency itself as an issue.
+- Check naming conventions, coding style, and file structure.
+
+## 7. Fix Plan
+Provide a prioritized checklist (most critical first):
+- **Estimated effort per item** using this scale:
+  * **S (Small)**: <2 hours, isolated change, no breaking changes.
+  * **M (Medium)**: 2-8 hours, may affect multiple files, local refactoring.
+  * **L (Large)**: >8 hours, architectural changes, breaking changes, or requires team coordination.
+- For each item, note any dependencies: "Requires #N" if another fix must be completed first.
+- Group independent fixes together to enable parallel work.
+
+## 8. Patch
+- Propose concrete code edits in **unified diff format** for **ALL critical severity issues**.
+- For **high severity** issues, provide patches for the top 5 most impactful.
+- For new files or large refactors (>50 lines changed), provide complete code blocks with clear before/after markers.
+
+## 9. Tests
+- Separate **unit tests** (isolated, mocked dependencies) from **integration tests** (real dependencies). Specify which category each test belongs to.
+- List the exact tests you'd add (test names + what each asserts).
+- For performance issues, include benchmark or load tests with acceptable thresholds.
+
+## Output Format Constraints
+- **Do not skip sections.** If a section has no findings, state "No issues found" and briefly explain why (e.g., "No concurrency issues: code is single-threaded").
+- **Assume production context**: Treat as a professional audit. If context is unclear, state assumptions (e.g., "assuming user-facing web application with moderate traffic").
+- **Be exhaustive and specific.** Do not be polite; be direct.
+- **Length Control**: If findings are extensive (would exceed ~5000 words), provide an **Executive Summary** of critical issues first, then offer to continue with medium/low priority items in a follow-up response.

package/.prettierrc

@@ -0,0 +1,7 @@
+{
+  "semi": true,
+  "singleQuote": true,
+  "tabWidth": 2,
+  "trailingComma": "es5",
+  "printWidth": 100
+}

package/ARCHITECTURE.md

@@ -8,24 +8,48 @@ This document outlines the modular architecture of Heuristic MCP.
 heuristic-mcp/
 ├── index.js                    # Main entry point, MCP server setup
 ├── package.json                # Package configuration
-├── config.json                 # User configuration
+├── config.json                 # Sample configuration
 ├── LICENSE                     # MIT License
 ├── README.md                   # Project documentation
-├── EXAMPLES.md                 # Usage examples
+├── ARCHITECTURE.md             # Architecture notes
+├── CONTRIBUTING.md             # Contribution guide
 ├── .gitignore                  # Git ignore rules
 │
 ├── lib/                        # Core libraries
-│   ├── config.js              # Configuration loader
-│   ├── cache.js               # Embeddings cache management
-│   └── utils.js               # Shared utilities (chunking, similarity)
+│   ├── cache-utils.js         # Stale cache detection/cleanup
+│   ├── cache.js               # Embeddings cache management + ANN index
+│   ├── call-graph.js          # Symbol extraction and call graph helpers
+│   ├── cli.js                 # CLI argument parsing helpers
+│   ├── config.js              # Configuration loader and env overrides
+│   ├── embedding-process.js   # Child-process embedder runner (isolation)
+│   ├── embedding-worker.js    # Worker-thread embedder runner
+│   ├── ignore-patterns.js     # Smart ignore patterns by project type
+│   ├── json-worker.js         # Off-thread JSON parsing
+│   ├── json-writer.js         # Streaming JSON writer
+│   ├── logging.js             # Log file + stderr redirection helpers
+│   ├── project-detector.js    # Language/project detection
+│   ├── tokenizer.js           # Token estimation and limits
+│   ├── utils.js               # Shared utilities (chunking, similarity)
+│   └── vector-store-binary.js # Binary on-disk vector store (mmap-friendly)
 │
 ├── features/                   # Pluggable features
-│   ├── hybrid-search.js       # Semantic search feature
+│   ├── hybrid-search.js       # Semantic + exact match search
 │   ├── index-codebase.js      # Code indexing feature
-│   └── clear-cache.js         # Cache management feature
+│   ├── clear-cache.js         # Cache management feature
+│   ├── find-similar-code.js   # Similarity search by code snippet
+│   ├── ann-config.js          # ANN configuration tool
+│   ├── lifecycle.js           # CLI lifecycle helpers
+│   └── register.js            # IDE registration logic
 │
-└── scripts/                    # Utility scripts
-    └── clear-cache.js         # Cache management utility
+├── scripts/                    # Utility scripts
+│   ├── clear-cache.js         # Cache management utility
+│   ├── download-model.js      # Optional model pre-download
+│   └── postinstall.js         # Auto-register on install
+│
+└── tools/                      # Developer-only helpers
+    └── scripts/
+        ├── cache-stats.js     # Cache inspection utility
+        └── manual-search.js   # Manual semantic search helper
 ```
 
 ## Module Responsibilities
@@ -39,9 +63,10 @@ heuristic-mcp/
 
 ### lib/config.js
 
-- Loads and validates configuration from config.json
+- Loads and validates configuration from `config.json`
 - Provides default configuration values
-- Resolves file paths
+- Resolves file paths and cache location
+- Applies `SMART_CODING_*` environment variable overrides
 
 ### lib/cache.js
 
@@ -50,114 +75,95 @@ heuristic-mcp/
 - File hash tracking for change detection
 - Load/save operations for disk cache
 - Optional ANN (HNSW) index build/load/save for fast search
+- Supports JSON or binary vector store formats
+
+### lib/cache-utils.js
+
+- Stale cache detection/cleanup for caches without metadata
+- Uses `progress.json` recency to avoid deleting active indexes
+
+### lib/embedding-process.js
+
+- Child-process embedding path for isolation
+- Used to recover from hung or crashing workers
+
+### lib/embedding-worker.js
+
+- Worker-thread embedding path for concurrency
+- Cooperates with worker circuit breaker logic in indexing
+
+### lib/ignore-patterns.js
+
+- Smart ignore patterns derived from detected project type
+
+### lib/json-worker.js / lib/json-writer.js
+
+- Streaming JSON writer and off-thread parsing helpers
+
+### lib/logging.js
+
+- Log file path and directory helpers
+- Console redirection for MCP stdout safety
+
+### lib/cli.js
+
+- Parses CLI flags for server and lifecycle commands
+
+### lib/vector-store-binary.js
+
+- Binary vector store with header + record table + content blocks
+- Mmap-friendly layout, content loaded on demand
 
 ### lib/utils.js
 
-- **cosineSimilarity()** - Vector similarity calculation
+- **dotSimilarity()** - Vector similarity calculation
 - **hashContent()** - MD5 hashing for change detection
 - **smartChunk()** - Language-aware code chunking
 
+### lib/call-graph.js
+
+- Extracts definitions and calls
+- Builds a lightweight call graph for proximity boosting
+
 ### features/hybrid-search.js
 
 - **HybridSearch** class
 - Combines semantic and exact matching
-- Weighted scoring algorithm
-- Result formatting with relevance scores
-- MCP tool: `semantic_search`
+- Recency and call-graph proximity boosting
+- MCP tool: `a_semantic_search`
 
 ### features/index-codebase.js
 
 - **CodebaseIndexer** class
 - File discovery via glob patterns
 - Incremental indexing
-- File watcher for real-time updates
-- MCP tool: `index_codebase`
-
-## Adding New Features
-
-To extend with a new feature:
-
-### 1. Create Feature Module
-
-Create `features/my-feature.js`:
-
-```javascript
-export class MyFeature {
-  constructor(embedder, cache, config) {
-    this.embedder = embedder;
-    this.cache = cache;
-    this.config = config;
-  }
-
-  async execute(params) {
-    // Implementation
-    return {
-      /* results */
-    };
-  }
-}
-
-export function getToolDefinition(config) {
-  return {
-    name: "my_tool",
-    description: "What this tool does",
-    inputSchema: {
-      type: "object",
-      properties: {
-        param1: { type: "string", description: "..." },
-      },
-      required: ["param1"],
-    },
-  };
-}
-
-export async function handleToolCall(request, instance) {
-  const params = request.params.arguments;
-  const result = await instance.execute(params);
-
-  return {
-    content: [
-      {
-        type: "text",
-        text: JSON.stringify(result, null, 2),
-      },
-    ],
-  };
-}
-```
-
-### 2. Register in index.js
+- Optional file watcher for real-time updates
+- Progress tracking via `progress.json`
+- MCP tool: `b_index_codebase`
 
-```javascript
-import * as MyFeature from "./features/my-feature.js";
+### features/clear-cache.js
 
-// In initialize():
-const myFeature = new MyFeature.MyFeature(embedder, cache, config);
+- **CacheClearer** class
+- Clears vector store and cache directory
+- MCP tool: `c_clear_cache`
 
-// Add to features array:
-const features = [
-  // ... existing features
-  {
-    module: MyFeature,
-    instance: myFeature,
-    handler: MyFeature.handleToolCall,
-  },
-];
-```
+### features/find-similar-code.js
 
-### 3. Done!
+- **FindSimilarCode** class
+- Finds semantically similar code snippets
+- MCP tool: `d_find_similar_code`
 
-The feature will automatically:
+### features/ann-config.js
 
-- Be listed in MCP tool discovery
-- Handle incoming tool requests
-- Have access to embeddings and cache
+- **AnnConfigTool** class
+- Runtime ANN tuning and stats
+- MCP tool: `d_ann_config`
 
 ## Configuration Flow
 
 1. User creates/edits `config.json`
 2. `lib/config.js` loads configuration on startup
-3. Configuration merged with defaults
+3. Configuration merged with defaults and env overrides
 4. Passed to all features via constructor
 
 ## Data Flow
@@ -167,7 +173,7 @@ The feature will automatically:
 ```
 User code files
     ↓
-glob pattern matching
+exclude patterns and smart indexing
     ↓
 smartChunk() - split into chunks
     ↓
@@ -187,22 +193,23 @@ embedder - query to vector
     ↓
 ANN candidate search (optional)
     ↓
-cosineSimilarity() - score candidates
+dotSimilarity() - score candidates
     ↓
-exact match boost - adjust scores
+exact match + recency + call-graph boosts
     ↓
 sort and filter - top N results
     ↓
-format output - markdown with syntax highlighting
+format output - markdown with code blocks
 ```
 
 ## Performance Considerations
 
 ### Caching Strategy
 
-- **First Run**: Download model (~90MB), index all files, save cache
+- **First Run**: Download model (if not cached), index all files, save cache
 - **Subsequent Runs**: Load cache from disk, only index changed files
-- **File Changes**: Incremental updates via file watcher
+- **File Changes**: Incremental updates via file watcher (if enabled)
+- **Binary Store**: Optional on-disk vector/content storage to reduce JS heap usage
 
 ### Memory Usage
 
@@ -218,75 +225,3 @@ Approximate memory usage:
 - Reduce `chunkSize` for large codebases
 - Disable `watchFiles` if not needed
 - Use `excludePatterns` aggressively
-- Limit `fileExtensions` to relevant types
-
-## Future Feature Ideas
-
-Potential features to add following this architecture:
-
-1. **Code Complexity Analysis**
-
-   - Cyclomatic complexity scoring
-   - Technical debt detection
-
-2. **Pattern Detection**
-
-   - Anti-pattern identification
-   - Best practice recommendations
-
-3. **Documentation Generation**
-
-   - Auto-generate function docs
-   - README generation from code
-
-4. **Refactoring Suggestions**
-
-   - Code smell detection
-   - Automated fix suggestions
-
-5. **Test Coverage Analysis**
-
-   - Identify untested code paths
-   - Generate test templates
-
-6. **Dependency Analysis**
-   - Import/export graph
-   - Dead code detection
-
-Each feature would follow the same pattern:
-
-- Class in `features/` directory
-- Access to embedder, cache, config
-- MCP tool definition and handler
-- Registration in feature array
-
-## Testing Strategy
-
-Recommended testing approach:
-
-1. **Unit Tests**: lib/ modules
-
-   - Test utilities in isolation
-   - Mock dependencies
-
-2. **Integration Tests**: features/
-
-   - Test with sample codebases
-   - Verify MCP tool contracts
-
-3. **E2E Tests**: Full workflow
-   - Index → Search → Results
-   - File watching behavior
-   - Cache persistence
-
-## Error Handling
-
-Each module follows defensive error handling:
-
-- Config errors → use defaults
-- File read errors → log and skip
-- Embedding errors → retry or skip chunk
-- Cache errors → log but continue
-- Unknown tools → return helpful error message
-
-All errors logged to stderr for MCP protocol compatibility.

package/CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing to Heuristic MCP
 
-Thank you for your interest in contributing! This document provides guidelines for contributing to the project.
+Thank you for your interest in contributing. This document provides guidelines for contributing to the project.
 
 ## Getting Started
 
@@ -26,13 +26,14 @@ npm run dev
 
 ## Project Structure
 
-See [ARCHITECTURE.md](./ARCHITECTURE.md) for detailed information about the modular architecture.
+See `ARCHITECTURE.md` for detailed information about the modular architecture.
 
 Key directories:
 
 - `lib/` - Core libraries and utilities
 - `features/` - Pluggable feature modules
 - `scripts/` - Utility scripts
+- `tools/` - Developer-only helpers
 
 ## Development Guidelines
 
@@ -42,7 +43,7 @@ Key directories:
 - Follow existing code patterns
 - Use meaningful variable and function names
 - Add comments for complex logic
-- No emojis in code or documentation
+- Avoid emojis in code comments and logs unless they are part of user-facing CLI output
 
 ### File Organization
 
@@ -50,7 +51,7 @@ Key directories:
 // Standard file structure for features:
 
 // 1. Imports
-import { dependency } from "package";
+import { dependency } from 'package';
 
 // 2. Class definition
 export class FeatureName {
@@ -83,20 +84,16 @@ export async function handleToolCall(request, instance) {
 try {
   // operation
 } catch (error) {
-  console.error("[Module] Error description:", error.message);
+  console.error('[Module] Error description:', error.message);
   // Continue execution or return default value
 }
 ```
 
 ### Logging
 
-Use `console.error()` for all logs (MCP protocol requirement):
-
-```javascript
-console.error("[FeatureName] Informational message");
-console.error("[FeatureName] Warning:", details);
-console.error("[FeatureName] Error:", error.message);
-```
+- Use `console.info()` for normal server lifecycle output (redirected to logs in MCP mode)
+- Use `console.warn()` for non-fatal issues and `console.error()` for errors
+- CLI utilities and install scripts may use `console.info()` for user-friendly output
 
 ## Adding New Features
 
@@ -116,23 +113,23 @@ export class YourFeature {
 
   async execute(params) {
     // Implementation
-    return { result: "data" };
+    return { result: 'data' };
   }
 }
 
 export function getToolDefinition(config) {
   return {
-    name: "your_tool",
-    description: "Clear description of what the tool does",
+    name: 'your_tool',
+    description: 'Clear description of what the tool does',
     inputSchema: {
-      type: "object",
+      type: 'object',
       properties: {
         param: {
-          type: "string",
-          description: "Parameter description",
+          type: 'string',
+          description: 'Parameter description',
         },
       },
-      required: ["param"],
+      required: ['param'],
     },
   };
 }
@@ -144,7 +141,7 @@ export async function handleToolCall(request, instance) {
   return {
     content: [
       {
-        type: "text",
+        type: 'text',
         text: JSON.stringify(result, null, 2),
       },
     ],
@@ -157,7 +154,7 @@ export async function handleToolCall(request, instance) {
 Update `index.js`:
 
 ```javascript
-import * as YourFeature from "./features/your-feature.js";
+import * as YourFeature from './features/your-feature.js';
 
 // In initialize():
 const yourFeature = new YourFeature.YourFeature(embedder, cache, config);
@@ -172,17 +169,27 @@ features.push({
 
 3. **Test Your Feature**
 
-- Test with sample codebase
+- Test with a sample codebase
 - Verify MCP tool contract
-- Check error handling
-- Validate output format
+- Check error handling and output format
 
 4. **Document Your Feature**
 
 - Add to README.md features section
-- Create examples in EXAMPLES.md
 - Update ARCHITECTURE.md if needed
 
+## Testing
+
+```bash
+npm test
+```
+
+By default, tests use a mock embedder to avoid network/model downloads. To run the real model tests:
+
+```bash
+USE_REAL_EMBEDDER=true npm test
+```
+
 ## Pull Request Process
 
 1. **Fork the repository**
@@ -218,91 +225,3 @@ Follow commit message conventions:
 ```bash
 git push origin feature/your-feature-name
 ```
-
-6. **Create Pull Request**
-
-- Provide clear description of changes
-- Reference any related issues
-- Include examples of usage
-- Explain testing performed
-
-## Testing
-
-### Manual Testing
-
-```bash
-# Test with a sample project
-cd /path/to/test/project
-node /path/to/heuristic-mcp/index.js
-
-# In another terminal, send MCP requests
-```
-
-### Testing MCP Tools
-
-Create a test script:
-
-```javascript
-// test-tool.js
-import { Server } from "@modelcontextprotocol/sdk/server/index.js";
-// ... setup and test your tool
-```
-
-## Configuration Changes
-
-If adding new configuration options:
-
-1. Update `lib/config.js` with new default values
-2. Document in README.md
-3. Add examples to EXAMPLES.md
-4. Consider backward compatibility
-
-## Documentation
-
-All documentation should:
-
-- Be clear and concise
-- Include code examples
-- Avoid emojis
-- Use proper markdown formatting
-- Be kept up-to-date with code changes
-
-## Code Review Checklist
-
-Before submitting a PR, verify:
-
-- [ ] Code follows project style guidelines
-- [ ] No console.log (use console.error for MCP)
-- [ ] Error handling is implemented
-- [ ] Configuration changes are documented
-- [ ] README.md is updated if needed
-- [ ] No breaking changes without discussion
-- [ ] Comments explain complex logic
-- [ ] No emojis in code or documentation
-
-## Feature Ideas
-
-Looking for ideas? Consider implementing:
-
-- Code complexity analysis
-- Pattern detection and anti-pattern identification
-- Documentation generation
-- Refactoring suggestions
-- Test coverage analysis
-- Dependency graph visualization
-- Performance profiling integration
-- Multi-language translation support
-
-## Questions and Support
-
-- **Issues**: Use GitHub Issues for bugs and feature requests
-- **Discussions**: Use GitHub Discussions for questions
-- **Email**: Contact Softerist via website
-
-## License
-
-By contributing, you agree that your contributions will be licensed under the MIT License.
-
----
-
-Thank you for contributing to Heuristic MCP!