@afterxleep/doc-bot 1.10.0 → 1.13.0

package/README.md

@@ -3,319 +3,233 @@
 [![npm version](https://img.shields.io/npm/v/@afterxleep/doc-bot)](https://www.npmjs.com/package/@afterxleep/doc-bot)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-A generic MCP (Model Context Protocol) server that provides intelligent documentation and rules for any project. Works with any MCP-compatible AI tools and IDEs.
-
-It's platform agnostic and designed to replace and extend the rule systems provided by different IDEs, such as Cursor (Cursor Rules) or Claude (CLAUDE.md). Instead of relying on separate rule-sets for each tool, you can use doc-bot to provide unified documentation for agentic coding across all your AI assistants.
+An intelligent MCP (Model Context Protocol) server that gives AI assistants like Claude and Cursor deep understanding of your project through smart documentation management.
 
 ## What is doc-bot?
 
-doc-bot is an intelligent documentation server that:
-- 🧠 **Auto-indexes** content for smart inference, based on metadata and keywords
-- 🤖 **Provides agentic tools** to query, and update your documentation
-- 🔄 **Updates** automatically when docs change
-- 📚 **Supports Docsets** for searching official API documentation alongside your custom docs
-
-## Why MCP Instead of Static Rules?
-
-IDE's use static rule files (like Cursor Rules or Copilot's .github/copilot-instructions.md), and each one has their own format, metadata and approach.
-
-### 🚀 Dynamic Search vs Static Rules
-
-**Static Systems:**
-- All rules must fit in a single file or limited token window
-- AI reads everything, even irrelevant rules
-- No way to search or filter documentation (besides plain 'grep')
-- Rules compete for context space
-
-**MCP with doc-bot:**
-- AI searches for exactly what it needs
-- Unlimited documentation size - only relevant parts are retrieved
-- Smart keyword and pattern matching
-- Context window used efficiently
-
-### 🧠 Contextual Intelligence
-
-**Static Systems:**
-- Duplicate or symlinked rules to work with multiple agents
-- Agents use `grep` for basic text-base searching
-
-**MCP with doc-bot:**
-- AI searches for relevant documentation based on your query
-- Context-aware suggestions from your actual questions
-- Different documentation retrieved for different tasks
-- Intelligent inference from keywords and search terms
+doc-bot is a documentation server that enhances AI coding assistants by providing:
+- 🧠 **Smart search** through your project documentation
+- 📖 **Contextual rules** that apply based on what you're working on
+- 🔄 **Live updates** as your documentation changes
+- 📚 **API references** from official documentation (via Docsets)
+- 🤖 **MCP tools** for AI agents to query and understand your project
 
-### 📈 Scalability Without Limits
+## Why doc-bot?
 
-**Static Systems:**
-- Limited by token count
-- Adding more rules has impact in your context window
-- Documentation competes with your actual code for context
+Traditional AI assistants have limited context windows and no understanding of your specific project. doc-bot solves this by:
 
-**MCP with doc-bot:**
-- Store thousands of documentation files
-- No token limit - documentation lives outside the context
-- AI retrieves only what's needed
-- Your context window stays free for actual work
+1. **Providing project-specific knowledge** - Your conventions, patterns, and rules
+2. **Searching intelligently** - AI finds exactly what it needs without cluttering context
+3. **Scaling infinitely** - Thousands of docs without token limits
+4. **Staying current** - Live reload ensures AI always has latest information
 
-### 🔄 Live Updates
+## How It Works
 
-**Static Systems:**
-- Changes require restarting your AI/IDE
-- No way to know if rules are current
-- Manual synchronization across tools and AI agents
-
-**MCP with doc-bot:**
-- Hot reload on documentation changes
-- Always serves the latest version
-- Single source of truth for all AI tools
-
-### 🎯 Smart Discovery
-
-**Static Systems:**
-- AI doesn't know what documentation exists
-- Users must know to ask specific questions
-- No exploration or discovery capabilities
-- AI agents rely on basic grep searches through codebases to infer project patterns
-
-**MCP with doc-bot:**
-- AI can list all available documentation
-- Discovers relevant docs automatically
-- Suggests documentation based on relevance
-- Searchable knowledge base with intelligent ranking
-- No need for AI to grep through your codebase - dedicated search engine
-
-## Installation
-
-1. **Create your documentation folder** in your project root (see organization section below)
-
-2. **Add doc-bot to your MCP-compatible AI tool configuration**:
-
-   ```json
-   {
-     "mcpServers": {
-       "doc-bot": {
-         "command": "npx",
-         "args": ["@afterxleep/doc-bot@latest"]
-       }
-     }
-   }
-   ```
-
-   **Custom docs folder:**
-   ```json
-   {
-     "mcpServers": {
-       "doc-bot": {
-         "command": "npx",
-         "args": ["@afterxleep/doc-bot@latest", "--docs", "./my-custom-docs"]
-       }
-     }
-   }
-   ```
-   
-   **With Docsets support (for official API documentation):**
-   ```json
-   {
-     "mcpServers": {
-       "doc-bot": {
-         "command": "npx",
-         "args": ["@afterxleep/doc-bot@latest", "--docs", "./docs", "--docsets", "~/MyDocSets"]
-       }
-     }
-   }
-   ```
-   Note: If a relative path does not work, you can use VSCode `${workspaceFolder}`environment variable `${workspaceFolder}/my-custom-docs`
+doc-bot acts as a bridge between your documentation and AI assistants:
 
+```
+Your Project Documentation → doc-bot → MCP Protocol → AI Assistant (Claude, Cursor, etc.)
+```
 
-   **With verbose logging (for debugging):**
-   ```json
-   {
-     "mcpServers": {
-       "doc-bot": {
-         "command": "npx",
-         "args": ["@afterxleep/doc-bot@latest", "--verbose"]
-       }
-     }
-   }
-   ```
+When you ask your AI assistant to write code, it can:
+1. Check your project's coding standards
+2. Search for relevant documentation
+3. Find API references and examples
+4. Follow your team's specific patterns
 
-3. **Restart your AI tool**
+## Quick Start
 
-4. **Ensure Agent Compliance** (Essential): Add the expert-engineered integration protocol to guarantee your agent uses doc-bot:
+### 1. Install doc-bot
 
-   **⚡ Setup**: Copy the rule from [`AGENT_INTEGRATION_RULE.txt`](./AGENT_INTEGRATION_RULE.txt) into your agent configuration.
-   **🎯 Why This Matters**: Without this rule, agents may default to general knowledge instead of your doc-bot documentation.
+Add doc-bot to your AI assistant's configuration:
 
-   **Platform-Specific Instructions**:
-   - **Claude Code**: Add rule to your global `CLAUDE.md`
-   - **Cursor**: Create a `.mdc` file in `.cursor/rules/` directory with `alwaysApply: true`
-   - **GitHub Copilot**: Add rule to `.github/copilot-instructions.md`
-   - **Continue.dev**: Add rule to system prompt configuration
+**For Claude Desktop or Claude Code:**
+```json
+{
+  "mcpServers": {
+    "doc-bot": {
+      "command": "npx",
+      "args": ["@afterxleep/doc-bot@latest"]
+    }
+  }
+}
+```
 
-   
+**Location of config file:**
+- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
+- Windows: `%APPDATA%\Claude\claude_desktop_config.json`
+- Linux: `~/.config/Claude/claude_desktop_config.json`
 
-## How to organize your documentation
+### 2. Create Your Documentation
 
-Create a `doc-bot/` folder in your project root with markdown files using frontmatter:
+Create a `doc-bot` folder in your project root and add markdown files:
 
 ```
 your-project/
 ├── doc-bot/
-│   ├── coding-standards.md     # Global rule (alwaysApply: true)
-│   ├── security.md             # Global rule (alwaysApply: true) 
-│   ├── testing.md              # Contextual rule (alwaysApply: false)
-│   ├── api-development.md      # Contextual rule (alwaysApply: false)
-│   └── troubleshooting.md      # Contextual rule (alwaysApply: false)
+│   ├── coding-standards.md
+│   ├── api-patterns.md
+│   ├── testing-guide.md
+│   └── architecture.md
+├── src/
 └── package.json
 ```
 
-**Note:** The `doc-bot` folder is the default location. You can use any folder name by specifying it with the `--docs` option.
+### 3. Test It
 
-### Documentation types:
+Ask your AI assistant: "What are the coding standards for this project?"
 
-- **Global Rules** (`alwaysApply: true`): Critical guidelines applied to every AI interaction
-- **Contextual Rules** (`alwaysApply: false`): Rules applied based on file patterns and context
+## Project Documentation
 
-### Example documentation files:
-
-**Global Rule Example** (`doc-bot/coding-standards.md`):
-```markdown
----
-alwaysApply: true
-title: "Coding Standards"
-description: "Core coding standards that apply to all code"
-keywords: ["code-quality", "standards", "best-practices"]
----
+doc-bot treats your project documentation as a searchable knowledge base for AI assistants.
 
-# Coding Standards
+### Documentation Format
 
-- Use 2 spaces for indentation
-- Maximum line length: 100 characters
-- Always use const/let, never var
-- Prefer async/await over promises
-- Write descriptive variable names
-```
+Create markdown files with frontmatter metadata:
 
-**Contextual Rule Example** (`doc-bot/testing.md`):
 ```markdown
 ---
-alwaysApply: false
-title: "Testing Guide"
-description: "How to write and run tests"
-keywords: ["testing", "jest", "tdd", "unit-tests"]
-filePatterns: ["*.test.js", "*.spec.js", "__tests__/**/*"]
+title: "React Component Guidelines"
+description: "Standards for building React components"
+keywords: ["react", "components", "frontend", "jsx"]
 ---
 
-# Testing Guide
-
-All test files should:
-- Use describe/it blocks for organization
-- Include both positive and negative test cases
-- Mock external dependencies
-- Aim for 80%+ code coverage
+# React Component Guidelines
 
-Run tests with: `npm test`
+- Use functional components with hooks
+- Follow PascalCase naming
+- Keep components under 200 lines
+- Write tests for all components
 ```
 
-## Frontmatter-Based Configuration
-
-doc-bot uses frontmatter in your markdown files to automatically detect and categorize rules.
-
-### Frontmatter Fields:
-
-- **`alwaysApply: true`** - Global rules applied to every AI interaction
-- **`alwaysApply: false`** - Contextual rules searched and applied based on relevance
-- **`keywords: ["list", "of", "keywords"]`** - For smart indexing and search
-- **`filePatterns: ["*.js", "src/**/*.ts"]`** - Apply docs to specific files (see below)
-- **`title`** and **`description`** - Standard metadata
-- **`confidence: 0.9`** - Relevance confidence score (0-1)
-
-### 🎯 Automatic Intelligence
+### Frontmatter Options
 
-doc-bot automatically analyzes your documentation to provide smart suggestions:
+| Field | Type | Description | Example |
+|-------|------|-------------|---------|
+| `title` | string | Document title (required) | "API Guidelines" |
+| `description` | string | Brief description | "REST API design patterns" |
+| `keywords` | array | Search keywords | ["api", "rest", "http"] |
+| `alwaysApply` | boolean | Apply to all queries | true/false |
+| `filePatterns` | array | Apply to specific files | ["*.test.js", "**/*.spec.ts"] |
 
-- **Keyword-based search** from frontmatter metadata
-- **Multi-term search** with fuzzy matching capabilities
-- **Smart inference** from documentation content
-- **Automatic indexing** - no manual configuration needed
+### How Search Works
 
-### Writing effective documentation
+1. **Intelligent Parsing** - Queries are parsed, stop words removed
+2. **Multi-field Matching** - Searches title, description, keywords, and content
+3. **Relevance Scoring** - Results ranked by relevance (exact matches score highest)
+4. **Context Extraction** - Returns snippets showing matched content
 
-For best results, include descriptive frontmatter:
+### Types of Documentation
 
+#### Global Rules (Always Apply)
 ```markdown
 ---
-alwaysApply: false
-title: "React Component Guidelines"
-description: "Best practices for building React components"
-keywords: ["react", "components", "hooks", "jsx"]
+title: "Coding Standards"
+alwaysApply: true
 ---
-
-# React Component Guidelines
-
-Your documentation content here...
+Rules that apply to every file in your project
 ```
 
-### File Pattern Matching
-
-doc-bot supports contextual documentation using file patterns. Documentation can be targeted to specific files:
+#### Contextual Documentation
+```markdown
+---
+title: "Testing Guide"
+filePatterns: ["*.test.js", "*.spec.ts"]
+---
+Documentation that only applies to test files
+```
 
+#### Searchable References
 ```markdown
 ---
-alwaysApply: false
-title: "React Testing Guide"
-keywords: ["testing", "jest", "react"]
-filePatterns: ["*.test.jsx", "*.test.tsx", "__tests__/**/*"]
+title: "Database Schema"
+keywords: ["database", "postgres", "schema", "migrations"]
 ---
+Documentation found through search queries
+```
 
-# React Testing Guide
 
-This documentation appears only when working with test files...
-```
+## Docsets (API Documentation)
 
-**Pattern Examples:**
-- `*.js` - All JavaScript files
-- `src/**/*.ts` - TypeScript files in src directory
-- `[Tt]est.js` - Test.js or test.js
-- `*.{jsx,tsx}` - React component files
+doc-bot can also search official API documentation from Docsets, giving your AI assistant access to comprehensive framework and library references.
 
-When AI requests documentation for a specific file (e.g., `Button.test.jsx`), only docs with matching patterns are returned.
+### What are Docsets?
 
-## Development setup
+Docsets are pre-built documentation databases containing official docs for:
+- Programming languages (Python, JavaScript, Go, etc.)
+- Frameworks (React, Vue, Django, Rails, etc.)
+- Libraries (NumPy, Express, jQuery, etc.)
+- Platforms (iOS, Android, AWS, etc.)
 
-### Running locally
+### Setting Up Docsets
 
-1. **Clone the repository:**
-   ```bash
-   git clone https://github.com/afterxleep/doc-bot.git
-   cd doc-bot
+1. **Option A: Ask your AI assistant to install directly**:
+   
+   From a URL:
    ```
-
-2. **Install dependencies:**
-   ```bash
-   npm install
+   Use the add_docset tool to install Swift documentation from https://kapeli.com/feeds/Swift.tgz
    ```
-
-3. **Run the server (uses built-in doc-bot/ folder):**
-   ```bash
-   npm start
+   
+   From a local file:
+   ```
+   Use the add_docset tool to install the docset at /Users/me/Downloads/React.docset
    ```
 
-4. **Run with file watching (recommended for development):**
-   ```bash
-   npm run dev
+2. **Manage your docsets**:
    ```
-5. **Run tests:**
-   ```bash
-   npm test
+   List all installed docsets
+   Remove docset with ID abc123
    ```
 
-**Note:** This is an MCP server that communicates via stdio transport, not HTTP. When running locally, it will start the MCP server and show you the configuration to add to your MCP client (like Claude Code).
+   Docsets are automatically stored in `~/Developer/DocSets` by default.
+
+### Docset Sources
 
-### Testing your setup
+- **User Contributed Docsets**: https://github.com/Kapeli/Dash-User-Contributions
+- **Docset Generation Tools**: https://github.com/Kapeli/docset-generator
 
-Ask your AI assistant something like "What documentation is available?" to test that doc-bot is working.
+Popular docsets available:
+- Programming Languages: Python, JavaScript, Go, Rust, Swift
+- Web Frameworks: React, Vue, Angular, Django, Rails
+- Mobile: iOS, Android, React Native, Flutter
+- Databases: PostgreSQL, MySQL, MongoDB, Redis
+- Cloud: AWS, Google Cloud, Azure
+
+3. **Configure custom path** (optional):
+   ```json
+   {
+     "mcpServers": {
+       "doc-bot": {
+         "command": "npx",
+         "args": ["@afterxleep/doc-bot@latest", "--docsets", "/path/to/docsets"]
+       }
+     }
+   }
+   ```
+
+### How Docset Search Works
+
+- **Unified Search**: One query searches both your docs and API docs
+- **Smart Prioritization**: Your project docs are boosted 5x in relevance
+- **API Exploration**: Use `explore_api` tool to discover related classes, methods
+- **Performance**: Parallel search across multiple docsets with caching
+
+## Available Tools
+
+doc-bot provides these tools to AI assistants:
+
+| Tool | Purpose | Example Use |
+|------|---------|-------------|
+| `check_project_rules` | Get rules before writing code | "What patterns should I follow?" |
+| `search_documentation` | Search all documentation | "How do I implement auth?" |
+| `get_global_rules` | Get always-apply rules | "What are the coding standards?" |
+| `get_file_docs` | Get file-specific docs | "Rules for Button.test.jsx" |
+| `explore_api` | Explore API documentation | "Show me URLSession methods" |
+| `add_docset` | Install new docset | "Add Swift docs from URL" |
+| `remove_docset` | Remove installed docset | "Remove docset abc123" |
+| `list_docsets` | List all docsets | "Show installed docsets" |
+
+## Configuration Options
 
 ### CLI Options
 
@@ -323,105 +237,93 @@ Ask your AI assistant something like "What documentation is available?" to test
 doc-bot [options]
 
 Options:
-  -d, --docs <path>        Path to docs folder (default: doc-bot)
+  -d, --docs <path>        Path to docs folder (default: ./doc-bot)
+  -s, --docsets <path>     Path to docsets folder (default: ~/Developer/DocSets)
   -v, --verbose           Enable verbose logging
   -w, --watch             Watch for file changes
-  -h, --help              Show help
+  -h, --help              Display help
 ```
 
-**Example usage:**
-```bash
-# Basic usage with default doc-bot folder
-doc-bot
-
-# Specify a custom docs folder
-doc-bot --docs ./my-docs
-
-# With verbose logging and file watching
-doc-bot --verbose --watch
+### Advanced Configuration
+
+```json
+{
+  "mcpServers": {
+    "doc-bot": {
+      "command": "npx",
+      "args": [
+        "@afterxleep/doc-bot@latest",
+        "--docs", "./documentation",
+        "--docsets", "/Library/Application Support/Dash/DocSets",
+        "--verbose",
+        "--watch"
+      ]
+    }
+  }
+}
 ```
 
-## Publishing and Development
+## Documentation
 
-### Local Development
+- **[API Reference](doc-bot/api-reference.md)** - Complete reference for all MCP tools
+- **[Architecture Guide](doc-bot/architecture.md)** - Technical architecture and components
+- **[Configuration Guide](doc-bot/configuration-guide.md)** - All configuration options
+- **[Troubleshooting Guide](doc-bot/troubleshooting.md)** - Common issues and solutions
+- **[Examples & Best Practices](doc-bot/examples-and-best-practices.md)** - Real-world usage examples
+- **[Contributing Guide](doc-bot/contributing.md)** - How to contribute to doc-bot
 
-1. **Clone and setup:**
-   ```bash
-   git clone https://github.com/afterxleep/doc-bot.git
-   cd doc-bot
-   npm install
-   ```
+## Best Practices
 
-2. **Run locally:**
-   ```bash
-   npm run dev    # With file watching
-   npm start      # Basic run
-   npm test       # Run tests
-   ```
+### Writing Effective Documentation
 
-3. **Test with Claude Code:**
-   Add to your Claude Code config:
-   ```json
-   {
-     "mcpServers": {
-       "doc-bot": {
-         "command": "node",
-         "args": ["/path/to/doc-bot/bin/doc-bot.js", "--verbose", "--watch"]
-       }
-     }
-   }
+1. **Use descriptive titles and keywords**
+   ```markdown
+   ---
+   title: "Authentication Flow"
+   keywords: ["auth", "login", "jwt", "security", "authentication"]
+   ---
    ```
 
-### Publishing to npm
-
-1. **Test the package:**
-   ```bash
-   npm run test
-   npm run lint
+2. **Apply rules contextually**
+   ```markdown
+   ---
+   filePatterns: ["**/auth/**", "*.auth.js"]
+   ---
    ```
 
-2. **Update version:**
-   ```bash
-   npm version patch|minor|major
-   ```
+3. **Keep docs focused** - One topic per file
 
-3. **Publish:**
-   ```bash
-   npm publish
-   ```
+4. **Include examples** - Show, don't just tell
 
-### Push Changes
+### Optimizing Search
 
-```bash
-git add .
-git commit -m "feat: your feature description"
-git push origin main
-git push --tags  # Push version tags
-```
+- Include synonyms in keywords: `["test", "testing", "spec", "jest"]`
+- Use clear section headers for better snippet extraction
+- Add descriptions to improve search relevance
 
-## Docset Support
+## Why MCP over Static Rules?
 
-doc-bot now supports [Docsets](https://kapeli.com/docsets) - pre-indexed documentation used by Dash, Zeal, and Velocity. This allows you to search official API documentation alongside your custom project docs.
+Unlike static `.cursorrules` or `.github/copilot-instructions.md` files:
 
-### Key Features
-- Search iOS, macOS, Swift, and other official documentation
-- Install docsets from URLs or local files
-- Unified search across both custom docs and official APIs
-- No manual HTML parsing required
+- **Dynamic**: AI searches for what it needs instead of reading everything
+- **Scalable**: Unlimited docs without token limits
+- **Intelligent**: Context-aware documentation based on current file
+- **Unified**: Works with any MCP-compatible AI tool
+- **Live**: Hot reload on documentation changes
 
-### Quick Start
-1. **Install a docset**: Use the `add_docset` tool with a URL or local path
-2. **Search documentation**: Use `search_docsets` for docsets only, or `search_all` for unified results
-3. **Manage docsets**: List installed docsets with `list_docsets`, remove with `remove_docset`
+## Contributing
 
-See [DOCSETS.md](./DOCSETS.md) for detailed documentation.
+See our [Contributing Guide](doc-bot/contributing.md) for development setup and guidelines.
 
 ## License
 
-MIT License - see the [LICENSE](LICENSE) file for details.
+MIT - See [LICENSE](LICENSE) for details.
+
+## Support
 
-## Links
+- **Issues**: [GitHub Issues](https://github.com/afterxleep/doc-bot/issues)
+- **Discussions**: [GitHub Discussions](https://github.com/afterxleep/doc-bot/discussions)
+
+---
 
-- [npm package](https://www.npmjs.com/package/@afterxleep/doc-bot)
-- [GitHub repository](https://github.com/afterxleep/doc-bot)
-- [Model Context Protocol](https://github.com/modelcontextprotocol/specification)
+Built with ❤️ in Spain