bluera-knowledge 0.13.0 → 0.13.2

package/README.md

@@ -35,28 +35,19 @@ All searchable in milliseconds, no rate limits, fully offline.
 
 - [Installation](#-installation)
 - [Why Bluera Knowledge?](#-why-bluera-knowledge)
-- [Token Efficiency](#-token-efficiency)
+- [When to Query BK](#-when-claude-code-should-query-bk)
 - [Quick Start](#-quick-start)
 - [Features](#-features)
 - [How It Works](#-how-it-works)
 - [User Interface](#-user-interface)
 - [Background Jobs](#-background-jobs)
-- [Commands](#-commands)
-- [Crawler Architecture](#-crawler-architecture)
 - [Use Cases](#-use-cases)
-- [Dependencies](#-dependencies)
-- [Troubleshooting](#-troubleshooting)
-- [MCP Integration](#-mcp-integration)
-- [CLI Tool](#️-cli-tool)
 - [Skills for Claude Code](#-skills-for-claude-code)
 - [Data Storage](#-data-storage)
-- [Development](#-development)
-  - [Setup](#-setup)
-  - [Claude Code Settings](#️-claude-code-settings-recommended)
-  - [Commands / Scripts](#-commands)
-  - [Releasing](#-releasing)
-  - [Testing](#-testing-locally)
+- [Troubleshooting](#-troubleshooting)
+- [Dependencies](#-dependencies)
 - [Technologies](#-technologies)
+- [Documentation](#-documentation)
 - [Contributing](#-contributing)
 - [License](#-license)
 
@@ -163,92 +154,6 @@ BK is cheap (~100ms, no rate limits), authoritative (actual source code), and co
 
 ---
 
-## 💰 Token Efficiency
-
-Beyond speed and accuracy, Bluera Knowledge can **significantly reduce token consumption** for code-related queries—typically saving 60-75% compared to web search approaches.
-
-### 📊 How It Works
-
-**Without Bluera Knowledge:**
-- Web searches return 5-10 results (~500-2,000 tokens each)
-- Total per search: **3,000-10,000 tokens**
-- Often need multiple searches to find the right answer
-- Lower signal-to-noise ratio (blog posts mixed with actual docs)
-
-**With Bluera Knowledge:**
-- Semantic search returns top 10 relevant code chunks (~200-400 tokens each)
-- Structured metadata (file paths, imports, purpose)
-- Total per search: **1,500-3,000 tokens**
-- Higher relevance due to vector search (fewer follow-up queries needed)
-
-### 🎯 Real-World Examples
-
-#### Example 1: Library Implementation Question
-
-**Question:** "How does Express handle middleware errors?"
-
-| Approach | Token Cost | Result |
-|----------|-----------|--------|
-| **Web Search** | ~8,000 tokens<br/>(3 searches: general query → refined query → source code) | Blog posts + Stack Overflow + eventual guess |
-| **Bluera Knowledge** | ~2,000 tokens<br/>(1 semantic search) | Actual Express source code, authoritative |
-| **Savings** | **75% fewer tokens** ✅ | Higher accuracy |
-
-#### Example 2: Dependency Exploration
-
-**Question:** "How does LanceDB's vector search work?"
-
-| Approach | Token Cost | Result |
-|----------|-----------|--------|
-| **Web Search** | ~9,500 tokens<br/>(General docs → API docs → fetch specific page) | Documentation, might miss implementation details |
-| **Bluera Knowledge** | ~1,500 tokens<br/>(Search returns source + tests + examples) | Source code from Python + Rust implementation |
-| **Savings** | **84% fewer tokens** ✅ | Complete picture |
-
-#### Example 3: Version-Specific Behavior
-
-**Question:** "What changed in React 18's useEffect cleanup?"
-
-| Approach | Token Cost | Result |
-|----------|-----------|--------|
-| **Training Data** | 0 tokens (but might be outdated) | Uncertain if accurate for React 18 |
-| **Web Search** | ~5,000 tokens<br/>(Search changelog → blog posts → docs) | Mix of React 17 and 18 info |
-| **Bluera Knowledge** | ~2,000 tokens<br/>(Search indexed React 18 source) | Exact React 18 implementation |
-| **Savings** | **60% fewer tokens** ✅ | Version-accurate |
-
-### ⚖️ When BK Uses More Tokens
-
-Bluera Knowledge isn't always the most token-efficient choice:
-
-| Scenario | Best Approach | Why |
-|----------|---------------|-----|
-| **Simple concept questions**<br/>("What is a JavaScript closure?") | Training data | Claude already knows this (0 tokens) |
-| **Current events**<br/>("Latest Next.js 15 release notes") | Web search | BK only has what you've indexed |
-| **General advice**<br/>("How to structure a React app?") | Training data | Opinion-based, not code-specific |
-
-### 📈 Summary: Token Savings by Query Type
-
-| Query Type | Typical Token Savings | When to Use BK |
-|------------|----------------------|----------------|
-| **Library internals** | 60-75% | ✅ Always |
-| **Version-specific behavior** | 50-70% | ✅ Always |
-| **"How does X work internally?"** | 70-85% | ✅ Always |
-| **API usage examples** | 40-60% | ✅ Recommended |
-| **General concepts** | -100% (uses more) | ❌ Skip BK |
-| **Current events** | -100% (uses more) | ❌ Skip BK |
-
-### 💡 Best Practice
-
-**Default to BK for library questions.** It's cheap, fast, and authoritative:
-
-| Question Type | Action | Why |
-|--------------|--------|-----|
-| Library internals, APIs, errors, versions, config | **Query BK first** | Source code is definitive, 60-85% token savings |
-| General programming concepts | Skip BK | Training data is sufficient |
-| Breaking news, release notes | Web search | BK only has indexed content |
-
-The plugin's Skills teach Claude Code these patterns automatically. When in doubt about a dependency, query BK—it's faster and more accurate than guessing or web searching.
-
----
-
 ## 🚀 Quick Start
 
 ### Using Claude Code Plugin
@@ -487,483 +392,54 @@ Background jobs include significant performance optimizations:
 
 ---
 
-## 📖 Quick Reference
-
-| Command | Purpose | Arguments |
-|---------|---------|-----------|
-| 🔬 `/bluera-knowledge:suggest` | Analyze project dependencies | None |
-| 📦 `/bluera-knowledge:add-repo` | Clone and index Git repository | `<url> [--name=<name>] [--branch=<branch>]` |
-| 📁 `/bluera-knowledge:add-folder` | Index local folder | `<path> --name=<name>` |
-| 🔍 `/bluera-knowledge:search` | Search knowledge stores | `"<query>" [--stores=<names>] [--limit=<N>]` |
-| 📋 `/bluera-knowledge:stores` | List all stores | None |
-| 🔄 `/bluera-knowledge:index` | Re-index a store | `<store-name-or-id>` |
-| 🗑️ `/bluera-knowledge:remove-store` | Delete a store and all data | `<store-name-or-id>` |
-| 🌐 `/bluera-knowledge:crawl` | Crawl web pages | `<url> <store-name> [--crawl "<instruction>"]` |
-| 🔁 `/bluera-knowledge:sync` | Sync stores from definitions config | `[--dry-run] [--prune]` |
-
----
-
-## 💻 Commands
+## 🎯 Use Cases
 
-### 🔬 `/bluera-knowledge:suggest`
+### Dependency Source Code
 
-**Analyze your project to suggest libraries worth indexing as knowledge stores**
+Provide AI agents with canonical dependency implementation details:
 
 ```bash
 /bluera-knowledge:suggest
-```
-
-Scans source files, counts import statements, and suggests the top 5 most-used dependencies with their repository URLs.
-
-**Supported languages:**
-| Language | Manifest File | Registry |
-|----------|---------------|----------|
-| JavaScript/TypeScript | `package.json` | NPM |
-| Python | `requirements.txt`, `pyproject.toml` | PyPI |
-| Rust | `Cargo.toml` | crates.io |
-| Go | `go.mod` | Go modules |
-
-<details>
-<summary><b>📊 Expected Output</b></summary>
-
-```
-## Dependency Analysis
-
-Scanned 342 source files and found 24 dependencies.
-
-### Top Dependencies by Usage
-
-1. **react** (156 imports across 87 files)
-   Repository: https://github.com/facebook/react
-
-   Add with:
-   /bluera-knowledge:add-repo https://github.com/facebook/react --name=react
-
-2. **vitest** (40 imports across 40 files)
-   Repository: https://github.com/vitest-dev/vitest
-
-   Add with:
-   /bluera-knowledge:add-repo https://github.com/vitest-dev/vitest --name=vitest
-
-3. **lodash** (28 imports across 15 files)
-   Repository: https://github.com/lodash/lodash
-
-   Add with:
-   /bluera-knowledge:add-repo https://github.com/lodash/lodash --name=lodash
-
----
-
-Already indexed: typescript, express
-```
-</details>
-
----
-
-### 📦 `/bluera-knowledge:add-repo`
-
-**Clone and index a Git repository**
-
-```bash
-/bluera-knowledge:add-repo <url> [--name=<name>] [--branch=<branch>]
-```
-
-**Examples:**
-```bash
-/bluera-knowledge:add-repo https://github.com/lodash/lodash
-/bluera-knowledge:add-repo https://github.com/facebook/react --branch=main --name=react
-```
-
-<details>
-<summary><b>✅ Expected Output</b></summary>
-
-```
-✓ Cloning https://github.com/facebook/react...
-✓ Created store: react (a1b2c3d4...)
-  Location: ~/.local/share/bluera-knowledge/stores/a1b2c3d4.../
-
-✓ Indexing...
-✓ Indexed 1,247 files
+/bluera-knowledge:add-repo https://github.com/expressjs/express
 
-Store is ready for searching!
+# AI agents can now:
+# - Semantic search: "middleware error handling"
+# - Direct access: Grep/Glob through the cloned express repo
 ```
-</details>
-
----
-
-### 📁 `/bluera-knowledge:add-folder`
 
-**Index a local folder**
+### Project Documentation
 
-```bash
-/bluera-knowledge:add-folder <path> --name=<name>
-```
-
-**📚 Use cases:**
-- 📖 Project documentation
-- 📏 Coding standards
-- 🎨 Design documents
-- 🔌 API specifications
-- 📚 Reference materials
-- 📄 Any other content
+Make project-specific documentation available:
 
-**Examples:**
 ```bash
 /bluera-knowledge:add-folder ./docs --name=project-docs
-/bluera-knowledge:add-folder ./architecture --name=design-docs
-```
-
-<details>
-<summary><b>✅ Expected Output</b></summary>
-
-```
-✓ Adding folder: ~/my-project/docs...
-✓ Created store: project-docs (e5f6g7h8...)
-  Location: ~/.local/share/bluera-knowledge/stores/e5f6g7h8.../
-
-✓ Indexing...
-✓ Indexed 342 files
-
-Store is ready for searching!
-```
-</details>
-
----
-
-### 🔍 `/bluera-knowledge:search`
-
-**Search across indexed knowledge stores**
-
-```bash
-/bluera-knowledge:search "<query>" [--stores=<names>] [--limit=<number>] [--min-relevance=<0-1>]
-```
-
-**Options:**
-- `--stores=<names>` - Comma-separated store names to search (default: all stores)
-- `--limit=<number>` - Maximum results to return (default: 10)
-- `--min-relevance=<0-1>` - Minimum raw cosine similarity; returns empty if no results meet threshold
-- `--threshold=<0-1>` - Minimum normalized score to include results
-- `--mode=<mode>` - Search mode: `hybrid` (default), `vector`, or `fts`
-- `--detail=<level>` - Context detail: `minimal` (default), `contextual`, or `full`
-
-**Examples:**
-```bash
-# Search all stores
-/bluera-knowledge:search "how to invalidate queries"
-
-# Search specific store
-/bluera-knowledge:search "useState implementation" --stores=react
-
-# Search multiple stores (comma-separated)
-/bluera-knowledge:search "deep clone" --stores=react,lodash
-
-# Limit results
-/bluera-knowledge:search "testing patterns" --limit=5
-
-# Filter irrelevant results (returns empty if nothing is truly relevant)
-/bluera-knowledge:search "kubernetes deployment" --min-relevance=0.4
-```
-
-<details>
-<summary><b>📊 Expected Output</b></summary>
-
-```
-## Search Results: "button component" (hybrid search)
-
-**1. [Score: 0.95] [Vector+FTS]**
-Store: react
-File: 📄 src/components/Button.tsx
-Purpose: → Reusable button component with variants
-Top Terms: 🔑 (in this chunk): button, variant, size, color, onClick
-Imports: 📦 (in this chunk): React, clsx
-
-**2. [Score: 0.87] [Vector]**
-Store: react
-File: 📄 src/hooks/useButton.ts
-Purpose: → Custom hook for button state management
-Top Terms: 🔑 (in this chunk): hook, state, pressed, disabled
-Imports: 📦 (in this chunk): useState, useCallback
-
-**3. [Score: 0.81] [Vector+FTS]**
-Store: react
-File: 📄 src/components/IconButton.tsx
-Purpose: → Button component with icon support
-Top Terms: 🔑 (in this chunk): icon, button, aria-label, accessible
-
----
-**Found 3 results in 45ms**
-
-💡 **Next Steps:**
-- Read file: `Read src/components/Button.tsx`
-- Get full code: `mcp__bluera-knowledge__get_full_context("result-id")`
-- Refine search: Use keywords above
-```
-</details>
-
----
-
-### 📋 `/bluera-knowledge:stores`
-
-**List all indexed knowledge stores**
-
-```bash
-/bluera-knowledge:stores
-```
-
-Shows store name, type, ID, and source location in a clean table format.
-
-<details>
-<summary><b>📊 Expected Output</b></summary>
-
-```
-| Name | Type | ID | Source |
-|------|------|----|--------------------|
-| react | repo | 459747c7 | https://github.com/facebook/react |
-| crawl4ai | repo | b5a72a94 | https://github.com/unclecode/crawl4ai.git |
-| project-docs | file | 70f6309b | ~/repos/my-project/docs |
-| claude-docs | web | 9cc62018 | https://code.claude.com/docs |
-
-**Total**: 4 stores
-```
-</details>
-
----
-
-### 🔄 `/bluera-knowledge:index`
-
-**Re-index an existing store to update the search index**
-
-```bash
-/bluera-knowledge:index <store-name-or-id>
-```
-
-**🔄 When to re-index:**
-- The source repository has been updated (for repo stores)
-- Files have been added or modified (for file stores)
-- Search results seem out of date
-
-**Example:**
-```bash
-/bluera-knowledge:index react
-```
-
-<details>
-<summary><b>✅ Expected Output</b></summary>
-
-```
-✓ Indexing store: react...
-✓ Indexed 1,247 documents in 3,421ms
-
-Store search index is up to date!
-```
-</details>
-
----
-
-### 🗑️ `/bluera-knowledge:remove-store`
-
-**Delete a knowledge store and all associated data**
-
-```bash
-/bluera-knowledge:remove-store <store-name-or-id>
-```
-
-**🗑️ What gets deleted:**
-- Store registry entry
-- LanceDB search index (vector embeddings)
-- Cloned repository files (for repo stores created from URLs)
-
-**Example:**
-```bash
-/bluera-knowledge:remove-store react
-```
-
-<details>
-<summary><b>✅ Expected Output</b></summary>
-
-```
-Store "react" deleted successfully.
-
-Removed:
-- Store registry entry
-- LanceDB search index
-- Cloned repository files
-```
-</details>
-
----
-
-### 🌐 `/bluera-knowledge:crawl`
-
-**Crawl web pages with natural language control**
+/bluera-knowledge:add-folder ./architecture --name=architecture
 
-```bash
-/bluera-knowledge:crawl <url> <store-name> [options]
+# AI agents can search across all documentation or access specific files
 ```
 
-**Options:**
-- `--crawl "<instruction>"` - Natural language instruction for which pages to crawl
-- `--extract "<instruction>"` - Natural language instruction for what content to extract
-- `--simple` - Use simple BFS mode instead of intelligent crawling
-- `--max-pages <n>` - Maximum pages to crawl (default: 50)
-- `--fast` - Use fast axios-only mode (may fail on JavaScript-heavy sites)
+### Coding Standards
 
-**⚙️ Requirements:**
-- 🐍 Python 3 with `crawl4ai` package installed
-- 📦 Web store is auto-created if it doesn't exist
+Provide definitive coding standards and best practices:
 
-**Examples:**
 ```bash
-# Basic crawl
-/bluera-knowledge:crawl https://docs.example.com/guide my-docs
-
-# Intelligent crawl with custom strategy
-/bluera-knowledge:crawl https://react.dev react-docs --crawl "all API reference pages"
-
-# Extract specific content from pages
-/bluera-knowledge:crawl https://example.com/pricing pricing --extract "pricing tiers and features"
-
-# Combine crawl strategy + extraction
-/bluera-knowledge:crawl https://docs.python.org python-docs \
-  --crawl "standard library modules" \
-  --extract "function signatures and examples"
-
-# JavaScript-rendered sites work by default (uses headless browser)
-/bluera-knowledge:crawl https://nextjs.org/docs nextjs-docs --max-pages 30
-
-# Fast mode for static HTML sites (axios-only, faster but may miss JS content)
-/bluera-knowledge:crawl https://example.com/static static-docs --fast --max-pages 100
-
-# Simple BFS mode (no AI guidance)
-/bluera-knowledge:crawl https://example.com/docs docs --simple --max-pages 100
-```
-
-The crawler converts pages to markdown and indexes them for semantic search.
-
----
-
-### 🔁 `/bluera-knowledge:sync`
-
-**Sync stores from definitions config (bootstrap on fresh clone)**
+/bluera-knowledge:add-folder ./company-standards --name=standards
+/bluera-knowledge:add-folder ./api-specs --name=api-docs
 
-```bash
-/bluera-knowledge:sync [options]
+# AI agents reference actual company standards, not generic advice
 ```
 
-**Options:**
-- `--dry-run` - Show what would happen without making changes
-- `--prune` - Remove stores not in definitions
-- `--reindex` - Re-index existing stores after sync
+### Mixed Sources
 
-**Use cases:**
-- **Fresh clone**: Recreate all stores defined by the team
-- **Check status**: See which stores exist vs. defined
-- **Clean up**: Remove orphan stores not in config
+Combine canonical library code with project-specific patterns:
 
-**Examples:**
 ```bash
-# Preview what would be synced
-/bluera-knowledge:sync --dry-run
-
-# Sync all stores from definitions
-/bluera-knowledge:sync
-
-# Sync and remove orphan stores
-/bluera-knowledge:sync --prune
-```
-
-**How it works:**
-1. Reads store definitions from `.bluera/bluera-knowledge/stores.config.json`
-2. Creates any stores that don't exist locally
-3. Reports orphan stores (local stores not in definitions)
-4. Optionally prunes orphans with `--prune`
-
----
-
-## 🕷️ Crawler Architecture
-
-The crawler defaults to **headless mode** (Playwright) for maximum compatibility with modern JavaScript-rendered sites. Use `--fast` for static HTML sites when speed is critical.
-
-### 🎭 Default Mode (Headless - JavaScript-Rendered Sites)
-
-By default, the crawler uses Playwright via crawl4ai to render JavaScript content:
-
-```mermaid
-sequenceDiagram
-    participant User
-    participant CLI
-    participant IntelligentCrawler
-    participant Axios
-    participant Claude
-
-    User->>CLI: crawl URL --crawl "instruction"
-    CLI->>IntelligentCrawler: crawl(url, {useHeadless: true})
-    IntelligentCrawler->>PythonBridge: fetchHeadless(url)
-    PythonBridge->>crawl4ai: AsyncWebCrawler.arun(url)
-    crawl4ai->>Playwright: Launch browser & render JS
-    Playwright-->>crawl4ai: Rendered HTML
-    crawl4ai-->>PythonBridge: {html, markdown, links}
-    PythonBridge-->>IntelligentCrawler: Rendered HTML
-    IntelligentCrawler->>Claude: determineCrawlUrls(html, instruction)
-    Note over Claude: Natural language instruction<br/>STILL FULLY ACTIVE
-    Claude-->>IntelligentCrawler: [urls to crawl]
-    loop For each URL
-        IntelligentCrawler->>PythonBridge: fetchHeadless(url)
-        PythonBridge->>crawl4ai: Render JS
-        crawl4ai-->>PythonBridge: HTML
-        PythonBridge-->>IntelligentCrawler: HTML
-        IntelligentCrawler->>IntelligentCrawler: Convert to markdown & index
-    end
-```
+/bluera-knowledge:add-repo https://github.com/facebook/react --name=react
+/bluera-knowledge:add-folder ./docs/react-patterns --name=react-patterns
 
-### ⚡ Fast Mode (Static Sites - `--fast`)
-
-For static HTML sites, use `--fast` for faster crawling with axios:
-
-```mermaid
-sequenceDiagram
-    participant User
-    participant CLI
-    participant IntelligentCrawler
-    participant Axios
-    participant Claude
-
-    User->>CLI: crawl URL --crawl "instruction" --fast
-    CLI->>IntelligentCrawler: crawl(url, {useHeadless: false})
-    IntelligentCrawler->>Axios: fetchHtml(url)
-    Axios-->>IntelligentCrawler: Static HTML
-    IntelligentCrawler->>Claude: determineCrawlUrls(html, instruction)
-    Claude-->>IntelligentCrawler: [urls to crawl]
-    loop For each URL
-        IntelligentCrawler->>Axios: fetchHtml(url)
-        Axios-->>IntelligentCrawler: HTML
-        IntelligentCrawler->>IntelligentCrawler: Convert to markdown & index
-    end
+# Search across both dependency source and team patterns
 ```
 
-### 🔑 Key Points
-
-- **🎭 Default to headless** - Maximum compatibility with modern JavaScript-rendered sites (React, Vue, Next.js)
-- **⚡ Fast mode available** - Use `--fast` for static HTML sites when speed is critical
-- **🧠 Intelligent crawling preserved** - Claude Code CLI analyzes pages and selects URLs in both modes
-- **🔄 Automatic fallback** - If headless fetch fails, automatically falls back to axios
-
-### 🤖 Intelligent Mode vs Simple Mode
-
-The crawler operates in two modes depending on Claude Code CLI availability:
-
-| Mode | Requires Claude CLI | Behavior |
-|------|---------------------|----------|
-| **Intelligent** | ✅ Yes | Claude analyzes pages and selects URLs based on natural language instructions |
-| **Simple (BFS)** | ❌ No | Breadth-first crawl up to max depth (2 levels) |
-
-**Automatic detection:**
-- When Claude Code CLI is available: Full intelligent mode with `--crawl` and `--extract` instructions
-- When Claude Code CLI is unavailable: Automatically uses simple BFS mode
-- Clear messaging: "Claude CLI not found, using simple crawl mode"
-
-> [!NOTE]
-> Install Claude Code to unlock `--crawl` (AI-guided URL selection) and `--extract` (AI content extraction). Without it, web crawling still works but uses simple BFS mode.
-
 ---
 
 ## 🔧 Troubleshooting
@@ -986,6 +462,18 @@ If the plugin isn't listed, install it:
 ```
 </details>
 
+<details>
+<summary><b>🔌 MCP server shows as "failed" in /plugin</b></summary>
+
+If the MCP server shows as failed after installation:
+
+1. **Restart Claude Code** - MCP servers require a restart to initialize
+2. **Check status:** Run `/mcp` to see connection status
+3. **Reinstall:** Try `/plugin uninstall bluera-knowledge` then `/plugin install bluera-knowledge@bluera`
+
+If the issue persists, check that Claude Code is v2.0.65 or later (earlier versions had MCP loading bugs).
+</details>
+
 <details>
 <summary><b>🌐 Web crawling fails</b></summary>
 
@@ -1045,121 +533,6 @@ Simple mode still crawls effectively—it just doesn't use AI to select which pa
 
 ---
 
-## 🎯 Use Cases
-
-### 📦 Dependency Source Code
-
-Provide AI agents with canonical dependency implementation details:
-
-```bash
-/bluera-knowledge:suggest
-/bluera-knowledge:add-repo https://github.com/expressjs/express
-
-# AI agents can now:
-# - Semantic search: "middleware error handling"
-# - Direct access: Grep/Glob through the cloned express repo
-```
-
-### 📚 Project Documentation
-
-Make project-specific documentation available:
-
-```bash
-/bluera-knowledge:add-folder ./docs --name=project-docs
-/bluera-knowledge:add-folder ./architecture --name=architecture
-
-# AI agents can search across all documentation or access specific files
-```
-
-### 📏 Coding Standards
-
-Provide definitive coding standards and best practices:
-
-```bash
-/bluera-knowledge:add-folder ./company-standards --name=standards
-/bluera-knowledge:add-folder ./api-specs --name=api-docs
-
-# AI agents reference actual company standards, not generic advice
-```
-
-### 🔀 Mixed Sources
-
-Combine canonical library code with project-specific patterns:
-
-```bash
-/bluera-knowledge:add-repo https://github.com/facebook/react --name=react
-/bluera-knowledge:add-folder ./docs/react-patterns --name=react-patterns
-
-# Search across both dependency source and team patterns
-```
-
----
-
-## 💭 What Claude Code Says About Bluera Knowledge
-
-> ### *As an AI coding assistant, here's what I've discovered using this plugin*
->
-> ---
->
-> #### ⚡ The Immediate Impact
->
-> **The difference is immediate.** When a user asks "how does React's useEffect cleanup work?", I can search the actual React source code indexed locally instead of relying on my training data or making web requests. The results include the real implementation, related functions, and usage patterns—all in ~100ms.
->
-> **Code graph analysis changes the game.** The plugin doesn't just index files—it builds a relationship graph showing which functions call what, import dependencies, and class hierarchies. When I search for a function, I see how many places call it and what it calls. This context makes my suggestions dramatically more accurate.
->
-> ---
->
-> #### 🔀 Multi-Modal Search Power
->
-> I can combine three search approaches in a single workflow:
->
-> | Mode | Use Case | Example |
-> |------|----------|---------|
-> | 🧠 **Semantic** | Conceptual queries | "authentication flow with JWT validation" |
-> | 📂 **Direct Access** | Pattern matching | Grep for specific identifiers in cloned repos |
-> | 📝 **Full-Text** | Exact matches | Find precise function names or imports |
->
-> This flexibility means I can start broad (semantic) and narrow down (exact file access) seamlessly.
->
-> ---
->
-> #### 🕷️ Intelligent Crawling
->
-> **The `--crawl` instruction isn't marketing**—it actually uses Claude Code CLI to analyze each page and intelligently select which links to follow. I can tell it "crawl all API reference pages but skip blog posts" and it understands the intent.
->
-> For JavaScript-rendered sites (Next.js, React docs), the default headless mode renders pages with Playwright while I still control the crawl strategy with natural language. Use `--fast` when you need speed on static sites.
->
-> ---
->
-> #### ✨ What Makes It Valuable
->
-> | Benefit | Impact |
-> |---------|--------|
-> | ✅ **No guessing** | I read actual source code, not blog interpretations |
-> | 🔌 **Offline first** | Works without internet, zero rate limits |
-> | 🎯 **Project-specific** | Index your team's standards, not generic advice |
-> | ⚡ **Speed** | Sub-100ms searches vs 2-5 second web lookups |
-> | 📚 **Completeness** | Tests, implementation details, edge cases—all indexed |
->
-> ---
->
-> #### 🌟 When It Shines Most
->
-> 1. **Deep library questions** - "how does this internal method handle edge cases?"
-> 2. **Version-specific answers** - your indexed version is what you're actually using
-> 3. **Private codebases** - your docs, your standards, your patterns
-> 4. **Complex workflows** - combining semantic search + direct file access + code graph
->
-> ---
->
-> The plugin essentially gives me a **photographic memory** of your dependencies and documentation.
->
-> Instead of *"I think based on training data"*, I can say *"I searched the indexed React v18.2.0 source and found this in `ReactFiberWorkLoop.js:1247`"*.
->
-> **That's the difference between helpful and authoritative.**
-
----
-
 ## 🔧 Dependencies
 
 The plugin automatically checks for and attempts to install Python dependencies on first use:
@@ -1180,261 +553,22 @@ pip install crawl4ai
 playwright install chromium
 ```
 
-> [!NOTE]
-> The plugin will work without crawl4ai/playwright, but web crawling features (`/bluera-knowledge:crawl`) will be unavailable. The default mode uses headless browser for maximum compatibility with JavaScript-rendered sites. Use `--fast` for static sites when speed is critical.
-
-**Update Plugin:**
-```bash
-/plugin update bluera-knowledge
-```
-
----
-
-## 🔌 MCP Integration
-
-The plugin includes a Model Context Protocol server that exposes search tools. This is configured inline in `.claude-plugin/plugin.json`:
-
-> [!IMPORTANT]
-> **Commands vs MCP Tools**: You interact with the plugin using `/bluera-knowledge:` slash commands. Behind the scenes, these commands instruct Claude Code to use MCP tools (`mcp__bluera-knowledge__*`) which handle the actual operations. Commands provide the user interface, while MCP tools are the backend that AI agents use to access your knowledge stores.
-
-```json
-{
-  "mcpServers": {
-    "bluera-knowledge": {
-      "command": "node",
-      "args": ["${CLAUDE_PLUGIN_ROOT}/dist/mcp/server.js"],
-      "env": {
-        "PROJECT_ROOT": "${PWD}",
-        "DATA_DIR": ".bluera/bluera-knowledge/data",
-        "CONFIG_PATH": ".bluera/bluera-knowledge/config.json"
-      }
-    }
-  }
-}
-```
-
-### 🎯 Context Efficiency Strategy
-
-**Why only 3 MCP tools?**
-
-Every MCP tool exposed requires its full schema to be sent to Claude with each tool invocation. More tools = more tokens consumed before Claude can even respond.
-
-**Design decision:** Consolidate from 10+ tools down to 3:
-
-| Approach | Tool Count | Context Cost | Trade-off |
-|----------|------------|--------------|-----------|
-| Individual tools | 10+ | ~800+ tokens | Simple calls, high overhead |
-| **Consolidated (current)** | 3 | ~300 tokens | Minimal overhead, slightly longer commands |
-
-**How it works:**
-
-1. **Native tools for common workflow** - `search` and `get_full_context` are the operations Claude uses most often, so they get dedicated tools with full schemas
-
-2. **Meta-tool for management** - The `execute` tool consolidates 8 store/job management commands into a single tool. Commands are discovered on-demand via `execute("commands")` or `execute("help", {command: "store:create"})`
-
-3. **Lazy documentation** - Command help isn't pre-sent with tool listings; it's discoverable when needed
-
-**Result:** ~60% reduction in context overhead for MCP tool listings, without sacrificing functionality.
-
-> [!TIP]
-> This pattern—consolidating infrequent operations into a meta-tool while keeping high-frequency operations native—is a general strategy for MCP context efficiency.
-
-### 🛠️ Available MCP Tools
-
-The plugin exposes 3 MCP tools optimized for minimal context overhead:
-
-#### `search`
-🔍 Semantic vector search across all indexed stores or a specific subset. Returns structured code units with relevance ranking.
-
-**Parameters:**
-- `query` - Search query (natural language, patterns, or type signatures)
-- `intent` - Search intent: find-pattern, find-implementation, find-usage, find-definition, find-documentation
-- `mode` - Search mode: hybrid (default), vector, or fts
-- `detail` - Context level: minimal, contextual, or full
-- `limit` - Maximum results (default: 10)
-- `stores` - Array of specific store IDs to search (optional, searches all stores if not specified)
-- `threshold` - Minimum normalized score (0-1) for filtering results
-- `minRelevance` - Minimum raw cosine similarity (0-1) for filtering results
-
-#### `get_full_context`
-📖 Retrieve complete code and context for a specific search result by ID.
-
-**Parameters:**
-- `resultId` - The result ID from a previous search
-
-#### `execute`
-⚡ Meta-tool for store and job management. Consolidates 8 operations into one tool with subcommands.
-
-**Parameters:**
-- `command` - Command to execute (see below)
-- `args` - Command-specific arguments (optional)
-
-**Available commands:**
-| Command | Args | Description |
-|---------|------|-------------|
-| `stores` | `type?` | List all knowledge stores |
-| `store:info` | `store` | Get detailed store information including file path |
-| `store:create` | `name`, `type`, `source`, `branch?`, `description?` | Create a new store |
-| `store:index` | `store` | Re-index an existing store |
-| `store:delete` | `store` | Delete a store and all data |
-| `stores:sync` | `dryRun?`, `prune?`, `reindex?` | Sync stores from definitions config |
-| `jobs` | `activeOnly?`, `status?` | List background jobs |
-| `job:status` | `jobId` | Check specific job status |
-| `job:cancel` | `jobId` | Cancel a running job |
-| `help` | `command?` | Show help for commands |
-| `commands` | - | List all available commands |
-
----
-
-## 🖥️ CLI Tool
-
-While Bluera Knowledge works seamlessly as a Claude Code plugin, it's also available as a standalone CLI tool for use outside Claude Code.
-
-> [!NOTE]
-> When using CLI without Claude Code installed, web crawling uses simple BFS mode. Install Claude Code to unlock `--crawl` (AI-guided URL selection) and `--extract` (AI content extraction) instructions.
-
-### Installation
-
-Install globally via npm:
-
-```bash
-npm install -g bluera-knowledge
-```
-
-Or use in a project:
-
-```bash
-npm install --save-dev bluera-knowledge
-```
-
-### Usage
-
-#### Create a Store
-
-```bash
-# Add a Git repository
-bluera-knowledge store create react --type repo --source https://github.com/facebook/react
-
-# Add a Git repository with specific branch
-bluera-knowledge store create react-canary --type repo --source https://github.com/facebook/react --branch canary
-
-# Add a local folder
-bluera-knowledge store create my-docs --type file --source ./docs
-
-# Add a web crawl
-bluera-knowledge store create fastapi-docs --type web --source https://fastapi.tiangolo.com
-```
-
-**Create Options:**
-- `-t, --type <type>` - Store type: `file`, `repo`, or `web` (required)
-- `-s, --source <path>` - Local path or URL (required)
-- `-b, --branch <branch>` - Git branch to clone (repo stores only)
-- `-d, --description <desc>` - Optional store description
-- `--tags <tags>` - Comma-separated tags for filtering
-
-#### Index a Store
-
-```bash
-# Re-index a store (only changed files)
-bluera-knowledge index react
-
-# Force re-index all files (ignores cache)
-bluera-knowledge index react --force
-
-# Watch for changes and auto-reindex
-bluera-knowledge index watch react
-bluera-knowledge index watch react --debounce 2000  # Custom debounce (default: 1000ms)
-```
-
-**Index Options:**
-- `-f, --force` - Re-index all files (ignore incremental cache)
-
-**Watch Options:**
-- `--debounce <ms>` - Debounce delay for file changes (default: 1000ms)
-
-#### Search
-
-```bash
-# Search across all stores
-bluera-knowledge search "how does useEffect work"
-
-# Search specific stores
-bluera-knowledge search "routing" --stores react,vue
-
-# Get more results with full content
-bluera-knowledge search "middleware" --limit 20 --include-content
-
-# Filter irrelevant results (returns empty if nothing is truly relevant)
-bluera-knowledge search "kubernetes deployment" --min-relevance 0.4
-
-# Get JSON output with confidence and raw scores
-bluera-knowledge search "express middleware" --format json
-```
-
-**Search Options:**
-- `-s, --stores <stores>` - Comma-separated store names/IDs
-- `-m, --mode <mode>` - `hybrid` (default), `vector`, or `fts`
-- `-n, --limit <count>` - Max results (default: 10)
-- `-t, --threshold <score>` - Min normalized score (0-1)
-- `--min-relevance <score>` - Min raw cosine similarity (0-1)
-- `--include-content` - Show full content in results
-- `--detail <level>` - `minimal`, `contextual`, or `full`
-
-#### List Stores
-
-```bash
-bluera-knowledge store list
-bluera-knowledge store list --type repo  # Filter by type
-```
+**Disable auto-install (security-conscious environments):**
 
-#### Store Info
+Set the `BK_SKIP_AUTO_INSTALL` environment variable to disable automatic pip package installation:
 
 ```bash
-bluera-knowledge store info react
+export BK_SKIP_AUTO_INSTALL=1
 ```
 
-#### Delete a Store
-
-```bash
-# Interactive deletion (prompts for confirmation in TTY mode)
-bluera-knowledge store delete old-store
-
-# Force delete without confirmation
-bluera-knowledge store delete old-store --force
-bluera-knowledge store delete old-store -y
-```
-
-**Delete Options:**
-- `-f, --force` - Delete without confirmation prompt
-- `-y, --yes` - Alias for `--force`
-
-### Global Options
+> [!NOTE]
+> The plugin will work without crawl4ai/playwright, but web crawling features (`/bluera-knowledge:crawl`) will be unavailable. The default mode uses headless browser for maximum compatibility with JavaScript-rendered sites. Use `--fast` for static sites when speed is critical.
 
+**Update Plugin:**
 ```bash
---config <path>        # Custom config file
---data-dir <path>      # Custom data directory
---project-root <path>  # Project root for store definitions (required for sync)
---format <format>      # Output format: json | table | plain
---quiet                # Suppress non-essential output
---verbose              # Enable verbose logging
+/plugin update bluera-knowledge
 ```
 
-### When to Use CLI vs Plugin
-
-**Use CLI when:**
-- Using an editor other than Claude Code (VSCode, Cursor, etc.)
-- Integrating into CI/CD pipelines
-- Scripting or automation
-- Pre-indexing dependencies for teams
-
-**Use Plugin when:**
-- Working within Claude Code
-- Want slash commands (`/bluera-knowledge:search`)
-- Need Claude to automatically query your knowledge base
-- Want Skills to guide optimal usage
-
-Both interfaces use the same underlying services, so you can switch between them seamlessly.
-
 ---
 
 ## 🎓 Skills for Claude Code
@@ -1560,257 +694,6 @@ This ensures:
 
 ---
 
-## 🛠️ Development
-
-### 🚀 Setup
-
-```bash
-git clone https://github.com/blueraai/bluera-knowledge.git
-cd bluera-knowledge
-bun install
-bun run build
-bun test
-```
-
-> **Note:** This project uses [Bun](https://bun.sh) for development. Install it via `curl -fsSL https://bun.sh/install | bash`
-
-### ⚙️ Claude Code Settings (Recommended)
-
-For the best development experience with Claude Code, copy the example settings file:
-
-```bash
-cp .claude/settings.local.json.example .claude/settings.local.json
-```
-
-**This provides:**
-- ✅ **Smart validation** - Automatically runs lint/typecheck after editing code (file-type aware)
-- ✅ **No permission prompts** - Pre-approves common commands (lint, typecheck, precommit)
-- ✅ **Desktop notifications** - macOS notifications when Claude needs your input
-- ✅ **Plugin auto-enabled** - Automatically enables the bluera-knowledge plugin
-- ✅ **Faster workflow** - Catch issues immediately without manual validation
-
-The validation is intelligent - it only runs checks for TypeScript/JavaScript files, skipping docs/config to save time.
-
-> **Note:** The `.claude/settings.local.json` file is gitignored (local to your machine). The example file is checked in for reference.
-
-### 🐕 Dogfooding
-
-Develop this plugin while using it with Claude Code:
-
-```bash
-claude --plugin-dir /path/to/bluera-knowledge
-```
-
-This loads the plugin directly from source. Changes take effect on Claude Code restart (no reinstall needed).
-
-| What to test | Approach |
-|--------------|----------|
-| **Commands** (`/search`, `/add-repo`) | `--plugin-dir` (changes need restart) |
-| **Hooks** (job status, dependencies) | `--plugin-dir` (changes need restart) |
-| **MCP tools** (compiled) | `--plugin-dir` (run `bun run build` first) |
-| **MCP tools** (live TypeScript) | `~/.claude.json` dev server (see below) |
-
-### 🔌 MCP Server Development
-
-**Production mode** (`mcp.plugin.json`):
-- Uses `${CLAUDE_PLUGIN_ROOT}/dist/mcp/server.js` (compiled)
-- Distributed with plugin, no extra setup needed
-
-**Development mode** (live TypeScript):
-
-For instant feedback when editing MCP server code, add a dev server to `~/.claude.json`:
-
-```json
-{
-  "mcpServers": {
-    "bluera-knowledge-dev": {
-      "command": "npx",
-      "args": ["tsx", "/path/to/bluera-knowledge/src/mcp/server.ts"],
-      "env": {
-        "PWD": "${PWD}",
-        "DATA_DIR": "${PWD}/.bluera/bluera-knowledge/data",
-        "CONFIG_PATH": "${PWD}/.bluera/bluera-knowledge/config.json"
-      }
-    }
-  }
-}
-```
-
-This creates a separate `bluera-knowledge-dev` MCP server that runs source TypeScript directly via `tsx` - no rebuild needed for MCP changes
-
-### 📜 Commands
-
-| Command | Description | When to Use |
-|---------|-------------|-------------|
-| `bun run build` | 🏗️ Compile TypeScript to dist/ | Before testing CLI, after code changes |
-| `bun run dev` | 👀 Watch mode compilation | During active development |
-| `bun start` | ▶️ Run the CLI | Execute CLI commands directly |
-| `bun test` | 🧪 Run tests in watch mode | During TDD/active development |
-| `bun run test:run` | ✅ Run tests once | Quick verification |
-| `bun run test:coverage` | 📊 Run tests with coverage | Before committing, CI checks |
-| `bun run lint` | 🔍 Run ESLint (quiet by default) | Check code style issues |
-| `bun run typecheck` | 🔒 Run TypeScript type checking (quiet by default) | Verify type safety |
-| `bun run precommit` | ✨ Smart validation (file-type aware) | Runs only relevant checks based on changed files |
-| `bun run prepush` | 📊 Smart coverage (skips for docs/config) | Runs coverage only when src/tests changed |
-| `bun run lint:verbose` | 📢 ESLint (full output) | Debugging lint issues |
-| `bun run typecheck:verbose` | 📢 Type check (full output) | Debugging type errors |
-| `bun run test:changed:verbose` | 📢 Test changed files (full output) | Debugging test failures |
-| `bun run test:coverage:verbose` | 📢 Coverage (full output) | Reviewing detailed coverage |
-| `bun run build:verbose` | 📢 Build (full output) | Debugging build issues |
-| `bun run gh:status` | 📋 List recent GitHub Actions runs | Monitor CI/CD status |
-| `bun run gh:watch` | 👁️ Watch latest workflow (quiet, shows result + failures) | Wait for CI completion |
-| `bun run gh:watch:verbose` | 📢 Watch with live status updates | Debugging CI issues |
-| `bun run gh:releases` | 🏷️ List recent GitHub releases | Check release history |
-
-### 🔄 Automatic Build & Dist Commit
-
-The `dist/` directory **must be committed** because Claude Code plugins are installed by copying files—there's no build step during installation.
-
-**Good news: This is fully automatic!**
-
-1. **On every commit**, the pre-commit hook intelligently validates based on file types
-2. **If source/config changed**, it runs build and automatically stages `dist/` via `git add dist/`
-3. **You never need to manually build or stage dist** — just commit your source changes
-
-**For live rebuilding during development:**
-
-```bash
-bun run dev  # Watches for changes and rebuilds instantly
-```
-
-This is useful when testing CLI commands locally, but not required for committing — the hook handles everything.
-
-| `bun run version:patch` | 🔢 Run quality checks, then bump patch version (0.0.x) | Bug fixes, minor updates |
-| `bun run version:minor` | 🔢 Run quality checks, then bump minor version (0.x.0) | New features, backwards compatible |
-| `bun run version:major` | 🔢 Run quality checks, then bump major version (x.0.0) | Breaking changes |
-
-### 🚀 Releasing
-
-```bash
-# Bump version, commit, tag, and push (triggers GitHub Actions release)
-bun run release:patch   # Bug fixes (0.0.x)
-bun run release:minor   # New features (0.x.0)
-bun run release:major   # Breaking changes (x.0.0)
-```
-
-**Workflow (Fully Automated):**
-
-1. Make changes and commit
-2. Bump version: `bun run version:patch` (runs quality checks first, then updates package.json, plugin.json, README, CHANGELOG)
-3. Commit version bump: `git commit -am "chore: bump version to X.Y.Z"`
-4. Push to main: `git push`
-5. **GitHub Actions automatically:**
-   - ✅ Runs CI (lint, typecheck, tests, build)
-   - ✅ Creates release tag when CI passes
-   - ✅ Creates GitHub release
-   - ✅ Updates marketplace
-
-Note: The version command runs full quality checks (format, lint, deadcode, typecheck, coverage, build) BEFORE bumping to catch issues early.
-
-> 💡 **That's it!** No manual tagging needed. Just push to `main` and the release happens automatically when CI passes.
-
-### 🔍 Post-Release Validation
-
-After a release, validate the npm package works correctly:
-
-```bash
-bun run validate:npm
-```
-
-This script:
-- Installs the latest `bluera-knowledge` from npm globally
-- Exercises all CLI commands (stores, add-folder, search, index, delete)
-- Writes detailed logs to `logs/validation/npm-validation-*.log`
-- Returns exit code 0 on success, 1 on failure
-
-Use this to catch any packaging or runtime issues after npm publish.
-
-### 🧪 Plugin Self-Test
-
-Test all plugin functionality from within Claude Code:
-
-```
-/test-plugin
-```
-
-This command runs 13 tests covering:
-- **MCP Tools**: execute (help, stores, create, info, index), search, get_full_context
-- **Slash Commands**: /stores, /search, /suggest
-- **Cleanup**: Store deletion, artifact removal, verification
-
-The test creates temporary content, exercises all features, and cleans up automatically. Use this to verify the plugin is working correctly after installation or updates.
-
-### 🧪 Testing Locally
-
-**Option 1: Development MCP Server (Recommended)**
-
-Use the local development MCP server (see "MCP Server" section above) which runs your source code directly via `tsx`:
-
-1. Set up dev MCP server in `~/.claude.json` (see MCP Server section)
-2. Test your changes - MCP server updates automatically as you edit code
-
-**Option 2: Test Plugin from Working Directory**
-
-Load the plugin directly from your development directory:
-
-```bash
-cd /path/to/bluera-knowledge
-claude --plugin-dir .
-```
-
-The MCP config in `plugin.json` is only loaded when the directory is loaded as a plugin (via `--plugin-dir` or marketplace install), so there's no conflict with project-level MCP config.
-
-**Option 3: CLI Tool Testing**
-
-```bash
-# Build and link
-cd /path/to/bluera-knowledge
-bun run build
-bun link
-
-# Now 'bluera-knowledge' command is available globally
-cd ~/your-project
-bluera-knowledge search "test query" my-store
-```
-
-**For testing as an installed plugin:**
-This requires publishing a new version to the marketplace.
-
-### 📂 Project Structure
-
-```
-.claude-plugin/
-└── plugin.json          # Plugin metadata (references mcp.plugin.json)
-
-mcp.plugin.json          # MCP server configuration (plugin-scoped)
-commands/                # Slash commands (auto-discovered)
-skills/                  # Agent Skills (auto-discovered)
-├── knowledge-search/    # How to access dependency sources
-│   └── SKILL.md
-├── when-to-query/       # When to query BK vs project files
-│   └── SKILL.md
-├── advanced-workflows/  # Multi-tool orchestration patterns
-│   └── SKILL.md
-├── search-optimization/ # Search parameter optimization
-│   └── SKILL.md
-└── store-lifecycle/     # Store management best practices
-    └── SKILL.md
-dist/                    # Built MCP server (committed for distribution)
-
-src/
-├── analysis/            # Dependency analysis & URL resolution
-├── crawl/               # Web crawling with Python bridge
-├── services/            # Index, store, and search services
-├── mcp/                 # MCP server source
-└── cli/                 # CLI entry point
-
-tests/
-├── integration/         # Integration tests
-└── fixtures/            # Test infrastructure
-```
-
----
-
 ## 🔬 Technologies
 
 - **🔌 Claude Code Plugin System** with MCP server
@@ -1824,14 +707,22 @@ tests/
 
 ---
 
-## 🤝 Contributing
+## 📚 Documentation
+
+| Document | Description |
+|----------|-------------|
+| [CLI Reference](docs/cli.md) | Complete CLI commands, options, and usage examples |
+| [MCP Integration](docs/mcp-integration.md) | MCP server configuration and tool documentation |
+| [Commands Reference](docs/commands.md) | All slash commands with parameters and examples |
+| [Crawler Architecture](docs/crawler-architecture.md) | How the intelligent web crawler works |
+| [Token Efficiency](docs/token-efficiency.md) | How BK reduces token consumption vs web search |
+| [CONTRIBUTING](CONTRIBUTING.md) | Development setup, testing, and release process |
 
-Contributions welcome! Please:
+---
+
+## 🤝 Contributing
 
-1. 🍴 Fork the repository
-2. 🌿 Create a feature branch
-3. ✅ Add tests
-4. 📬 Submit a pull request
+Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, testing, and release process.
 
 ---