bluera-knowledge 0.31.0 → 0.33.0

package/skills/store-lifecycle/SKILL.md

@@ -0,0 +1,470 @@
+---
+name: store-lifecycle
+description: Create, index, and manage BK stores
+---
+
+# Bluera Knowledge Store Lifecycle Management
+
+Master the lifecycle of knowledge stores from creation to deletion, including best practices for naming, indexing, and maintenance.
+
+## Choosing the Right Source Type
+
+Bluera Knowledge supports three source types. Choose based on your content:
+
+### Git Repositories (`add-repo` / `create_store` with git URL)
+
+**✅ Use for:**
+- Public library source code (React, Vue, Pydantic, etc.)
+- Private repositories with auth
+- Code you want to track and update
+- Multi-file projects with git history
+
+**Advantages:**
+- Preserves git history
+- Can pull updates (`git pull` in repo directory)
+- Standard structure recognized by analyzers
+- Automatic language detection
+
+**Example:**
+```
+/bluera-knowledge:add-repo https://github.com/vuejs/core --name=vue
+
+# Or via MCP:
+create_store(
+  source='https://github.com/vuejs/core',
+  name='vue',
+  type='repo'
+)
+```
+
+**Best practices:**
+- Use package/library name for consistency: `vue`, `fastapi`, `pydantic`
+- For monorepos: `org-project` format: `microsoft-typescript`, `vercel-next`
+- Include version if tracking specific release: `vue-3.4`, `python-3.11`
+
+### Local Folders (`add-folder`)
+
+**✅ Use for:**
+- Private codebases not in git
+- Work-in-progress code
+- Local documentation
+- Specific subdirectories of larger projects
+
+**Advantages:**
+- No git required
+- Fast indexing (no clone step)
+- Perfect for proprietary code
+- Can index subset of larger repo
+
+**Example:**
+```
+/bluera-knowledge:add-folder /path/to/my-project/api --name=my-api
+
+# Or via MCP:
+create_store(
+  source='/Users/me/projects/my-app/backend',
+  name='my-backend',
+  type='folder'
+)
+```
+
+**Best practices:**
+- Use descriptive names: `my-api`, `auth-service`, `shared-utils`
+- Index focused directories (not entire ~/ )
+- Update by re-indexing: `/bluera-knowledge:index my-api`
+
+### Web Documentation (`crawl`)
+
+**✅ Use for:**
+- Official documentation sites
+- API references hosted online
+- Tutorials and guides
+- Content only available via web
+
+**Advantages:**
+- Access web-only content
+- Handles JavaScript-rendered sites (headless mode)
+- Follows links automatically
+- Converts HTML to searchable text
+
+**Example:**
+```
+/bluera-knowledge:crawl https://fastapi.tiangolo.com --name=fastapi-docs --max-pages=100
+
+# Or via MCP:
+create_store(
+  source='https://fastapi.tiangolo.com',
+  name='fastapi-docs',
+  type='web',
+  max_pages=100
+)
+```
+
+**Best practices:**
+- Append `-docs` to library name: `fastapi-docs`, `vue-docs`
+- Set `max-pages` to avoid crawling entire internet
+- Use `--headless` for JavaScript-heavy sites
+- Crawl specific documentation paths, not marketing pages
+
+## Naming Conventions
+
+Good names make stores easy to find and filter.
+
+### Recommended Patterns
+
+**Library source code:**
+```
+vue          # Official package name
+react
+fastapi
+pydantic
+```
+
+**Documentation sites:**
+```
+vue-docs
+fastapi-docs
+python-3.11-docs
+```
+
+**Organization/project format:**
+```
+microsoft-typescript
+vercel-next
+acme-payment-api      # Your company's code
+```
+
+**Versioned stores:**
+```
+vue-3.4
+python-3.11
+react-18
+```
+
+**Specialized content:**
+```
+coding-standards      # Company standards
+api-spec-v2          # API specification
+architecture-docs    # Design docs
+```
+
+### Naming Anti-Patterns
+
+❌ Avoid:
+- Generic names: `docs`, `code`, `library`
+- Unclear abbreviations: `fp`, `lib1`, `proj`
+- Dates without context: `2024-01-15`
+- Redundant words: `my-project-library-code`
+
+✅ Prefer:
+- Specific, descriptive: `fastapi-docs`, `vue-source`
+- Standard package names: `pydantic`, `lodash`
+- Clear context: `api-spec-v2`, `coding-standards`
+
+## Indexing Strategies
+
+### Initial Indexing
+
+When creating a store, indexing happens automatically in the background:
+
+```
+create_store(url, name)
+→ Returns: job_id
+→ Background: clone/download → analyze → index
+→ Status: pending → running → completed
+
+# Monitor progress
+check_job_status(job_id)
+→ Progress: 45% (processing src/core.ts)
+→ Estimated: ~2 minutes remaining
+```
+
+**Indexing time estimates:**
+- Small library (<1k files): 30-60 seconds
+- Medium library (1k-5k files): 1-3 minutes
+- Large library (>5k files): 3-10 minutes
+- Documentation crawl (100 pages): 1-2 minutes
+
+### Re-indexing (Updates)
+
+When library code changes or you modify indexed content:
+
+```
+# For git repos: pull latest changes
+cd .bluera/bluera-knowledge/repos/vue
+git pull origin main
+cd -
+
+# Re-index
+/bluera-knowledge:index vue
+
+# Or via MCP:
+index_store(store='vue')
+→ Re-processes all files
+→ Updates vector embeddings
+→ Rebuilds search index
+```
+
+**When to re-index:**
+- Library released new version
+- You modified local folder content
+- Search results seem outdated
+- After significant codebase changes
+
+**Re-indexing is incremental** - only changed files are re-processed.
+
+### Selective Indexing
+
+For large repos, you might want to index specific directories:
+
+```
+# Clone full repo manually
+git clone https://github.com/microsoft/vscode
+cd vscode
+
+# Index only specific dirs
+/bluera-knowledge:add-folder ./src/vs/editor --name=vscode-editor
+/bluera-knowledge:add-folder ./src/vs/workbench --name=vscode-workbench
+
+# Result: Multiple focused stores instead of one massive store
+```
+
+## Storage Management
+
+### Monitoring Storage
+
+Check what's using space:
+
+```
+list_stores()
+→ vue: 487 files, 2.3 MB
+→ react: 312 files, 1.8 MB
+→ fastapi-docs: 156 pages, 0.9 MB
+→ my-api: 89 files, 0.4 MB
+
+Total storage: ~5.4 MB
+
+# Detailed info
+get_store_info('vue')
+→ Location: .bluera/bluera-knowledge/repos/vue/
+→ Indexed: 487 files
+→ Size: 2.3 MB (source) + 4.1 MB (vectors)
+→ Last indexed: 2 hours ago
+```
+
+### When to Delete Stores
+
+**✅ Delete when:**
+- Library no longer relevant to your project
+- Documentation outdated (re-crawl instead)
+- Testing/experimental stores no longer needed
+- Running low on disk space
+- Duplicate stores exist
+
+**How to delete:**
+```
+/bluera-knowledge:remove-store old-library
+
+# Or via MCP:
+delete_store(store='old-library')
+→ Removes: source files, vector index, metadata
+→ Frees: ~6-8 MB per store (varies by size)
+```
+
+**⚠️ Cannot undo!** Make sure you don't need the store before deleting.
+
+## Background Job Monitoring
+
+All expensive operations run as background jobs: cloning, indexing, crawling.
+
+### Job Lifecycle
+
+```
+1. create_store() or index_store() → Returns job_id
+
+2. Job states:
+   - pending: In queue, not started
+   - running: Actively processing
+   - completed: Finished successfully
+   - failed: Error occurred
+
+3. Monitor progress:
+   check_job_status(job_id)
+   → Current state, percentage, current file
+
+4. List all jobs:
+   list_jobs()
+   → See pending, running, completed jobs
+
+5. Cancel if needed:
+   cancel_job(job_id)
+   → Stops running job, cleans up
+```
+
+### Best Practices for Job Monitoring
+
+**Do poll, but not too frequently:**
+```
+# ❌ Too frequent - wastes resources
+while status != 'completed':
+    check_job_status(job_id)  # Every second!
+    sleep(1)
+
+# ✅ Reasonable polling interval
+while status != 'completed':
+    check_job_status(job_id)
+    sleep(15)  # Every 15 seconds is fine
+```
+
+**Do handle failures gracefully:**
+```
+status = check_job_status(job_id)
+
+if status['state'] == 'failed':
+    error = status['error']
+
+    if 'auth' in error.lower():
+        print("Authentication required - try SSH URL or provide credentials")
+    elif 'not found' in error.lower():
+        print("Repository/URL not found - check the source")
+    elif 'disk' in error.lower():
+        print("Disk space issue - delete unused stores")
+    else:
+        print(f"Unexpected error: {error}")
+```
+
+**Do list jobs to avoid duplicates:**
+```
+# Before creating new store
+jobs = list_jobs()
+existing = [j for j in jobs if j['store'] == 'vue' and j['state'] in ['pending', 'running']]
+
+if existing:
+    print(f"Job already running for 'vue': {existing[0]['id']}")
+    # Wait for it instead of creating duplicate
+else:
+    create_store(...)
+```
+
+## Handling Indexing Failures
+
+### Common Failure Scenarios
+
+**1. Authentication Required (Private Repos)**
+```
+Error: "Authentication required"
+
+Fix options:
+  - Use SSH URL: git@github.com:org/repo.git
+  - Use HTTPS with token: https://token@github.com/org/repo.git
+  - Make repo public (if appropriate)
+```
+
+**2. Invalid URL/Path**
+```
+Error: "Repository not found" or "Path does not exist"
+
+Fix:
+  - Verify URL is correct (typos common!)
+  - Check path exists and is accessible
+  - Ensure network connectivity
+```
+
+**3. Disk Space**
+```
+Error: "No space left on device"
+
+Fix:
+  - Check available space: df -h
+  - Delete unused stores: delete_store(old_store)
+  - Clear .bluera/bluera-knowledge/repos/ manually if needed
+```
+
+**4. Network Timeout**
+```
+Error: "Connection timeout" or "Failed to fetch"
+
+Fix:
+  - Retry after checking network
+  - Use --shallow for large repos
+  - Clone manually then add-folder
+```
+
+**5. Unsupported File Types**
+```
+Warning: "Skipped 45 binary files"
+
+This is normal!
+  - Binary files (images, compiled code) are skipped
+  - Only text files are indexed
+  - Check indexed count vs total to see ratio
+```
+
+### Recovery Workflow
+
+```
+1. Attempt fails:
+   create_store(url, name) → job fails
+
+2. Check error:
+   job_status = check_job_status(job_id)
+   error_msg = job_status['error']
+
+3. Determine fix based on error type (see above)
+
+4. Retry with fix:
+   create_store(corrected_url, name)
+
+5. Verify success:
+   check_job_status(new_job_id)
+   → Status: completed
+
+   list_stores()
+   → Store appears in list
+
+6. Test search:
+   search(test_query, stores=[name], limit=3)
+   → Returns results: ✅ Ready to use!
+```
+
+## Store Lifecycle Checklist
+
+**Creating a Store:**
+- [ ] Choose appropriate source type (repo/folder/crawl)
+- [ ] Use descriptive, consistent naming
+- [ ] Start indexing job
+- [ ] Monitor job status until complete
+- [ ] Verify with list_stores()
+- [ ] Test with sample search
+
+**Maintaining a Store:**
+- [ ] Re-index after significant changes
+- [ ] Pull git updates periodically for repo stores
+- [ ] Monitor storage usage
+- [ ] Check search relevance quality
+
+**Deleting a Store:**
+- [ ] Confirm no longer needed
+- [ ] Note storage freed
+- [ ] Remove from any documentation referencing it
+
+## Quick Reference Commands
+
+```
+# Create
+/bluera-knowledge:add-repo <url> --name=<name>
+/bluera-knowledge:add-folder <path> --name=<name>
+/bluera-knowledge:crawl <url> --name=<name>
+
+# Monitor
+/bluera-knowledge:check-status <job-id>
+
+# Maintain
+/bluera-knowledge:index <name>
+/bluera-knowledge:stores
+
+# Remove
+/bluera-knowledge:remove-store <name>
+```
+
+Master these lifecycle management practices to maintain a clean, efficient, and useful knowledge base.

package/skills/stores/SKILL.md

@@ -0,0 +1,54 @@
+---
+description: List all indexed library stores
+allowed-tools: ["mcp__bluera-knowledge__execute"]
+---
+
+# List Knowledge Stores
+
+Show all configured knowledge stores in the project.
+
+## Steps
+
+1. Use the mcp__bluera-knowledge__execute tool with command "stores" to retrieve all stores
+
+2. Present results in a clean table format:
+
+```
+| Name | Type | ID | Source |
+|------|------|----|--------------------|
+| react | repo | a1b2c3d4 | https://github.com/facebook/react |
+| lodash | repo | e5f6g7h8 | https://github.com/lodash/lodash |
+| my-docs | file | i9j0k1l2 | ~/docs |
+
+**Total**: 3 stores
+```
+
+3. Format each row:
+   - **Name**: The store name
+   - **Type**: Store type (repo, file, or web)
+   - **ID**: First 8 characters of the store ID (no ellipsis)
+   - **Source**:
+     - For repo stores: The git URL
+     - For file stores: The local path (use ~ for home directory to keep it concise)
+     - For web stores: The base URL
+
+## If No Stores Found
+
+If no stores exist, show:
+
+```
+## No Knowledge Stores Found
+
+You haven't created any knowledge stores yet.
+
+To get started:
+- `/bluera-knowledge:add-repo <url> --name=<name>` - Clone and index a library repository
+- `/bluera-knowledge:add-folder <path> --name=<name>` - Index a local folder of documentation
+
+Example:
+```
+/bluera-knowledge:add-repo https://github.com/facebook/react --name=react
+```
+
+After creating stores, they will be searchable via the MCP search tool.
+```

package/skills/suggest/SKILL.md

@@ -0,0 +1,118 @@
+---
+description: Suggest important dependencies to add to knowledge stores
+allowed-tools: ["Glob", "Read", "mcp__bluera-knowledge__execute", "WebSearch", "AskUserQuestion", "Skill"]
+---
+
+# Suggest Dependencies to Index
+
+Analyze project dependencies and suggest important libraries to add to knowledge stores.
+
+## Steps
+
+1. **Find dependency files** using Glob tool:
+   - `**/package.json` (JavaScript/TypeScript projects)
+   - `**/requirements.txt` (Python projects)
+   - `**/go.mod` (Go projects)
+   - `**/Cargo.toml` (Rust projects)
+
+2. **Read and parse dependencies**:
+   - Use Read tool to read each dependency file
+   - Extract package names and usage patterns
+   - For package.json: dependencies and devDependencies
+   - For requirements.txt: all packages
+   - For go.mod: require statements
+   - For Cargo.toml: dependencies section
+
+3. **Scan for import statements** using Glob + Read:
+   - Find all source files (*.js, *.ts, *.py, *.go, *.rs)
+   - Count imports/requires for each dependency
+   - Rank dependencies by frequency of use
+
+4. **Get existing stores** using mcp__bluera-knowledge__execute with command "stores":
+   - Filter out dependencies already in stores
+   - Focus on new suggestions
+
+5. **Find repository URLs** using WebSearch:
+   - For top 5 most-used dependencies
+   - Search for official GitHub/GitLab repositories
+   - Prefer official repos over forks
+
+6. **Present selectable list** using AskUserQuestion:
+
+First, show a summary of what was found:
+```
+## Dependency Analysis
+
+Scanned 342 source files and found 24 dependencies.
+Already indexed: typescript, express, jest
+```
+
+Then use AskUserQuestion with multiSelect to let the user choose which to add:
+
+```json
+{
+  "questions": [{
+    "question": "Which dependencies would you like to add to your knowledge stores?",
+    "header": "Add deps",
+    "multiSelect": true,
+    "options": [
+      {
+        "label": "react (Recommended)",
+        "description": "147 imports across 52 files - https://github.com/facebook/react"
+      },
+      {
+        "label": "lodash",
+        "description": "89 imports across 31 files - https://github.com/lodash/lodash"
+      },
+      {
+        "label": "axios",
+        "description": "45 imports across 18 files - https://github.com/axios/axios"
+      },
+      {
+        "label": "zod",
+        "description": "32 imports across 12 files - https://github.com/colinhacks/zod"
+      }
+    ]
+  }]
+}
+```
+
+**Note:** Maximum 4 options per question (AskUserQuestion limit). If more than 4 dependencies, show top 4 and mention others in summary.
+
+7. **Execute selected add-repo commands**:
+
+For each selected dependency, invoke the Skill tool:
+```
+Skill(skill="bluera-knowledge:add-repo", args="<repo-url> --name=<package-name>")
+```
+
+Example for react selection:
+```
+Skill(skill="bluera-knowledge:add-repo", args="https://github.com/facebook/react --name=react")
+```
+
+8. **Show completion summary**:
+```
+## Added to Knowledge Stores
+
+✓ react - https://github.com/facebook/react
+✓ lodash - https://github.com/lodash/lodash
+
+Indexing started. Check progress with /bluera-knowledge:check-status
+```
+
+## If No Dependencies Found
+
+```
+No external dependencies found in this project.
+
+Make sure you have a dependency manifest file:
+- package.json (JavaScript/TypeScript)
+- requirements.txt or pyproject.toml (Python)
+- go.mod (Go)
+- Cargo.toml (Rust)
+```
+
+## If User Selects "Other"
+
+If the user types a custom response instead of selecting options, try to parse package names from their input and search for the corresponding repositories.