bluera-knowledge 0.34.0 → 0.34.2

package/skills/store-lifecycle/SKILL.md

@@ -3,452 +3,24 @@ name: store-lifecycle
 description: Create, index, and manage BK stores
 ---
 
-# Bluera Knowledge Store Lifecycle Management
+# Store Lifecycle Management
 
-Master the lifecycle of knowledge stores from creation to deletion, including best practices for naming, indexing, and maintenance.
+## Source Types
 
-## Choosing the Right Source Type
-
-Bluera Knowledge supports three source types. Choose based on your content:
-
-### Git Repositories (`add-repo` / `create_store` with git URL)
-
-**✅ Use for:**
-- Public library source code (React, Vue, Pydantic, etc.)
-- Private repositories with auth
-- Code you want to track and update
-- Multi-file projects with git history
-
-**Advantages:**
-- Preserves git history
-- Can pull updates (`git pull` in repo directory)
-- Standard structure recognized by analyzers
-- Automatic language detection
-
-**Example:**
-```
-/bluera-knowledge:add-repo https://github.com/vuejs/core --name=vue
-
-# Or via MCP:
-create_store(
-  source='https://github.com/vuejs/core',
-  name='vue',
-  type='repo'
-)
-```
-
-**Best practices:**
-- Use package/library name for consistency: `vue`, `fastapi`, `pydantic`
-- For monorepos: `org-project` format: `microsoft-typescript`, `vercel-next`
-- Include version if tracking specific release: `vue-3.4`, `python-3.11`
-
-### Local Folders (`add-folder`)
-
-**✅ Use for:**
-- Private codebases not in git
-- Work-in-progress code
-- Local documentation
-- Specific subdirectories of larger projects
-
-**Advantages:**
-- No git required
-- Fast indexing (no clone step)
-- Perfect for proprietary code
-- Can index subset of larger repo
-
-**Example:**
-```
-/bluera-knowledge:add-folder /path/to/my-project/api --name=my-api
-
-# Or via MCP:
-create_store(
-  source='/Users/me/projects/my-app/backend',
-  name='my-backend',
-  type='folder'
-)
-```
-
-**Best practices:**
-- Use descriptive names: `my-api`, `auth-service`, `shared-utils`
-- Index focused directories (not entire ~/ )
-- Update by re-indexing: `/bluera-knowledge:index my-api`
-
-### Web Documentation (`crawl`)
-
-**✅ Use for:**
-- Official documentation sites
-- API references hosted online
-- Tutorials and guides
-- Content only available via web
-
-**Advantages:**
-- Access web-only content
-- Handles JavaScript-rendered sites (headless mode)
-- Follows links automatically
-- Converts HTML to searchable text
-
-**Example:**
-```
-/bluera-knowledge:crawl https://fastapi.tiangolo.com --name=fastapi-docs --max-pages=100
-
-# Or via MCP:
-create_store(
-  source='https://fastapi.tiangolo.com',
-  name='fastapi-docs',
-  type='web',
-  max_pages=100
-)
-```
-
-**Best practices:**
-- Append `-docs` to library name: `fastapi-docs`, `vue-docs`
-- Set `max-pages` to avoid crawling entire internet
-- Use `--headless` for JavaScript-heavy sites
-- Crawl specific documentation paths, not marketing pages
+| Type | Command | Best For |
+|------|---------|----------|
+| Git repo | `/add-repo <url>` | Public libraries, trackable code |
+| Local folder | `/add-folder <path>` | Private code, subdirectories |
+| Web docs | `/crawl <url>` | Documentation sites, API refs |
 
 ## Naming Conventions
 
-Good names make stores easy to find and filter.
-
-### Recommended Patterns
-
-**Library source code:**
-```
-vue          # Official package name
-react
-fastapi
-pydantic
-```
-
-**Documentation sites:**
-```
-vue-docs
-fastapi-docs
-python-3.11-docs
-```
-
-**Organization/project format:**
-```
-microsoft-typescript
-vercel-next
-acme-payment-api      # Your company's code
-```
-
-**Versioned stores:**
-```
-vue-3.4
-python-3.11
-react-18
-```
-
-**Specialized content:**
-```
-coding-standards      # Company standards
-api-spec-v2          # API specification
-architecture-docs    # Design docs
-```
-
-### Naming Anti-Patterns
-
-❌ Avoid:
-- Generic names: `docs`, `code`, `library`
-- Unclear abbreviations: `fp`, `lib1`, `proj`
-- Dates without context: `2024-01-15`
-- Redundant words: `my-project-library-code`
-
-✅ Prefer:
-- Specific, descriptive: `fastapi-docs`, `vue-source`
-- Standard package names: `pydantic`, `lodash`
-- Clear context: `api-spec-v2`, `coding-standards`
-
-## Indexing Strategies
-
-### Initial Indexing
-
-When creating a store, indexing happens automatically in the background:
-
-```
-create_store(url, name)
-→ Returns: job_id
-→ Background: clone/download → analyze → index
-→ Status: pending → running → completed
-
-# Monitor progress
-check_job_status(job_id)
-→ Progress: 45% (processing src/core.ts)
-→ Estimated: ~2 minutes remaining
-```
-
-**Indexing time estimates:**
-- Small library (<1k files): 30-60 seconds
-- Medium library (1k-5k files): 1-3 minutes
-- Large library (>5k files): 3-10 minutes
-- Documentation crawl (100 pages): 1-2 minutes
-
-### Re-indexing (Updates)
-
-When library code changes or you modify indexed content:
-
-```
-# For git repos: pull latest changes
-cd .bluera/bluera-knowledge/repos/vue
-git pull origin main
-cd -
-
-# Re-index
-/bluera-knowledge:index vue
-
-# Or via MCP:
-index_store(store='vue')
-→ Re-processes all files
-→ Updates vector embeddings
-→ Rebuilds search index
-```
-
-**When to re-index:**
-- Library released new version
-- You modified local folder content
-- Search results seem outdated
-- After significant codebase changes
-
-**Re-indexing is incremental** - only changed files are re-processed.
-
-### Selective Indexing
-
-For large repos, you might want to index specific directories:
-
-```
-# Clone full repo manually
-git clone https://github.com/microsoft/vscode
-cd vscode
-
-# Index only specific dirs
-/bluera-knowledge:add-folder ./src/vs/editor --name=vscode-editor
-/bluera-knowledge:add-folder ./src/vs/workbench --name=vscode-workbench
-
-# Result: Multiple focused stores instead of one massive store
-```
-
-## Storage Management
-
-### Monitoring Storage
-
-Check what's using space:
-
-```
-list_stores()
-→ vue: 487 files, 2.3 MB
-→ react: 312 files, 1.8 MB
-→ fastapi-docs: 156 pages, 0.9 MB
-→ my-api: 89 files, 0.4 MB
-
-Total storage: ~5.4 MB
-
-# Detailed info
-get_store_info('vue')
-→ Location: .bluera/bluera-knowledge/repos/vue/
-→ Indexed: 487 files
-→ Size: 2.3 MB (source) + 4.1 MB (vectors)
-→ Last indexed: 2 hours ago
-```
-
-### When to Delete Stores
-
-**✅ Delete when:**
-- Library no longer relevant to your project
-- Documentation outdated (re-crawl instead)
-- Testing/experimental stores no longer needed
-- Running low on disk space
-- Duplicate stores exist
-
-**How to delete:**
-```
-/bluera-knowledge:remove-store old-library
-
-# Or via MCP:
-delete_store(store='old-library')
-→ Removes: source files, vector index, metadata
-→ Frees: ~6-8 MB per store (varies by size)
-```
-
-**⚠️ Cannot undo!** Make sure you don't need the store before deleting.
-
-## Background Job Monitoring
-
-All expensive operations run as background jobs: cloning, indexing, crawling.
-
-### Job Lifecycle
-
-```
-1. create_store() or index_store() → Returns job_id
-
-2. Job states:
-   - pending: In queue, not started
-   - running: Actively processing
-   - completed: Finished successfully
-   - failed: Error occurred
-
-3. Monitor progress:
-   check_job_status(job_id)
-   → Current state, percentage, current file
-
-4. List all jobs:
-   list_jobs()
-   → See pending, running, completed jobs
-
-5. Cancel if needed:
-   cancel_job(job_id)
-   → Stops running job, cleans up
-```
-
-### Best Practices for Job Monitoring
-
-**Do poll, but not too frequently:**
-```
-# ❌ Too frequent - wastes resources
-while status != 'completed':
-    check_job_status(job_id)  # Every second!
-    sleep(1)
-
-# ✅ Reasonable polling interval
-while status != 'completed':
-    check_job_status(job_id)
-    sleep(15)  # Every 15 seconds is fine
-```
-
-**Do handle failures gracefully:**
-```
-status = check_job_status(job_id)
-
-if status['state'] == 'failed':
-    error = status['error']
-
-    if 'auth' in error.lower():
-        print("Authentication required - try SSH URL or provide credentials")
-    elif 'not found' in error.lower():
-        print("Repository/URL not found - check the source")
-    elif 'disk' in error.lower():
-        print("Disk space issue - delete unused stores")
-    else:
-        print(f"Unexpected error: {error}")
-```
-
-**Do list jobs to avoid duplicates:**
-```
-# Before creating new store
-jobs = list_jobs()
-existing = [j for j in jobs if j['store'] == 'vue' and j['state'] in ['pending', 'running']]
-
-if existing:
-    print(f"Job already running for 'vue': {existing[0]['id']}")
-    # Wait for it instead of creating duplicate
-else:
-    create_store(...)
-```
-
-## Handling Indexing Failures
+- **Library source**: package name (`vue`, `fastapi`, `pydantic`)
+- **Documentation**: append `-docs` (`vue-docs`, `fastapi-docs`)
+- **Org/project**: `org-project` (`microsoft-typescript`)
+- **Versioned**: `name-version` (`react-18`, `python-3.11`)
 
-### Common Failure Scenarios
-
-**1. Authentication Required (Private Repos)**
-```
-Error: "Authentication required"
-
-Fix options:
-  - Use SSH URL: git@github.com:org/repo.git
-  - Use HTTPS with token: https://token@github.com/org/repo.git
-  - Make repo public (if appropriate)
-```
-
-**2. Invalid URL/Path**
-```
-Error: "Repository not found" or "Path does not exist"
-
-Fix:
-  - Verify URL is correct (typos common!)
-  - Check path exists and is accessible
-  - Ensure network connectivity
-```
-
-**3. Disk Space**
-```
-Error: "No space left on device"
-
-Fix:
-  - Check available space: df -h
-  - Delete unused stores: delete_store(old_store)
-  - Clear .bluera/bluera-knowledge/repos/ manually if needed
-```
-
-**4. Network Timeout**
-```
-Error: "Connection timeout" or "Failed to fetch"
-
-Fix:
-  - Retry after checking network
-  - Use --shallow for large repos
-  - Clone manually then add-folder
-```
-
-**5. Unsupported File Types**
-```
-Warning: "Skipped 45 binary files"
-
-This is normal!
-  - Binary files (images, compiled code) are skipped
-  - Only text files are indexed
-  - Check indexed count vs total to see ratio
-```
-
-### Recovery Workflow
-
-```
-1. Attempt fails:
-   create_store(url, name) → job fails
-
-2. Check error:
-   job_status = check_job_status(job_id)
-   error_msg = job_status['error']
-
-3. Determine fix based on error type (see above)
-
-4. Retry with fix:
-   create_store(corrected_url, name)
-
-5. Verify success:
-   check_job_status(new_job_id)
-   → Status: completed
-
-   list_stores()
-   → Store appears in list
-
-6. Test search:
-   search(test_query, stores=[name], limit=3)
-   → Returns results: ✅ Ready to use!
-```
-
-## Store Lifecycle Checklist
-
-**Creating a Store:**
-- [ ] Choose appropriate source type (repo/folder/crawl)
-- [ ] Use descriptive, consistent naming
-- [ ] Start indexing job
-- [ ] Monitor job status until complete
-- [ ] Verify with list_stores()
-- [ ] Test with sample search
-
-**Maintaining a Store:**
-- [ ] Re-index after significant changes
-- [ ] Pull git updates periodically for repo stores
-- [ ] Monitor storage usage
-- [ ] Check search relevance quality
-
-**Deleting a Store:**
-- [ ] Confirm no longer needed
-- [ ] Note storage freed
-- [ ] Remove from any documentation referencing it
-
-## Quick Reference Commands
+## Quick Reference
 
 ```
 # Create
@@ -467,4 +39,7 @@ This is normal!
 /bluera-knowledge:remove-store <name>
 ```
 
-Master these lifecycle management practices to maintain a clean, efficient, and useful knowledge base.
+## Detailed Guides
+
+- Source type details and examples: [references/source-types.md](references/source-types.md)
+- Indexing, storage, and job monitoring: [references/operations.md](references/operations.md)

package/skills/store-lifecycle/references/operations.md

@@ -0,0 +1,75 @@
+# Store Operations
+
+## Indexing
+
+### Initial Indexing
+
+When creating a store, indexing happens automatically in the background:
+
+```
+create_store(url, name) → Returns: job_id
+→ Background: clone/download → analyze → index
+→ Status: pending → running → completed
+```
+
+**Time estimates:**
+- Small library (<1k files): 30-60 seconds
+- Medium library (1k-5k files): 1-3 minutes
+- Large library (>5k files): 3-10 minutes
+- Documentation crawl (100 pages): 1-2 minutes
+
+### Re-indexing
+
+Re-indexing is incremental — only changed files are re-processed.
+
+**When to re-index:**
+- Library released new version
+- You modified local folder content
+- Search results seem outdated
+
+### Selective Indexing
+
+For large repos, index specific directories:
+
+```
+git clone https://github.com/microsoft/vscode
+/bluera-knowledge:add-folder ./vscode/src/vs/editor --name=vscode-editor
+/bluera-knowledge:add-folder ./vscode/src/vs/workbench --name=vscode-workbench
+```
+
+## Background Job Monitoring
+
+```
+1. create_store() or index_store() → Returns job_id
+2. Job states: pending → running → completed | failed
+3. Monitor: check_job_status(job_id) → state, percentage, current file
+4. List all: list_jobs() → pending, running, completed jobs
+5. Cancel: cancel_job(job_id) → stops and cleans up
+```
+
+**Polling:** Every 15 seconds is reasonable. Don't poll every second.
+
+## Common Failure Scenarios
+
+| Error | Cause | Fix |
+|-------|-------|-----|
+| Authentication required | Private repo | Use SSH URL or HTTPS with token |
+| Repository not found | Typo in URL | Verify URL |
+| No space left | Disk full | Delete unused stores |
+| Connection timeout | Network issue | Retry, or clone manually then add-folder |
+| Skipped binary files | Normal behavior | Only text files are indexed |
+
+## Storage Management
+
+```
+list_stores()
+→ vue: 487 files, 2.3 MB
+→ react: 312 files, 1.8 MB
+
+get_store_info('vue')
+→ Location, indexed files, size, last indexed
+```
+
+**Delete when:** Library no longer relevant, documentation outdated, duplicates exist, disk space needed.
+
+**Cannot undo deletion!**

package/skills/store-lifecycle/references/source-types.md

@@ -0,0 +1,48 @@
+# Source Types
+
+## Git Repositories
+
+**Advantages:** Preserves git history, can pull updates, automatic language detection.
+
+```
+/bluera-knowledge:add-repo https://github.com/vuejs/core --name=vue
+```
+
+**Best practices:**
+- Use package/library name: `vue`, `fastapi`, `pydantic`
+- For monorepos: `org-project` format: `microsoft-typescript`, `vercel-next`
+- Include version if tracking specific release: `vue-3.4`, `python-3.11`
+
+## Local Folders
+
+**Advantages:** No git required, fast indexing, perfect for proprietary code.
+
+```
+/bluera-knowledge:add-folder /path/to/my-project/api --name=my-api
+```
+
+**Best practices:**
+- Use descriptive names: `my-api`, `auth-service`, `shared-utils`
+- Index focused directories (not entire ~/)
+- Update by re-indexing: `/bluera-knowledge:index my-api`
+
+## Web Documentation
+
+**Advantages:** Access web-only content, handles JS-rendered sites, follows links.
+
+```
+/bluera-knowledge:crawl https://fastapi.tiangolo.com --name=fastapi-docs --max-pages=100
+```
+
+**Best practices:**
+- Append `-docs` to library name: `fastapi-docs`, `vue-docs`
+- Set `max-pages` to avoid crawling entire internet
+- Use `--headless` for JavaScript-heavy sites
+- Crawl specific documentation paths, not marketing pages
+
+## Naming Anti-Patterns
+
+- Generic names: `docs`, `code`, `library`
+- Unclear abbreviations: `fp`, `lib1`, `proj`
+- Dates without context: `2024-01-15`
+- Redundant words: `my-project-library-code`