bluera-knowledge 0.31.0 → 0.33.0

package/skills/eval/SKILL.md

@@ -0,0 +1,222 @@
+---
+description: Evaluate agent quality across three modes — without BK, BK grep-only, and BK full
+argument-hint: "[query | --predefined | --predefined N]"
+allowed-tools: ["mcp__bluera-knowledge__execute", "mcp__bluera-knowledge__search", "mcp__bluera-knowledge__get_full_context", "Read", "Grep", "Glob", "WebSearch", "Bash"]
+context: fork
+---
+
+# Agent Quality Evaluation
+
+Compare how well Claude answers library questions across three access levels.
+
+For each query, three agents run in parallel:
+- **Without BK** — uses only web search and training knowledge
+- **BK Grep** — can Grep/Read/Glob the cloned source repos but has no vector search
+- **BK Full** — uses BK vector search + get_full_context + Grep/Read (all BK tools)
+
+Then score all three answers on accuracy, specificity, completeness, and source grounding.
+
+## Arguments
+
+Parse `$ARGUMENTS`:
+
+- **No arguments or empty**: Show usage help
+- **Quoted string** (not starting with `--`): Arbitrary query mode — run eval for that single question
+- **`--predefined`**: Run all predefined queries (skip any whose stores are not indexed)
+- **`--predefined N`**: Run predefined query #N only (1-based index)
+
+If no arguments provided, show:
+```
+Usage:
+  /bluera-knowledge:eval "How does Express handle errors?"     # Arbitrary query
+  /bluera-knowledge:eval --predefined                          # Run all predefined queries
+  /bluera-knowledge:eval --predefined 3                        # Run predefined query #3
+```
+
+## Step 1: Prerequisites Check
+
+1. Call MCP `execute` with `{ command: "stores" }` to list indexed stores
+2. If no stores are indexed, show error and abort:
+   ```
+   No knowledge stores indexed. Add at least one library first:
+     /bluera-knowledge:add-repo https://github.com/expressjs/express --name express
+   ```
+3. Record the list of available store names — you'll pass these to the BK Full agent
+4. Build a `STORE_PATHS` mapping from the store response: for each store with a `path` field, record `- **<name>**: \`<path>\`` (one per line, as a markdown list). This gets passed to the BK Grep agent.
+
+## Step 2: Resolve Queries
+
+### Predefined mode (`--predefined`)
+
+1. Read the predefined queries file: `$CLAUDE_PLUGIN_ROOT/evals/agent-quality/queries/predefined.yaml`
+2. Parse the YAML content
+3. For each query, check if ANY of its `store_hint` values match an available store name
+4. Split into **runnable** (store available) and **skipped** (store not available) lists
+5. If `--predefined N` was specified, select only query at index N from the full list (skip if store not available)
+6. If no queries are runnable, show what stores to add and abort
+
+### Arbitrary mode (bare query string)
+
+1. Use the raw query string as the question
+2. Set `expected_topics` and `anti_patterns` to empty lists
+3. Set `id` to "arbitrary", `category` to "general", `difficulty` to "unknown"
+
+## Step 3: Load Templates
+
+Read these files from `$CLAUDE_PLUGIN_ROOT/evals/agent-quality/templates/`:
+
+1. `without-bk-agent.md` — instructions for the baseline agent
+2. `bk-grep-agent.md` — instructions for the BK Grep agent
+3. `with-bk-agent.md` — instructions for the BK Full agent
+4. `judge.md` — grading rubric
+
+## Step 4: Run Eval (for each query)
+
+### Spawn ALL THREE agents in parallel (same turn, three Task tool calls)
+
+**Without-BK agent** — Use the Task tool with `subagent_type: "general-purpose"`:
+- Take the content from `without-bk-agent.md`
+- Replace `{{QUESTION}}` with the actual question
+- Send as the task prompt
+
+**BK Grep agent** — Use the Task tool with `subagent_type: "general-purpose"`:
+- Take the content from `bk-grep-agent.md`
+- Replace `{{QUESTION}}` with the actual question
+- Replace `{{STORE_PATHS}}` with the store name-to-path mapping built in Step 1
+- Send as the task prompt
+
+**BK Full agent** — Use the Task tool with `subagent_type: "general-purpose"`:
+- Take the content from `with-bk-agent.md`
+- Replace `{{QUESTION}}` with the actual question
+- Replace `{{STORES}}` with the list of available store names (one per line, as a markdown list)
+- Send as the task prompt
+
+Wait for all three agents to complete.
+
+### Capture Token Usage
+
+From each Task tool response, parse the `<usage>` block to extract:
+- `total_tokens` — the total tokens consumed by the agent
+- `duration_ms` — wall-clock time for the agent
+
+If usage data is not available in a Task response, show "N/A" for that agent.
+
+### Judge the results
+
+Using the rubric from `judge.md`, evaluate all three answers yourself:
+
+1. Read all three agent responses
+2. For each answer, score all 4 criteria (1-5):
+   - **Factual Accuracy**: Are the claims correct?
+   - **Specificity**: Does it cite specific files, functions, code?
+   - **Completeness**: Does it cover the full answer?
+   - **Source Grounding**: Are claims backed by evidence?
+3. If the query has `expected_topics`, check which answers mention each topic
+4. If the query has `anti_patterns`, flag if any answer makes those claims
+5. Calculate totals (max 20 each), determine winner and deltas
+
+## Step 5: Output Results
+
+### Single query output (arbitrary or `--predefined N`)
+
+Show the full comparison:
+
+```
+## Eval: "<question>"
+
+| Criterion         | Without BK | BK Grep | BK Full |
+|-------------------|:----------:|:-------:|:-------:|
+| Accuracy          |     X      |    X    |    X    |
+| Specificity       |     X      |    X    |    X    |
+| Completeness      |     X      |    X    |    X    |
+| Source Grounding   |     X      |    X    |    X    |
+| **Total**         |   **X**    |  **X**  |  **X**  |
+
+| Usage             | Without BK | BK Grep | BK Full |
+|-------------------|:----------:|:-------:|:-------:|
+| Tokens            |   X,XXX    |  X,XXX  |  X,XXX  |
+| Duration (s)      |    X.X     |   X.X   |   X.X   |
+
+**Winner:** [BK Full | BK Grep | Without BK | Tie] ([significant | marginal | none])
+**Key Difference:** [One sentence explaining the most important quality gap]
+**Grep vs Full:** [One sentence on whether vector search outperformed manual grep, and if so how]
+```
+
+If expected topics were provided:
+```
+### Expected Topics
+- [x] topic covered by all three
+- [x] topic covered by BK Full + BK Grep only
+- [x] topic covered by BK Full only
+- [ ] topic missed by all
+```
+
+### Multi-query output (`--predefined`)
+
+Show a summary row per query, then aggregate:
+
+```
+## Agent Quality Eval Summary
+
+Ran X/8 queries (Y skipped — stores not indexed)
+
+| # | Query | Difficulty | w/o BK | Grep | Full | Winner | Delta |
+|---|-------|:----------:|:------:|:----:|:----:|--------|-------|
+| 1 | query-id | medium | 9/20 | 15/20 | 19/20 | Full | significant |
+| 2 | query-id | easy | 14/20 | 17/20 | 18/20 | Full | marginal |
+| ... |
+
+### Token Usage
+
+| # | Query | w/o BK tokens | Grep tokens | Full tokens |
+|---|-------|:-------------:|:-----------:|:-----------:|
+| 1 | query-id | 2,340 | 8,120 | 5,670 |
+| 2 | query-id | 1,890 | 6,450 | 4,230 |
+| ... |
+
+### Aggregate
+- **Without BK mean:** X.X/20 (avg X,XXX tokens)
+- **BK Grep mean:** X.X/20 (avg X,XXX tokens)
+- **BK Full mean:** X.X/20 (avg X,XXX tokens)
+- **Full vs Without:** +X.X points (+XX%)
+- **Full vs Grep:** +X.X points (+XX%)
+- **Grep vs Without:** +X.X points (+XX%)
+- **Full win rate:** X/X (XX%)
+- **Significant wins (Full):** X
+
+### By Category
+| Category | w/o BK | Grep | Full | Full delta |
+|----------|:------:|:----:|:----:|------------|
+| implementation | X.X | X.X | X.X | +X.X |
+| api | X.X | X.X | X.X | +X.X |
+
+### By Difficulty
+| Difficulty | w/o BK | Grep | Full | Full delta |
+|------------|:------:|:----:|:----:|------------|
+| easy | X.X | X.X | X.X | +X.X |
+| medium | X.X | X.X | X.X | +X.X |
+| hard | X.X | X.X | X.X | +X.X |
+
+### Token Efficiency
+| Agent | Mean Score | Mean Tokens | Score/1K Tokens |
+|-------|:----------:|:-----------:|:---------------:|
+| Without BK | X.X | X,XXX | X.XX |
+| BK Grep | X.X | X,XXX | X.XX |
+| BK Full | X.X | X,XXX | X.XX |
+```
+
+If any queries were skipped:
+```
+### Skipped (store not indexed)
+- vue-reactivity-tracking — add with: /bluera-knowledge:add-repo https://github.com/vuejs/core --name vue
+- fastapi-dependency-injection — add with: /bluera-knowledge:add-repo https://github.com/fastapi/fastapi --name fastapi
+```
+
+## Important Notes
+
+- Each query spawns 3 subagents. For `--predefined` with 8 queries, that's up to 24 agent runs. Process one query at a time (but spawn all three agents for each query in parallel).
+- The without-BK agent may use WebSearch — this is intentional. We're comparing against "the best Claude can do without BK."
+- The BK Grep agent may NOT use WebSearch. It tests what an agent can discover by exploring raw source code, to isolate the value of vector search.
+- Scoring is somewhat subjective. The value is in the comparison (relative scores) rather than absolute numbers. Look at the delta and key differences.
+- The Token Efficiency table reveals cost-effectiveness: if BK Grep achieves similar scores to BK Full with fewer tokens, it suggests vector search isn't adding much for that query type.
+- For arbitrary queries without expected topics, grading relies entirely on the 4 general criteria. This is fine — it still reveals whether BK adds value.

package/skills/health/SKILL.md

@@ -0,0 +1,72 @@
+---
+description: Check health of all stores (path existence, model compatibility)
+allowed-tools: ["mcp__bluera-knowledge__execute"]
+---
+
+# Store Health Check
+
+Diagnose issues with knowledge stores: missing paths, schema migrations, model mismatches.
+
+## Steps
+
+1. Use the mcp__bluera-knowledge__execute tool with command "stores:health" to check all stores
+
+2. Present results grouped by status:
+
+```
+## Store Health Report
+
+### Errors (require action)
+| Store | Type | Issue | Fix |
+|-------|------|-------|-----|
+| my-repo | repo | Path not found | Re-create store or fix projectRoot |
+
+### Warnings (recommended action)
+| Store | Type | Issue | Fix |
+|-------|------|-------|-----|
+| old-docs | web | Schema v1 | Run: /bluera-knowledge:index old-docs |
+
+### Healthy
+- react-docs (web)
+- lodash (repo)
+
+**Summary**: 2 healthy, 1 warning, 1 error
+```
+
+## Exit Codes
+
+The health check returns an exit code for scripting:
+
+| Exit Code | Meaning |
+|-----------|---------|
+| 0 | All stores healthy |
+| 1 | At least one store has an error (path not found) |
+| 2 | No errors, but at least one warning (model/schema issue) |
+
+## Issue Types
+
+### PATH_NOT_FOUND (Error)
+The store's source path no longer exists. This happens when:
+- A local folder was deleted or moved
+- The project was relocated and paths weren't updated
+- A cloned repo directory was removed
+
+**Fix**: Re-create the store or update the projectRoot setting.
+
+### SCHEMA_V1 (Warning)
+The store was created before model tracking was added. It needs to be re-indexed to be searchable.
+
+**Fix**: Run `/bluera-knowledge:index <store-name>`
+
+### MODEL_MISMATCH (Warning)
+The store was indexed with a different embedding model than the current configuration.
+
+**Fix**: Run `/bluera-knowledge:index <store-name>` to re-index with the current model.
+
+## Check Single Store
+
+To check a specific store only:
+
+```
+stores:health --store=<store-name>
+```

package/skills/index/SKILL.md

@@ -0,0 +1,48 @@
+---
+description: Re-index a knowledge store
+argument-hint: "[store-name-or-id]"
+allowed-tools: ["mcp__bluera-knowledge__execute"]
+---
+
+# Re-index Knowledge Store
+
+Re-index a knowledge store: **$ARGUMENTS**
+
+## Steps
+
+1. Parse the store name or ID from $ARGUMENTS (required)
+
+2. Use mcp__bluera-knowledge__execute tool with command "store:index":
+   - args.store: The store name or ID from $ARGUMENTS
+
+3. Display results showing job ID for background indexing:
+
+```
+✓ Indexing store: react...
+🔄 Indexing started in background
+   Job ID: job_def456ghi789
+
+Check status with: /bluera-knowledge:check-status job_def456ghi789
+Or view all jobs: /bluera-knowledge:check-status
+```
+
+## When to Re-index
+
+Re-index a store when:
+- The source repository has been updated (for repo stores)
+- Files have been added or modified (for file stores)
+- You want to refresh embeddings with an updated model
+- Search results seem out of date
+
+## Error Handling
+
+If indexing fails:
+
+```
+✗ Failed to index store: [error message]
+
+Common issues:
+- Store name or ID not found - use /bluera-knowledge:stores to list available stores
+- Source directory no longer exists (for file stores)
+- Network issues pulling latest changes (for repo stores)
+```

package/skills/knowledge-search/SKILL.md

@@ -0,0 +1,110 @@
+---
+name: knowledge-search
+description: Query BK for library internals via vector search or direct Grep/Read
+---
+
+# Using Bluera Knowledge (BK)
+
+BK provides access to **definitive library sources** for your project dependencies.
+
+## The Rule: Query BK for External Code
+
+**Any question about libraries, dependencies, or indexed reference material should query BK.**
+
+BK is:
+- **Cheap**: ~100ms response, unlimited queries, no rate limits
+- **Authoritative**: Actual source code, not blog posts or training data
+- **Complete**: Includes tests, examples, internal APIs, configuration
+
+## Always Query BK For:
+
+**Library implementation:**
+- "How does Express handle middleware errors?"
+- "What does React's useEffect cleanup actually do?"
+- "How is Pydantic validation implemented?"
+
+**API signatures and options:**
+- "What parameters does axios.create() accept?"
+- "What options can I pass to hono.use()?"
+- "What's the signature of zod.object()?"
+
+**Error handling:**
+- "What errors can this library throw?"
+- "Why might this function return undefined?"
+- "What validation does Zod perform?"
+
+**Version-specific behavior:**
+- "What changed in React 18?"
+- "Is this deprecated in Express 5?"
+- "Does my version support this?"
+
+**Configuration:**
+- "What config options exist for Vite?"
+- "What are the default values?"
+- "What environment variables does this use?"
+
+**Testing:**
+- "How do the library authors test this?"
+- "How should I mock this in tests?"
+- "What edge cases do the tests cover?"
+
+**Performance:**
+- "Is this cached internally?"
+- "What's the complexity of this operation?"
+- "Does this run async or sync?"
+
+**Security:**
+- "How does this validate input?"
+- "Is this safe against injection?"
+- "How are credentials handled?"
+
+**Integration:**
+- "How do I integrate X with Y?"
+- "What's the idiomatic usage pattern?"
+- "How do examples in the library do this?"
+
+## Two Ways to Access Library Sources
+
+### 1. Vector Search (Discovery)
+Find concepts and patterns across indexed content:
+```
+search("vue reactivity system")
+/bluera-knowledge:search "pydantic custom validators"
+```
+
+### 2. Direct File Access (Precision)
+Precise lookups in cloned library source:
+```
+Grep: pattern="defineReactive" path=".bluera/bluera-knowledge/repos/vue/"
+Read: .bluera/bluera-knowledge/repos/pydantic/pydantic/validators.py
+```
+
+Both are valid! Use vector search for discovery, Grep/Read for specific functions.
+
+## DO NOT Query BK For:
+
+- **Your project code** → Use Grep/Read directly
+- **General concepts** → Use training data ("What is a closure?")
+- **Breaking news** → Use web search ("Latest React release")
+
+## Example Workflow
+
+User: "How does Vue's computed properties work internally?"
+
+Claude:
+1. Check stores: `list_stores` MCP tool → vue store exists
+2. Vector search: `search("vue computed properties")` → finds computed.ts
+3. Read file: `.bluera/bluera-knowledge/repos/vue/packages/reactivity/src/computed.ts`
+4. Grep for implementation: pattern="class ComputedRefImpl"
+5. Explain with authoritative source code examples
+
+## Quick Reference
+
+```
+[library] question        → Query BK
+[your code] question      → Grep/Read directly
+[concept] question        → Training data
+[news/updates] question   → Web search
+```
+
+BK is cheap and fast. Query it liberally for library questions.

package/skills/remove-store/SKILL.md

@@ -0,0 +1,52 @@
+---
+description: Delete a knowledge store and all associated data
+argument-hint: "[store-name-or-id]"
+allowed-tools: ["mcp__bluera-knowledge__execute"]
+---
+
+# Remove Knowledge Store
+
+Delete a knowledge store and all associated data: **$ARGUMENTS**
+
+## Steps
+
+1. Parse the store name or ID from $ARGUMENTS (required)
+   - If no store provided, show error and suggest using /bluera-knowledge:stores to list available stores
+
+2. Use mcp__bluera-knowledge__execute tool with command "store:delete":
+   - args.store: The store name or ID from $ARGUMENTS
+
+3. Display deletion result:
+
+```
+Store "react" deleted successfully.
+
+Removed:
+- Store registry entry
+- LanceDB search index
+- Cloned repository files (for repo stores)
+```
+
+## What Gets Deleted
+
+When you remove a store:
+- **Registry entry** - Store is removed from the list
+- **Search index** - LanceDB vector embeddings are dropped
+- **Cloned files** - For repo stores created from URLs, the cloned repository is deleted
+
+## Warning
+
+This action is **permanent**. The store and all indexed data will be deleted.
+To re-create, you'll need to add and re-index the source again.
+
+## Error Handling
+
+If store not found:
+
+```
+Store not found: nonexistent-store
+
+Available stores:
+- Use /bluera-knowledge:stores to list all stores
+- Check spelling of store name or ID
+```

package/skills/search/SKILL.md

@@ -0,0 +1,80 @@
+---
+description: Search indexed library sources
+argument-hint: "[query] [--stores names] [--limit N] [--mode vector|fts|hybrid] [--detail minimal|contextual|full] [--threshold 0-1] [--min-relevance 0-1]"
+allowed-tools: ["mcp__bluera-knowledge__search"]
+---
+
+# Search Knowledge Stores
+
+Search indexed library sources for: **$ARGUMENTS**
+
+## Steps
+
+1. Parse the query from $ARGUMENTS:
+   - Extract the search query (required)
+   - Extract --stores parameter (optional, comma-separated store names)
+   - Extract --limit parameter (optional, default 10)
+   - Extract --mode parameter (optional: vector, fts, hybrid; default hybrid)
+   - Extract --detail parameter (optional: minimal, contextual, full; default minimal)
+   - Extract --threshold parameter (optional, 0-1 range for normalized score filtering)
+   - Extract --min-relevance parameter (optional, 0-1 range for raw cosine similarity filtering)
+
+2. Call mcp__bluera-knowledge__search with:
+   - query: The search query string
+   - stores: Array of store names (if --stores specified)
+   - limit: Number of results (if --limit specified, default 10)
+   - mode: Search mode (if --mode specified, default "hybrid")
+   - detail: Detail level (if --detail specified, default "minimal")
+   - threshold: Minimum normalized score (if --threshold specified)
+   - minRelevance: Minimum raw cosine similarity (if --min-relevance specified)
+   - intent: "find-implementation"
+
+3. Format and display results with rich context:
+
+   ```
+   ## Search Results: "query" (hybrid search)
+
+   **1. [Score: 0.95] [Vector+FTS]**
+   Store: claude-code
+   File: 📄 path/to/file.ts
+   Purpose: → Purpose description here
+   Top Terms: 🔑 (in this chunk): concept1, concept2, concept3
+   Imports: 📦 (in this chunk): package1, package2
+
+   **2. [Score: 0.87] [Vector]**
+   Store: another-store
+   File: 📄 path/to/file.js
+   Purpose: → Another purpose here
+   Top Terms: 🔑 (in this chunk): other-concept
+
+   ---
+   **Found 10 results in 45ms**
+
+   💡 **Next Steps:**
+   - Read file: `Read /path/to/file.ts`
+   - Get full code: `mcp__bluera-knowledge__get_full_context("result-id")`
+   - Refine search: Use keywords above
+   ```
+
+   **Formatting rules:**
+   - Header: `## Search Results: "query" (mode search)` - Extract mode from response (vector/fts/hybrid)
+   - Each result on its own block with blank line between
+   - Result header: `**N. [Score: X.XX]**`
+   - Store: `Store: storeName`
+   - File: `File: 📄 filename` (from summary.location, strip repoRoot prefix)
+   - Purpose: `Purpose: → purpose text` (from summary.purpose)
+   - Footer: `**Found {{totalResults}} results in {{timeMs}}ms**` with separator line above
+
+4. For the footer next steps, include:
+   - First result's ID in the get_full_context example
+   - First result's actual file path in the Read example
+   - Use the actual keywords from top results
+
+5. If no results:
+   ```
+   No results found for "query"
+
+   Try:
+   - Broadening your search terms
+   - Checking indexed stores: /bluera-knowledge:stores
+   ```

package/skills/search/search.sh

@@ -0,0 +1,63 @@
+#!/bin/bash
+# Search command that uses Python formatter for deterministic table output
+
+set -euo pipefail
+
+# Parse arguments
+QUERY=""
+STORES=""
+LIMIT=10
+
+while [[ $# -gt 0 ]]; do
+  case $1 in
+    --stores)
+      STORES="$2"
+      shift 2
+      ;;
+    --limit)
+      LIMIT="$2"
+      shift 2
+      ;;
+    *)
+      # Everything else is the query
+      if [ -z "$QUERY" ]; then
+        QUERY="$1"
+      else
+        QUERY="$QUERY $1"
+      fi
+      shift
+      ;;
+  esac
+done
+
+# Remove quotes from query if present
+QUERY=$(echo "$QUERY" | sed 's/^["'\'']*//;s/["'\'']*$//')
+
+if [ -z "$QUERY" ]; then
+  echo "Error: No search query provided"
+  exit 1
+fi
+
+# Build MCP tool call JSON
+TOOL_INPUT=$(cat <<EOF
+{
+  "query": "$QUERY",
+  "limit": $LIMIT,
+  "detail": "contextual",
+  "intent": "find-implementation"
+}
+EOF
+)
+
+# Call MCP tool (this would need to be done via Claude)
+# For now, output instructions for Claude
+cat <<EOF
+Please call mcp__bluera-knowledge__search with these parameters and pipe the results through the formatter:
+
+Query: $QUERY
+Limit: $LIMIT
+Detail: contextual
+Intent: find-implementation
+
+Then execute: echo '<results_json>' | ${CLAUDE_PLUGIN_ROOT}/hooks/format-search-results.py
+EOF