ultimate-pi 0.1.2 → 0.1.3

package/vault/wiki/questions/Research: semantic code search tools.md

@@ -0,0 +1,69 @@
+---
+type: synthesis
+title: "Research: semantic code search tools"
+created: 2026-04-30
+updated: 2026-04-30
+tags:
+  - research
+  - code-search
+  - semantic-search
+  - agentic-harness
+  - mcp
+status: developing
+related:
+  - "[[ck-tool]]"
+  - "[[vgrep-tool]]"
+  - "[[hybrid-code-search]]"
+  - "[[agent-search-enforcement]]"
+  - "[[mcp-tool-routing]]"
+sources:
+  - "[[ck-semantic-search]]"
+  - "[[vgrep-semantic-search]]"
+
+---# Research: semantic code search tools
+
+## Overview
+
+Self-hostable, free (zero API keys) semantic code search tools for an agentic AI coding harness. ck (BeaconBay/ck) is the strongest candidate: 1,572 stars, Rust, grep-compatible, MCP-native, fully offline. Its primary limitations are no code-aware AST embeddings and no custom model fine-tuning. Alternatives exist but are younger and less mature. Agent enforcement requires system-prompt rules + MCP tool registration + optional shell-level routing.
+
+## Key Findings
+
+- **ck is the best drop-in replacement for grep** in an AI coding harness. It preserves grep flags (`-n`, `-A`, `-B`, `-r`, `-l`), adds `--sem` and `--hybrid` modes, and ships an MCP server. Fully offline. (Source: [[ck-semantic-search]])
+- **vgrep is the strongest alternative** with client-server architecture and GPU acceleration, but lacks grep-compatible flags and MCP integration (Source: [[vgrep-semantic-search]])
+- **No tool currently enforces agent usage automatically**. Enforcement is a system-design problem, not a tool feature. Three-layer approach recommended: system prompt rules, MCP tool registration, optional shell wrapper (Source: [[agent-search-enforcement]])
+- **ast-grep** and **semgrep** are structural/pattern-based, not embedding-based. Complementary but different category. (Source: GitHub API search)
+- **ck limitations are real but manageable**: no code-aware embeddings, 80-language max, no custom training, ~4-8GB RAM for 10M+ LOC codebases (Source: [[ck-semantic-search]])
+
+## Key Entities
+
+- [[ck-tool]]: Hybrid code search (BM25 + embeddings), grep-compatible, MCP server, fully offline. 1,572 ⭐
+- [[vgrep-tool]]: Vector embedding search, client-server, GPU acceleration, privacy-first. 144 ⭐
+- [[autodev-codebase]]: TypeScript MCP server with call graphs + Ollama embeddings. 116 ⭐
+- [[ops-codegraph-tool]]: Python dependency graph engine + semantic search + boundary enforcement. 41 ⭐
+- [[codesearch]]: Rust MCP server designed for Claude Code/OpenCode. 16 ⭐
+
+## Key Concepts
+
+- [[hybrid-code-search]]: BM25 lexical + embedding semantic + Reciprocal Rank Fusion (RRF) for best-of-both-worlds results
+- [[agent-search-enforcement]]: Strategies to force AI agents to use semantic search instead of raw grep/cat pipes
+- [[mcp-tool-routing]]: Using MCP protocol to register semantic search as a first-class agent tool
+
+## Contradictions
+
+- **ck claims "drop-in grep replacement"** but its semantic mode (--sem) changes the semantics entirely. grep results are deterministic; semantic results depend on the embedding model. (Source: [[ck-semantic-search]]) This is a design tradeoff, not a bug.
+- **vgrep is "privacy-first"** (Source: [[vgrep-semantic-search]]) — but ck is also fully offline. Both have the same privacy posture. The distinction is marketing.
+- **Some sources claim tree-sitter-based search is "semantic"** but tree-sitter is structural (AST), not semantic (meaning). Tools like ast-grep and semgrep are complementary, not competing, with embedding-based search.
+
+## Open Questions
+
+- [ ] How does ck's embedding quality compare to code-specific models (CodeBERT, UniXCoder) on real-world queries? No independent benchmarks found.
+- [ ] Can shell wrapper interception be made reliable enough for production use, or does it introduce too many false positives?
+- [ ] What is the MCP tool-calling preference in Claude Code vs Cursor vs other agents when native bash/grep is also available? Empirical testing needed.
+- [ ] vgrep MCP integration roadmap — author says "planned" but no timeline.
+- [ ] What embedding model does ck download by default? The docs say fastembed but not which specific model.
+
+## Sources
+
+- [[ck-semantic-search]]: Official ck documentation (BeaconBay, 2025-2026)
+- [[vgrep-semantic-search]]: vgrep GitHub readme and docs (CortexLM, 2025-2026)
+- GitHub API: Repository metadata for all alternatives (April 2026)

package/vault/wiki/questions/Research: vcc extension for pi coding agent.md

@@ -0,0 +1,73 @@
+---
+type: synthesis
+title: "Research: vcc extension for pi coding agent"
+created: 2026-05-05
+updated: 2026-05-05
+tags:
+  - research
+  - pi-agent
+  - vscode
+  - vcc
+status: developing
+related:
+  - "[[pi-vscode-extension-landscape]]"
+  - "[[vcc-conversation-compaction-for-pi]]"
+  - "[[pi-coding-agent]]"
+  - "[[skill-first-architecture]]"
+sources:
+  - "[[pi-vscode-marketplace]]"
+  - "[[pi-vscode-model-provider-marketplace]]"
+  - "[[vscode-pi-community-extension]]"
+  - "[[pi-vcc-github-repo]]"
+---
+
+# Research: vcc extension for pi coding agent
+
+## Overview
+
+Topic "vcc extension for pi coding agent" is ambiguous. Research shows two valid interpretations: (1) VS Code extension(s) for Pi coding agent, and (2) literal VCC compaction extension (`pi-vcc`) for Pi sessions. Both exist. They solve different problems.
+
+## Key Findings
+
+1. **Official VS Code extension exists and is active** (Source: [[pi-vscode-marketplace]])
+   - `pi0.pi-vscode` is official, terminal-first, includes `@pi` chat participant and VS Code bridge tools.
+
+2. **Pi can also integrate as Language Model Provider** (Source: [[pi-vscode-model-provider-marketplace]])
+   - `tintinweb.vscode-pi-model-chat-provider` exposes Pi models through `vscode.lm.*` for Copilot Chat and LM API consumers.
+
+3. **Community extension offers full sidebar UX** (Source: [[vscode-pi-community-extension]])
+   - `cdervis.vscode-pi` is unofficial but feature-rich, RPC-driven, and rapidly updated.
+
+4. **Literal VCC means compaction package, not VS Code UI extension** (Source: [[pi-vcc-github-repo]])
+   - `pi-vcc` is deterministic session compaction + recall package for Pi CLI sessions.
+
+5. **Best answer depends on user intent** (Source: [[pi-vscode-extension-landscape]], [[vcc-conversation-compaction-for-pi]])
+   - If intent is IDE UX: choose among VS Code extensions.
+   - If intent is long-session memory/token control: use `pi-vcc`.
+
+## Key Entities
+
+- [[pi-coding-agent]]: Base platform
+- `pi0` / `tintinweb` / `Cem Dervis`: Extension publishers
+- `sting8k`: pi-vcc maintainer
+
+## Key Concepts
+
+- [[pi-vscode-extension-landscape]]
+- [[vcc-conversation-compaction-for-pi]]
+
+## Contradictions
+
+- "VCC" often used casually as "VS Code companion/client" in conversation, but in Pi ecosystem it is also specific acronym tied to conversation compiler approach. Both are used; naming collision causes confusion.
+
+## Open Questions
+
+- Should Pi ecosystem standardize naming to avoid VCC ambiguity (VS Code companion vs View-oriented compaction)?
+- Should official `pi0.pi-vscode` adopt native VCC/recall visual controls for parity with power-user workflows?
+
+## Sources
+
+- [[pi-vscode-marketplace]]
+- [[pi-vscode-model-provider-marketplace]]
+- [[vscode-pi-community-extension]]
+- [[pi-vcc-github-repo]]

package/vault/wiki/questions/how-to-enable-semantic-code-search-now.md

@@ -0,0 +1,111 @@
+---
+type: question
+title: "How to enable semantic code search now"
+question: "What can be done now to enable semantic code search and improve overall model performance?"
+answer_quality: solid
+created: 2026-05-04
+updated: 2026-05-04
+tags: [question, semantic-search, harness, implementation]
+related:
+  - "[[ck-tool]]"
+  - "[[agent-search-enforcement]]"
+  - "[[hybrid-code-search]]"
+  - "[[Semantic Codebase Indexing]]"
+  - "[[harness-implementation-plan]]"
+sources:
+  - "[[Research: semantic code search tools]]"
+  - "[[ck-semantic-search]]"
+  - "[[agent-search-enforcement]]"
+status: resolved
+---
+
+# How to enable semantic code search now
+
+## Problem
+
+AI coding agents waste context on `grep`/`find` because those tools are lexical-only, noisy, non-indexed, and token-inefficient. Every query scans the full codebase. Semantic searches like "error handling patterns" find zero matches with grep.
+
+## Solution: Three layers
+
+### Layer 1 (Immediate) — Install ck + system prompt rules
+
+**Tool**: ck (BeaconBay/ck, 1,572 ★, Rust, MIT, fully offline). Grep-compatible with `--sem` (semantic) and `--hybrid` (BM25 + embeddings via RRF) modes. MCP-native.
+
+```bash
+npm install -g @beaconbay/ck-search
+ck index .              # Build semantic index once
+ck --hybrid "query"     # Then search
+```
+
+System prompt rules: NEVER raw grep for exploration, ALWAYS `ck --hybrid` for conceptual searches.
+
+### Layer 2 (Medium-term) — Skill-based enforcement
+
+Create a skill that teaches the agent when/how to use ck. Trigger on codebase exploration tasks. Progressive disclosure: only loads when agent searches.
+
+### Layer 3 (Optional) — Pre-exec hook
+
+Intercept grep in lean-ctx bash tool, reroute conceptual queries to ck. Catches all grep invocations regardless of model compliance.
+
+## Impact
+
+| Metric | Before | After |
+|--------|--------|-------|
+| Search type | Lexical only | BM25 + semantic + RRF fusion |
+| Conceptual queries | 0% recall | ~80-90% recall |
+| Context waste | Raw grep output, full scan | Ranked, indexed, token-efficient |
+| Tool loading | Always | Progressive (only when needed) |
+
+## Validation
+
+Augment Code's Context Engine proves semantic indexing at scale: #1 SWE-bench Pro (51.80%), beating Cursor by 1.59% and Claude Code by 2.05% — same model, better context. (Source: [[Semantic Codebase Indexing]], [[Context Engine (AI Coding)]])
+
+## Implementation status (2026-05-04)
+
+### Layer 1 (Complete)
+- **ck**: Installed globally (`/home/aryaniyaps/.nvm/.../bin/ck`)
+- **Semantic index**: Built — 342 files, 1,593 chunks, 2.7MB (BAAI/bge-small-en-v1.5)
+- **Skill**: Created `.pi/skills/ck-search/SKILL.md` — decision tree, usage patterns, token efficiency
+- **System prompt**: CODEBASE SEARCH POLICY section in `SYSTEM.md` — NEVER raw grep, ALWAYS ck --hybrid
+- **Gitignore**: `.ck/` added (regenerable index)
+
+### Layer 2 (Complete)
+- **pi extension** `.pi/extensions/ck-enforce.ts` — overrides lean-ctx's `grep` tool
+  - Registers on `session_start` (loads AFTER lean-ctx to properly override)
+  - Detects conceptual patterns: multi-word + no regex chars → reroutes to `ck --hybrid --json`
+  - Literal/exact/regex patterns → pass through to native ripgrep via lean-ctx
+  - Status command: `/ck-enforce`
+- **File watcher** `.pi/scripts/ck-watch.sh` — auto-reindexes on source changes
+  - Node.js `fs.watch` (recursive, cross-platform)
+  - 1500ms debounce, excludes node_modules/.ck/.git/dist/build
+  - Verified: reindex in ~400ms on change
+  - Run: `.pi/scripts/ck-watch.sh &` for background mode
+- **Dependency**: `@sinclair/typebox` added as devDependency
+
+### Architecture
+```
+Model calls grep("error handling", pages/")
+  → ck-enforce intercepts (conceptual? yes)
+  → ck --hybrid "error handling" pages/ --json
+  → Returns semantic + lexical results, ranked by RRF
+
+Model calls grep("processPayment", src/)
+  → ck-enforce intercepts (conceptual? no — single word)
+  → lean-ctx rg --line-number "processPayment" src/
+  → Returns exact matches only
+```
+
+### File watcher
+```
+Files change (edit, save, git checkout)
+  → fs.watch detects change
+  → debounce 1500ms
+  → ck index . → 400ms reindex
+  → Index stays fresh
+```
+
+## Open questions
+
+- ck's embedding quality vs code-specific models (CodeBERT, UniXCoder) on real-world queries — no independent benchmarks
+- MCP tool preference when native bash/grep also available — empirical testing needed
+- Shell wrapper false-positive rate on production agent queries