tea-rags 1.19.0 → 1.20.0

package/README.md

@@ -1,70 +1,221 @@
 <p align="center">
   <a href="https://artk0de.github.io/TeaRAGs-MCP/">
-    <img src="public/logo.png">
+    <img src="public/logo.png" alt="TeaRAGs logo">
   </a>
 </p>
 
-<h1 align="center">TeaRAGs</h1>
+<h1 align="center">TeaRAGs 🦖🍵</h1>
 
 <p align="center">
   <strong>Trajectory Enrichment-Aware RAG for Coding Agents</strong>
 </p>
 
-![MCP compatible](https://img.shields.io/badge/MCP-compatible-%234f46e5)
-[![quickstart < 15 min](https://img.shields.io/badge/quickstart-%3C%2015%20min-f59e0b)](#-quick-start)
-[![local-first](https://img.shields.io/badge/deployment-local--first-15803d)](#-quick-start)
-[![reproducible: docker](https://img.shields.io/badge/reproducible-docker-0f172a)](#-quick-start)
-[![provider agnostic](https://img.shields.io/badge/provider-agnostic-0891b2)](#-quick-start)
-
-[![CI](https://github.com/artk0de/TeaRAGs-MCP/actions/workflows/ci.yml/badge.svg)](https://github.com/artk0de/TeaRAGs-MCP/actions/workflows/ci.yml)
-[![codecov](https://codecov.io/gh/artk0de/TeaRAGs-MCP/graph/badge.svg?token=BU255N03YF)](https://codecov.io/gh/artk0de/TeaRAGs-MCP)
+<p align="center">
+  <img src="https://img.shields.io/badge/MCP-compatible-%234f46e5" alt="MCP compatible">
+  <a href="https://artk0de.github.io/TeaRAGs-MCP/quickstart/installation"><img src="https://img.shields.io/badge/quickstart-%3C%2015%20min-f59e0b" alt="15-minute quickstart"></a>
+  <img src="https://img.shields.io/badge/deployment-local--first-15803d" alt="local-first">
+  <img src="https://img.shields.io/badge/provider-agnostic-0891b2" alt="provider agnostic">
+  <br>
+  <a href="https://github.com/artk0de/TeaRAGs-MCP/actions/workflows/ci.yml"><img src="https://github.com/artk0de/TeaRAGs-MCP/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
+  <a href="https://codecov.io/gh/artk0de/TeaRAGs-MCP"><img src="https://codecov.io/gh/artk0de/TeaRAGs-MCP/graph/badge.svg?token=BU255N03YF" alt="codecov"></a>
+</p>
 
 ---
 
-**MCP server** for semantic code search with **git trajectory reranking**. AST-aware chunking, incremental indexing, millions of LOC. Reranks results using authorship, churn, bug-fix rates, and 19 other signals — not just embedding similarity. Built on Qdrant. Works with Ollama (local) or cloud providers (OpenAI, Cohere, Voyage).
+**Your coding agent copies the first code it finds — not the right one.**
+
+TeaRAGs is an MCP server for code search that enriches every retrieved chunk
+with git history: authorship, churn, bug-fix rate, ownership. Your agent stops
+learning from hotspots and starts learning from **stable, owned, battle-tested
+code**.
+
+📖 **[Full documentation](https://artk0de.github.io/TeaRAGs-MCP/)** · 🏁
+**[15-minute quickstart](https://artk0de.github.io/TeaRAGs-MCP/quickstart/installation)**
+· 🧠
+**[Core concepts](https://artk0de.github.io/TeaRAGs-MCP/introduction/core-concepts)**
+
+## The Problem
+
+### 1. Understanding a monorepo is expensive — for humans AND agents
+
+Every new developer pays in hours. Every fresh agent session pays in tokens.
+Naming conventions, domain logic, local idioms — all of it has to be rebuilt
+from scratch, every time.
+
+### 2. Bad code hygiene is a tax on your agent
+
+Confusing names mean the agent reads more files. More files mean more tokens,
+slower responses, and a higher chance of picking the wrong example. Your
+codebase's technical debt is now your AI bill.
+
+### 3. Agents can't tell stable code from a hotspot
+
+Standard code search ranks by embedding similarity alone. It doesn't know which
+function gets bug-fixed every sprint, which module hasn't been touched in two
+years, or whose name is on the commits. So the agent copies whatever looks
+similar — including the broken examples.
+
+## The Solution
+
+TeaRAGs gives your agent two things it can't get from vanilla code search.
+
+### 1. Every chunk carries its own history
+
+Retrieved code comes with signals about **who wrote it, how stable it is, how
+often it gets bug-fixed**, and **how impactful a change would be**. Semantic
+similarity stops being the whole answer — it becomes the floor.
+
+### 2. Pre-built skills, not just raw tools
+
+TeaRAGs ships agent **skills** — ready-made playbooks that tell your agent when
+and how to use the signals. No prompt engineering required:
+
+- `explore` — orient in an unfamiliar codebase
+- `data-driven-generation` — write code backed by stable, owned templates
+- `risk-assessment` — know what you'd break before you break it
+- `refactoring-scan` · `bug-hunt` · `pattern-search` — and more
+
+Install the plugin, your agent learns the workflow.
+[See all skills →](https://artk0de.github.io/TeaRAGs-MCP/usage/skills/)
+
+**Bonus: `dinopowers`** — a companion plugin with 10 wrappers over
+[`superpowers:*`](https://github.com/obra/superpowers) skills (Jesse Vincent's
+skills library for Claude Code) that inject tea-rags signals into brainstorming,
+planning, debugging, TDD, review, and completion flows. Mean eval delta +71pp
+across 136 cases.
+[Learn more →](https://artk0de.github.io/TeaRAGs-MCP/usage/skills/#dinopowers--wrappers-over-superpowers)
+
+## Use Cases
 
-> 📖 **[Full documentation](https://artk0de.github.io/TeaRAGs-MCP/)** — 15-minute quickstart, agent workflows, architecture deep dives.
+### 🛡️ Safe code generation
 
-## 🧬 Trajectory Enrichment
+Your agent writes new code backed by **stable, canonical templates** — modules
+with a low bug-fix rate, long stability, and a clear owner. No more copying from
+last sprint's hotspot. _Skill: `data-driven-generation` ·
+[Why stable code is safer →](https://artk0de.github.io/TeaRAGs-MCP/knowledge-base/code-churn-research)_
 
-Standard code RAG retrieves by similarity alone. **Trajectory enrichment** augments each chunk with signals about how code *evolves* — at the function level, not just file level.
+### 🔧 Refactoring planning & problem-pattern discovery
 
-- 🔀 **Git trajectory** — churn, authorship, volatility, bug-fix rates, task traceability. **19 signals** feed composable rerank presets (`hotspots`, `ownership`, `techDebt`, `securityAudit`...)
-- 🕸️ **Topological trajectory** *(planned)* — symbol graphs, cross-file coupling, blast radius
+Find the 5% of code responsible for 80% of incidents. **High churn + high
+bug-fix rate + concentrated ownership = your next production issue** — and your
+next refactoring candidate. _Skills: `refactoring-scan`, `bug-hunt`_
 
-Opt-in via `CODE_ENABLE_GIT_METADATA=true`. Without it — standard semantic search with AST-aware chunking.
+### 🎯 Risk assessment before changes
 
-> 💡 An agent can **find stable templates**, **avoid anti-patterns**, **match domain owner's style**, and **assess modification risk** — all backed by empirical data. [Read more →](https://artk0de.github.io/TeaRAGs-MCP/introduction/core-concepts)
+Before modifying a function, the agent checks **who depends on it, how often it
+breaks, and what its ticket history says**. Know the blast radius before you
+blast. _Skill: `risk-assessment` ·
+[Coupling & blast radius theory →](https://artk0de.github.io/TeaRAGs-MCP/knowledge-base/code-quality-metrics)_
+
+### 🗺️ Learning an unfamiliar codebase
+
+Ask questions instead of reading directory trees. _"How does auth work?"_
+returns the **stable, canonical implementation** with its history attached — not
+a random similar-looking snippet. _Skill: `explore`_
+
+## How It Works
+
+```mermaid
+flowchart LR
+    User([👤 You])
+
+    subgraph mcp["TeaRAGs MCP Server"]
+        Agent[🤖 Agent<br/>runs skills]
+        TeaRAGs[🍵 TeaRAGs<br/>search · enrich · rerank]
+        Agent <--> TeaRAGs
+    end
+
+    Qdrant[(🗄️ Qdrant<br/>vector DB)]
+    Embeddings[✨ Embeddings<br/>Ollama/OpenAI]
+    Codebase[📁 Your Codebase<br/>+ Git History]
+
+    User <--> Agent
+    TeaRAGs <--> Qdrant
+    TeaRAGs <--> Embeddings
+    TeaRAGs <--> Codebase
+```
+
+You talk to your agent. The agent runs a TeaRAGs skill. TeaRAGs searches your
+code, enriches each result with git history, and ranks by what the skill needs —
+stability, ownership, risk, or pure relevance.
+
+## What You Get
+
+- 🧬 **Trajectory-aware retrieval** — the only open-source code RAG that scores
+  results by git history, not just embedding similarity
+- 📚 **Ships with agent skills** — 6 ready-made playbooks for exploration,
+  generation, risk assessment, and index management (plus 2 internal strategies)
+- 🔒 **Local-first, privacy-first** — works fully offline with Ollama; your code
+  never leaves your machine (cloud providers optional)
+- 🚀 **Built for monorepos** — AST-aware chunking across 10+ languages,
+  incremental reindexing, parallel pipelines, millions of LOC tested
+
+## Who It's For
+
+- **Developers in large monorepos** — where "find similar code" returns a dozen
+  near-duplicates and you need the _canonical_ one
+- **Solo devs doing agentic development** — agent-driven workflows produce
+  bursts of micro-commits that wreck churn metrics. TeaRAGs ships a
+  [**GIT SESSIONS**](https://artk0de.github.io/TeaRAGs-MCP/architecture/git-enrichment-pipeline#git-sessions)
+  mode (`TRAJECTORY_GIT_SQUASH_AWARE_SESSIONS=true`) that groups commits by
+  `(author, time gap)` so a 20-commit refactor session counts as **one**. Churn,
+  bug-fix rate, and ownership stay meaningful even with a single human + an
+  agent as the only contributors.
+- **Tech leads worried about AI code quality** — who want their team's agents to
+  learn from stable modules, not from last sprint's hotspot
+- **Privacy-sensitive teams** — finance, healthcare, defense, or anyone who
+  can't send source code to a cloud API
+
+**Not for:** repos without git history (no signal to enrich) or teams that only
+need autocomplete (use Copilot).
 
 ## 🚀 Quick Start
 
-```bash
-git clone https://github.com/artk0de/TeaRAGs-MCP.git
-cd TeaRAGs-MCP
-npm install && npm run build
+Inside **Claude Code**, install the TeaRAGs plugins and run the setup wizard:
+
+```
+/plugin marketplace add artk0de/TeaRAGs-MCP
+/plugin install tea-rags-setup@tea-rags
+/tea-rags-setup:install
+```
+
+Then install the skills plugin (Claude-only, final step):
+
+```
+/plugin install tea-rags@tea-rags
+```
 
-# Start Qdrant + Ollama
-podman compose up -d
-podman exec ollama ollama pull unclemusclez/jina-embeddings-v2-base-code:latest
+Optionally install `dinopowers` for wrappers over `superpowers:*` skills:
 
-# Add to Claude Code
-claude mcp add tea-rags -s user -- node /path/to/tea-rags/build/index.js \
-  -e QDRANT_URL=http://localhost:6333 \
-  -e EMBEDDING_BASE_URL=http://localhost:11434
 ```
+/plugin install dinopowers@tea-rags
+```
+
+Index your codebase:
+
+```
+/tea-rags:index
+```
+
+Ask your agent anything: _"How does auth work in this project?"_, _"Find stable
+examples of retry logic"_, _"What should I know before touching the payment
+module?"_.
 
-Then ask your agent: *"Index this codebase for semantic search"*
+For other MCP clients, CI, or air-gapped setups, see the
+[manual install](https://artk0de.github.io/TeaRAGs-MCP/quickstart/installation#option-b--manual-install)
+(Node + `npm install -g tea-rags` + Ollama/ONNX/OpenAI/Cohere/Voyage).
 
 ## 📚 Documentation
 
 **[artk0de.github.io/TeaRAGs-MCP](https://artk0de.github.io/TeaRAGs-MCP/)**
 
-| | Section | What's inside |
-|---|---------|---------------|
-| 🏁 | [Quickstart](https://artk0de.github.io/TeaRAGs-MCP/quickstart/installation) | Installation, first index & query |
-| ⚙️ | [Configuration](https://artk0de.github.io/TeaRAGs-MCP/usage/configuration) | Env vars, providers, tuning |
-| 🤖 | [Agent Integration](https://artk0de.github.io/TeaRAGs-MCP/agent-integration/search-strategies) | Prompt strategies, generation modes, deep analysis |
-| 🏗️ | [Architecture](https://artk0de.github.io/TeaRAGs-MCP/architecture/overview) | Pipeline, data model, reranker internals |
+| I want to…                   | Start here                                                                                                                          |
+| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
+| **Get it running**           | [Quickstart (15 min)](https://artk0de.github.io/TeaRAGs-MCP/quickstart/installation) — install, index, first query                  |
+| **Understand the concept**   | [Core Concepts](https://artk0de.github.io/TeaRAGs-MCP/introduction/core-concepts) — vectorization, trajectory enrichment, reranking |
+| **See what my agent can do** | [Skills](https://artk0de.github.io/TeaRAGs-MCP/usage/skills/) — 6 ready-made agent playbooks for exploration, generation, risk      |
+| **Look under the hood**      | [Architecture](https://artk0de.github.io/TeaRAGs-MCP/architecture/overview) — pipelines, data model, reranker internals             |
+| **Learn the theory**         | [Knowledge Base](https://artk0de.github.io/TeaRAGs-MCP/knowledge-base/rag-fundamentals) — RAG, code search, software evolution      |
 
 ## 🤝 Contributing
 
@@ -72,7 +223,12 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for workflow and conventions.
 
 ## 🙏 Acknowledgments
 
-Built on a fork of **[mhalder/qdrant-mcp-server](https://github.com/mhalder/qdrant-mcp-server)** — clean architecture, solid tests, open-source spirit. And its ancestor **[qdrant/mcp-server-qdrant](https://github.com/qdrant/mcp-server-qdrant)**. Code vectorization inspired by **[claude-context](https://github.com/zilliztech/claude-context)** (Zilliz).
+Built on a fork of
+**[mhalder/qdrant-mcp-server](https://github.com/mhalder/qdrant-mcp-server)** —
+clean architecture, solid tests, open-source spirit. And its ancestor
+**[qdrant/mcp-server-qdrant](https://github.com/qdrant/mcp-server-qdrant)**.
+Code vectorization inspired by
+**[claude-context](https://github.com/zilliztech/claude-context)** (Zilliz).
 
 _Feel free to fork this fork. It's forks all the way down._ 🐢
 

package/benchmarks/benchmark-embeddings.mjs

@@ -0,0 +1,148 @@
+#!/usr/bin/env node
+/**
+ * Embedding Diagnostic Benchmark
+ *
+ * Automatically calibrates EMBEDDING_BATCH_SIZE and EMBEDDING_CONCURRENCY
+ * using a three-phase plateau-detection algorithm.
+ *
+ * Phase 1: Find batch size plateau (CONCURRENCY=1)
+ * Phase 2: Test concurrency on plateau batches
+ * Phase 3: Select robust configuration (within 2% of max, prefer lower concurrency/batch)
+ *
+ * Run: npm run benchmark-embeddings
+ */
+import { c, printBox } from "./lib/colors.mjs";
+import { AVG_LOC_PER_CHUNK, config, MEDIAN_CODE_CHUNK_SIZE } from "./lib/config.mjs";
+import { calibrateEmbeddings } from "./lib/embedding-calibration.mjs";
+import { checkProviderConnectivity, createEmbeddingProvider } from "./lib/provider.mjs";
+
+/**
+ * Format time in human readable format
+ */
+function formatTime(ms) {
+  if (ms < 1000) return `${ms}ms`;
+  if (ms < 60000) return `${(ms / 1000).toFixed(1)}s`;
+  const minutes = Math.floor(ms / 60000);
+  const seconds = Math.round((ms % 60000) / 1000);
+  return `${minutes}m ${seconds}s`;
+}
+
+async function main() {
+  console.clear();
+  printBox("EMBEDDING CALIBRATION BENCHMARK", "Three-phase plateau detection");
+
+  // Show configuration
+  console.log(`${c.bold}Configuration:${c.reset}`);
+  console.log(`  ${c.dim}Ollama:${c.reset}        ${config.EMBEDDING_BASE_URL}`);
+  console.log(`  ${c.dim}Model:${c.reset}         ${config.EMBEDDING_MODEL}`);
+  console.log(`  ${c.dim}Chunk size:${c.reset}    ${MEDIAN_CODE_CHUNK_SIZE} chars (median from production)`);
+  console.log();
+
+  // Check embedding provider
+  process.stdout.write(`${c.dim}Checking embedding provider...${c.reset} `);
+  const embeddingCheck = await checkProviderConnectivity();
+  if (!embeddingCheck.ok) {
+    console.log(`${c.red}FAILED${c.reset}`);
+    console.log(`\n${c.red}Error:${c.reset} ${embeddingCheck.error}`);
+    process.exit(1);
+  }
+  console.log(`${c.green}OK${c.reset}`);
+
+  // Initialize embeddings
+  const { provider: embeddings, name: providerName } = await createEmbeddingProvider();
+  console.log(`  ${c.green}✓${c.reset} Embedding provider: ${providerName}`);
+  console.log(`  ${c.green}✓${c.reset} Vector dimension: ${embeddings.getDimensions()}`);
+  console.log();
+
+  // Run calibration
+  const result = await calibrateEmbeddings(embeddings, { verbose: true });
+
+  // ========== OUTPUT ==========
+  console.log();
+
+  // Detect setup type
+  const isOnnx = providerName === "onnx";
+  const isRemote =
+    !isOnnx && !config.EMBEDDING_BASE_URL.includes("localhost") && !config.EMBEDDING_BASE_URL.includes("127.0.0.1");
+  const setupIcon = isOnnx ? "⚡" : isRemote ? "🌐" : "🏠";
+  const setupName = isOnnx ? "Local ONNX" : isRemote ? "Remote GPU" : "Local GPU";
+
+  printBox(`${setupIcon} ${setupName.toUpperCase()} - OPTIMAL CONFIGURATION`, "");
+
+  // Main result
+  console.log(
+    `  ${c.bold}EMBEDDING_BATCH_SIZE${c.reset}   = ${c.green}${c.bold}${result.EMBEDDING_BATCH_SIZE}${c.reset}`,
+  );
+  console.log(
+    `  ${c.bold}EMBEDDING_CONCURRENCY${c.reset}  = ${c.green}${c.bold}${result.EMBEDDING_CONCURRENCY}${c.reset}`,
+  );
+  console.log();
+  console.log(`  ${c.bold}Throughput:${c.reset} ${c.cyan}${result.throughput_chunks_per_sec} chunks/s${c.reset}`);
+  console.log();
+
+  // Explain the choice
+  console.log(`${c.bold}Why this configuration?${c.reset}`);
+  if (isOnnx) {
+    console.log(`  ${c.dim}•${c.reset} Local ONNX runtime (${providerName})`);
+    console.log(`  ${c.dim}•${c.reset} In-process inference, no network overhead`);
+    if (result.EMBEDDING_BATCH_SIZE <= 16) {
+      console.log(`  ${c.dim}•${c.reset} Small batches optimal for ONNX memory management`);
+    }
+  } else if (isRemote) {
+    console.log(`  ${c.dim}•${c.reset} Remote GPU detected (${config.EMBEDDING_BASE_URL})`);
+    console.log(`  ${c.dim}•${c.reset} Lower batch + higher concurrency hides network latency`);
+    console.log(`  ${c.dim}•${c.reset} While one batch transfers, GPU processes another`);
+    if (result.EMBEDDING_CONCURRENCY > 1) {
+      console.log(
+        `  ${c.dim}•${c.reset} CONCURRENCY=${result.EMBEDDING_CONCURRENCY} overlaps network I/O with GPU compute`,
+      );
+    }
+  } else {
+    console.log(`  ${c.dim}•${c.reset} Local GPU detected (minimal network latency)`);
+    console.log(`  ${c.dim}•${c.reset} Higher batch + lower concurrency minimizes overhead`);
+    if (result.EMBEDDING_CONCURRENCY === 1) {
+      console.log(`  ${c.dim}•${c.reset} CONCURRENCY=1 indicates GPU-bound workload`);
+    }
+  }
+  console.log();
+
+  // Environment export
+  console.log(`${c.bold}Add to your environment:${c.reset}`);
+  console.log();
+  console.log(`  ${c.cyan}export EMBEDDING_BATCH_SIZE=${result.EMBEDDING_BATCH_SIZE}${c.reset}`);
+  console.log(`  ${c.cyan}export EMBEDDING_CONCURRENCY=${result.EMBEDDING_CONCURRENCY}${c.reset}`);
+  console.log();
+
+  // Time estimates
+  console.log(`${c.bold}Estimated indexing times:${c.reset}`);
+  const projects = [
+    { name: "10K LoC", loc: 10_000 },
+    { name: "100K LoC", loc: 100_000 },
+    { name: "1M LoC", loc: 1_000_000 },
+    { name: "VS Code (3.5M)", loc: 3_500_000 },
+  ];
+  for (const p of projects) {
+    const chunks = Math.ceil(p.loc / AVG_LOC_PER_CHUNK);
+    const seconds = Math.ceil(chunks / result.throughput_chunks_per_sec);
+    console.log(`  ${c.dim}${p.name.padEnd(20)}${c.reset} ${c.bold}${formatTime(seconds * 1000)}${c.reset}`);
+  }
+  console.log();
+
+  // Stats
+  console.log(`${c.dim}────────────────────────────────────────${c.reset}`);
+  console.log(
+    `${c.dim}Configs tested: ${result.stable_configs_count} stable, ${result.discarded_configs_count} discarded${c.reset}`,
+  );
+  console.log(`${c.bold}Total benchmark time: ${formatTime(result.calibration_time_ms)}${c.reset}`);
+
+  // Terminate provider (ONNX keeps socket alive)
+  if ("terminate" in embeddings && typeof embeddings.terminate === "function") {
+    await embeddings.terminate();
+  }
+}
+
+main().catch((err) => {
+  console.error(`${c.red}Fatal error:${c.reset}`, err.message);
+  console.error(err.stack);
+  process.exit(1);
+});