smart-context-mcp 1.1.0 → 1.3.0

package/README.md

@@ -7,16 +7,70 @@ MCP server that reduces AI agent token usage by 90% with intelligent context com
 
 ## Installation
 
+### Cursor
 ```bash
-npm install smart-context-mcp
+npm install -g smart-context-mcp
+npx smart-context-init --target . --clients cursor
+```
+Restart Cursor. Done.
+
+### Codex CLI
+```bash
+npm install -g smart-context-mcp
+npx smart-context-init --target . --clients codex
+```
+Restart Codex. Done.
+
+### Claude Desktop
+```bash
+npm install -g smart-context-mcp
+npx smart-context-init --target . --clients claude
+```
+Restart Claude Desktop. Done.
+
+### Qwen Code
+```bash
+npm install -g smart-context-mcp
+npx smart-context-init --target . --clients qwen
+```
+Restart Qwen Code. Done.
+
+### All Clients
+```bash
+npm install -g smart-context-mcp
 npx smart-context-init --target .
 ```
+Restart your AI client. Done.
+
+## How it Works in Practice
+
+**The reality:** This MCP does not intercept prompts automatically. Here's the actual flow:
 
-Restart your AI client. Tools are immediately available.
+1. **You:** "Fix the login bug"
+2. **Agent reads rules:** Sees debugging workflow
+3. **Agent decides:** Uses `smart_search(intent=debug)`
+4. **MCP returns:** Ranked results (errors prioritized)
+5. **Agent continues:** Calls `smart_read(symbol)` for function
+6. **Agent fixes:** Makes changes
+7. **Agent verifies:** Calls `smart_shell('npm test')`
+8. **Agent checkpoints:** Calls `smart_turn(end)`
+
+**Key points:**
+- ✅ Agent **chooses** to use devctx tools (not forced)
+- ✅ Rules **guide** the agent (not enforce)
+- ✅ Agent can use built-in tools when appropriate
+- ✅ Token savings: 85-90% on complex tasks
+
+Check actual usage:
+- **Real-time feedback** - See usage immediately (enable with `export DEVCTX_SHOW_USAGE=true`)
+- `npm run report:metrics` - Tool-level savings + adoption analysis
+- `npm run report:workflows` - Workflow-level savings (requires `DEVCTX_WORKFLOW_TRACKING=true`)
 
 ## What it does
 
-Replaces inefficient file reading and searching with 12 specialized tools:
+Provides **two key components**:
+
+### 1. Specialized Tools (12 tools)
 
 | Tool | Purpose | Savings |
 |------|---------|---------|
@@ -24,8 +78,8 @@ Replaces inefficient file reading and searching with 12 specialized tools:
 | `smart_read_batch` | Read multiple files in one call | 90% |
 | `smart_search` | Intent-aware code search with ranking | 95% |
 | `smart_context` | One-call context builder | 85% |
-| `smart_summary` | Session state management | 98% |
-| `smart_turn` | Turn orchestration | - |
+| `smart_summary` | Task checkpoint management | 98% |
+| `smart_turn` | Task recovery orchestration | - |
 | `smart_metrics` | Token usage inspection | - |
 | `smart_shell` | Safe command execution | 94% |
 | `build_index` | Symbol index builder | - |
@@ -33,10 +87,150 @@ Replaces inefficient file reading and searching with 12 specialized tools:
 | `git_blame` | Function-level code attribution | - |
 | `cross_project` | Multi-project context | - |
 
+### 2. Agent Rules (Task-Specific Guidance)
+
+Installation generates rules that teach agents optimal workflows:
+
+**Debugging:** `smart_search(intent=debug)` → `smart_read(symbol)` → fix (90% savings)  
+**Code Review:** `smart_context(diff=true)` → `smart_read(signatures)` → review (87% savings)  
+**Refactoring:** `smart_context(entryFile)` → `smart_read(signatures)` → refactor (89% savings)  
+**Testing:** `smart_search(intent=tests)` → `smart_read(symbol)` → write test (90% savings)  
+**Architecture:** `smart_context(detail=minimal)` → `smart_read(signatures)` → analyze (90% savings)
+
+**Key insight:** The value isn't just in the tools—it's in teaching agents **when** and **how** to use them.
+
 ## Real Metrics
 
 Production usage: **14.5M tokens → 1.6M tokens** (89.87% reduction)
 
+## Verify It's Working
+
+### Real-Time Feedback (Auto-enabled for First 10 Calls)
+
+Feedback is **automatically enabled** for your first 10 tool calls (onboarding mode), then auto-disables.
+
+**To keep it enabled permanently:**
+```bash
+export DEVCTX_SHOW_USAGE=true
+```
+
+**To disable immediately:**
+```bash
+export DEVCTX_SHOW_USAGE=false
+```
+
+You'll see at the end of agent responses:
+
+```markdown
+---
+
+📊 **devctx usage this session:**
+- **smart_read**: 3 calls | ~45.0K tokens saved (file1.js, file2.js, file3.js)
+- **smart_search**: 1 call | ~12.0K tokens saved (query)
+
+**Total saved:** ~57.0K tokens
+
+*Onboarding mode: showing for 3 more tool calls. To keep: `export DEVCTX_SHOW_USAGE=true`*
+```
+
+**Why this is useful:**
+- ✅ Verify agent is following rules
+- ✅ See token savings in real-time
+- ✅ Debug adoption issues instantly
+- ✅ Validate forcing prompts worked
+
+### Historical Metrics
+
+```bash
+npm run report:metrics
+```
+
+Shows adoption analysis + token savings over time.
+
+### Decision Explanations (Optional)
+
+Understand **why** the agent chose devctx tools:
+
+```bash
+export DEVCTX_EXPLAIN=true
+```
+
+You'll see explanations like:
+
+```markdown
+🤖 **Decision explanations:**
+
+**smart_read** (read server.js (outline mode))
+- **Why:** File is large (2500 lines), outline mode extracts structure only
+- **Instead of:** Read (full file)
+- **Expected benefit:** ~45.0K tokens saved
+
+**smart_search** (search "bug" (intent: debug))
+- **Why:** Intent-aware search prioritizes relevant results
+- **Expected benefit:** Better result ranking
+```
+
+**When to use:**
+- Learning how devctx works
+- Debugging tool selection
+- Understanding best practices
+
+### Missed Opportunities Detection (Optional)
+
+Detect when devctx **should have been used but wasn't**:
+
+```bash
+export DEVCTX_DETECT_MISSED=true
+```
+
+You'll see warnings like:
+
+```markdown
+⚠️ **Missed devctx opportunities detected:**
+
+**Session stats:**
+- devctx operations: 2
+- Estimated total: 25
+- Adoption: 8%
+
+🟡 **low devctx adoption**
+- **Issue:** Low adoption (8%). Target: >50%
+- **Potential savings:** ~184.0K tokens
+```
+
+**Detects:**
+- No devctx usage in long sessions
+- Low adoption (<30%)
+- Usage dropped mid-session
+
+**Combine all features:**
+
+```bash
+export DEVCTX_SHOW_USAGE=true    # See what's used
+export DEVCTX_EXPLAIN=true       # Understand why
+export DEVCTX_DETECT_MISSED=true # Detect gaps
+```
+
+## MCP Prompts
+
+The MCP server provides **prompts** for automatic forcing:
+
+```
+/prompt use-devctx
+```
+
+**Available prompts:**
+- `use-devctx` - Ultra-short forcing prompt
+- `devctx-workflow` - Complete workflow template
+- `devctx-preflight` - Preflight checklist
+
+**Benefits:**
+- No manual typing
+- Centrally managed
+- No typos
+
+See [MCP Prompts Documentation](../../docs/mcp-prompts.md).
+
 ## Core Tools
 
 ### smart_read
@@ -80,16 +274,18 @@ Returns: relevant files + compressed content + symbol details + graph relationsh
 
 ### smart_summary
 
-Maintain session state:
+Maintain task checkpoint:
 
 ```javascript
-// Start
-{ action: 'update', update: { goal: 'Implement OAuth', status: 'in_progress' }}
+// Save checkpoint
+{ action: 'update', update: { goal: 'Implement OAuth', status: 'in_progress', nextStep: '...' }}
 
-// Resume
+// Resume task
 { action: 'get' }
 ```
 
+Stores compressed task state (~100 tokens: goal, status, decisions, blockers), not full conversation.
+
 ## New Features
 
 ### Diff-Aware Context
@@ -189,8 +385,8 @@ npm run verify
 
 Data stored in `.devctx/`:
 - `index.json` - Symbol index
-- `state.sqlite` - Sessions, metrics, patterns
-- `metrics.jsonl` - Legacy fallback
+- `state.sqlite` - Task checkpoints, metrics, patterns (Node 22+)
+- `metrics.jsonl` - Legacy fallback (Node 18-20)
 
 Add to `.gitignore`:
 ```
@@ -202,22 +398,45 @@ Add to `.gitignore`:
 - Node.js 18+ (22+ for SQLite features)
 - Git (for diff and blame features)
 
+## Security
+
+This MCP is **secure by default**:
+
+- ✅ **Allowlist-only commands** - Only safe diagnostic commands (`ls`, `git status`, `npm test`, etc.)
+- ✅ **No shell operators** - Blocks `|`, `&`, `;`, `>`, `<`, `` ` ``, `$()`
+- ✅ **Path validation** - Cannot escape project root
+- ✅ **No write access** - Cannot modify your code
+- ✅ **Repository safety** - Prevents accidental commit of local state
+- ✅ **Resource limits** - 15s timeout, 10MB buffer
+
+**Configuration:**
+
+```bash
+# Disable shell execution entirely
+export DEVCTX_SHELL_DISABLED=true
+
+# Disable cache warming
+export DEVCTX_CACHE_WARMING=false
+```
+
+See [SECURITY.md](https://github.com/Arrayo/smart-context-mcp/blob/main/SECURITY.md) for complete security documentation.
+
 ## Documentation
 
-Full documentation in [GitHub repository](https://github.com/Arrayo/devctx-mcp-mvp):
+Full documentation in [GitHub repository](https://github.com/Arrayo/smart-context-mcp):
 
-- [STREAMING.md](https://github.com/Arrayo/devctx-mcp-mvp/blob/main/STREAMING.md) - Progress notifications
-- [CONTEXT-PREDICTION.md](https://github.com/Arrayo/devctx-mcp-mvp/blob/main/CONTEXT-PREDICTION.md) - File prediction
-- [DIFF-AWARE.md](https://github.com/Arrayo/devctx-mcp-mvp/blob/main/DIFF-AWARE.md) - Change analysis
-- [CACHE-WARMING.md](https://github.com/Arrayo/devctx-mcp-mvp/blob/main/CACHE-WARMING.md) - Cold-start optimization
-- [GIT-BLAME.md](https://github.com/Arrayo/devctx-mcp-mvp/blob/main/GIT-BLAME.md) - Code attribution
-- [CROSS-PROJECT.md](https://github.com/Arrayo/devctx-mcp-mvp/blob/main/CROSS-PROJECT.md) - Multi-project support
+- [Streaming Progress](https://github.com/Arrayo/smart-context-mcp/blob/main/docs/features/streaming.md) - Progress notifications
+- [Context Prediction](https://github.com/Arrayo/smart-context-mcp/blob/main/docs/features/context-prediction.md) - File prediction
+- [Diff-Aware Context](https://github.com/Arrayo/smart-context-mcp/blob/main/docs/features/diff-aware.md) - Change analysis
+- [Cache Warming](https://github.com/Arrayo/smart-context-mcp/blob/main/docs/features/cache-warming.md) - Cold-start optimization
+- [Git Blame](https://github.com/Arrayo/smart-context-mcp/blob/main/docs/features/git-blame.md) - Code attribution
+- [Cross-Project Context](https://github.com/Arrayo/smart-context-mcp/blob/main/docs/features/cross-project.md) - Multi-project support
 
 ## Links
 
-- [GitHub](https://github.com/Arrayo/devctx-mcp-mvp)
+- [GitHub](https://github.com/Arrayo/smart-context-mcp)
 - [npm](https://www.npmjs.com/package/smart-context-mcp)
-- [Issues](https://github.com/Arrayo/devctx-mcp-mvp/issues)
+- [Issues](https://github.com/Arrayo/smart-context-mcp/issues)
 
 ## Author
 

package/package.json

@@ -1,16 +1,16 @@
 {
   "name": "smart-context-mcp",
-  "version": "1.1.0",
-  "description": "MCP server that reduces agent token usage and improves response quality with compact file summaries, ranked code search, and curated context.",
+  "version": "1.3.0",
+  "description": "MCP server that reduces agent token usage by 90% with intelligent context compression, task checkpoint persistence, and workflow-aware agent guidance.",
   "author": "Francisco Caballero Portero <fcp1978@hotmail.com>",
   "type": "module",
   "repository": {
     "type": "git",
-    "url": "git+https://github.com/Arrayo/devctx-mcp-mvp.git"
+    "url": "git+https://github.com/Arrayo/smart-context-mcp.git"
   },
-  "homepage": "https://github.com/Arrayo/devctx-mcp-mvp#readme",
+  "homepage": "https://github.com/Arrayo/smart-context-mcp#readme",
   "bugs": {
-    "url": "https://github.com/Arrayo/devctx-mcp-mvp/issues"
+    "url": "https://github.com/Arrayo/smart-context-mcp/issues"
   },
   "bin": {
     "smart-context-headless": "scripts/headless-wrapper.js",
@@ -30,7 +30,8 @@
     "scripts/devctx-server.js",
     "scripts/headless-wrapper.js",
     "scripts/init-clients.js",
-    "scripts/report-metrics.js"
+    "scripts/report-metrics.js",
+    "scripts/report-workflow-metrics.js"
   ],
   "engines": {
     "node": ">=18"
@@ -52,12 +53,14 @@
     "smoke:formats": "node ./scripts/format-smoke.js",
     "test": "node --test --test-concurrency=1 ./tests/*.test.js",
     "verify": "node ./scripts/verify-features-direct.js",
+    "benchmark": "node ./scripts/run-benchmark.js",
     "eval": "node ./evals/harness.js",
     "eval:context": "node ./evals/harness.js --tool=context",
     "eval:both": "node ./evals/harness.js --tool=both",
     "eval:self": "node ./evals/harness.js --root=../.. --corpus=./evals/corpus/self-tasks.json",
     "eval:report": "node ./evals/report.js",
-    "report:metrics": "node ./scripts/report-metrics.js"
+    "report:metrics": "node ./scripts/report-metrics.js",
+    "report:workflows": "node ./scripts/report-workflow-metrics.js"
   },
   "dependencies": {
     "@modelcontextprotocol/sdk": "^1.13.0",

package/scripts/init-clients.js

@@ -375,42 +375,71 @@ const updatePreCommitHook = (targetDir, dryRun) => {
 // Agent rules — instruct agents to prefer devctx tools over built-in ones
 // ---------------------------------------------------------------------------
 
-const agentRuleBody = `Always prefer devctx MCP tools over built-in file reading, code search, diagnostics, and context recovery.
-
-- For every non-trivial user prompt, start with smart_turn phase=start before exploring the codebase. Pass the current user prompt and ensureSession=true for a meaningful new task.
-- Treat smart_turn phase=start as the default entrypoint for context recovery instead of manually deciding whether to call smart_summary(get).
-- Inspect repoSafety from smart_turn or smart_summary responses. If .devctx/state.sqlite is tracked or staged, mutating context writes are blocked until git hygiene is fixed.
-- If smart_turn phase=start reports ambiguous_resume, choose the recommended session explicitly with smart_summary sessionId="auto" before continuing.
-- After meaningful milestones, end the turn with smart_turn phase=end and the appropriate event (milestone, decision, blocker, task_complete, session_end) so persistence stays event-driven.
-- Before ending work, persist the latest nextStep with smart_turn phase=end and event=session_end.
-- Use smart_read outline or signatures before full (~90% token savings).
-- Use smart_read symbol (string or array) to extract specific functions/classes before editing.
-- Use smart_read range for specific lines when you know the location.
-- Use full mode only when outline/signatures/symbol are insufficient.
-- Use smart_search instead of grep/ripgrep — it groups, ranks, and filters automatically.
-- Pass intent to smart_search to get task-aware ranking (implementation/debug/tests/config/docs/explore).
-- Use smart_shell for diagnostics: git status, ls, find, pwd, test output.
-- Use smart_metrics to report token savings or recent devctx usage instead of reading metrics files manually.
-
-By task:
-- Debugging: smart_turn(start) → smart_search with intent=debug → read signatures → inspect symbol → smart_shell for tests/errors → smart_turn(end event=milestone or blocker).
-- Review: smart_turn(start) → smart_search with intent=implementation → read outline/signatures, focus on changed symbols, minimal changes → smart_turn(end event=milestone).
-- Refactor: smart_turn(start) → smart_search with intent=implementation → signatures for public API, preserve behavior, small edits, verify with tests → smart_turn(end event=milestone).
-- Tests: smart_turn(start) → smart_search with intent=tests → find existing tests, read symbol of function under test → smart_turn(end event=milestone).
-- Config: smart_search with intent=config → find settings, env vars, infrastructure files.
-- Architecture: smart_turn(start) → smart_search with intent=explore → directory structure, outlines of key modules and API boundaries → smart_turn(end event=task_switch or milestone).`;
+const agentRuleBody = `**First time in project?** Run build_index to enable search/context quality.
+
+Prefer devctx MCP for non-trivial tasks:
+- smart_read(outline|signatures|symbol) instead of Read → 90% savings
+- smart_search(intent=...) instead of Grep → ranked results
+- smart_context instead of multiple reads → one-call builder
+- smart_shell instead of Shell → safe diagnostics
+
+For non-trivial tasks: smart_turn(start) → [work with devctx tools] → smart_turn(end)
+
+Reading cascade: outline → signatures → symbol → full (last resort)
+
+Detailed workflows: .cursor/rules/profiles-compact/ (debugging, code-review, refactoring, testing, architecture)
+
+---
+
+If you didn't use devctx tools in a non-trivial programming task, add at the end:
+
+**Note:** devctx not used because: [task too simple | MCP unavailable | index not built | already had sufficient context | native tool more direct for this case]. To use devctx next time: "Use smart-context-mcp: smart_turn(start) → smart_context/smart_search → smart_read → smart_turn(end)"`;
 
 const cursorRuleContent = `---
-description: Prefer devctx MCP tools for file reading, code search, and diagnostics
+description: Prefer devctx MCP tools for non-trivial tasks
 alwaysApply: true
 ---
 
 ${agentRuleBody}
 `;
 
+const cursorProfilesNote = `
+# Task-Specific Profiles
+
+For detailed workflows, see rules in this directory:
+- debugging.mdc - Error-first, symbol-focused (90% savings)
+- code-review.mdc - Diff-aware, API-focused (87% savings)
+- refactoring.mdc - Graph-aware, test-verified (89% savings)
+- testing.mdc - Coverage-aware, TDD-friendly (90% savings)
+- architecture.mdc - Index-first, minimal-detail (90% savings)
+
+These profiles are **conditionally applied** based on file globs and task context.
+The base rule (devctx.mdc) is **always active** but kept minimal to reduce fixed context cost.
+`;
+
 const updateCursorRule = (targetDir, dryRun) => {
-  const filePath = path.join(targetDir, '.cursor', 'rules', 'devctx.mdc');
-  writeFile(filePath, cursorRuleContent, dryRun);
+  const rulesDir = path.join(targetDir, '.cursor', 'rules');
+  const profilesDir = path.join(rulesDir, 'profiles-compact');
+  
+  // Write base rule (always active)
+  const baseFilePath = path.join(rulesDir, 'devctx.mdc');
+  writeFile(baseFilePath, cursorRuleContent, dryRun);
+  
+  // Write profiles README
+  const profilesReadmePath = path.join(profilesDir, 'README.md');
+  writeFile(profilesReadmePath, cursorProfilesNote, dryRun);
+  
+  // Copy compact profiles from package
+  const sourceProfilesDir = path.join(devctxDir, 'agent-rules', 'profiles-compact');
+  if (fs.existsSync(sourceProfilesDir)) {
+    const profiles = fs.readdirSync(sourceProfilesDir).filter(f => f.endsWith('.mdc'));
+    profiles.forEach(profile => {
+      const sourcePath = path.join(sourceProfilesDir, profile);
+      const targetPath = path.join(profilesDir, profile);
+      const content = fs.readFileSync(sourcePath, 'utf8');
+      writeFile(targetPath, content, dryRun);
+    });
+  }
 };
 
 const SECTION_START = '<!-- devctx:start -->';

package/scripts/report-metrics.js

@@ -2,6 +2,7 @@
 import path from 'node:path';
 import { fileURLToPath } from 'node:url';
 import { smartMetrics } from '../src/tools/smart-metrics.js';
+import { formatAdoptionReport } from '../src/analytics/adoption.js';
 
 const requireValue = (argv, index, flag) => {
   const value = argv[index + 1];
@@ -88,6 +89,10 @@ const printHuman = (report) => {
       `  ${tool.tool.padEnd(14)} count=${formatNumber(tool.count)} raw=${formatNumber(tool.rawTokens)} final=${formatNumber(tool.compressedTokens)} saved=${formatNumber(tool.savedTokens)} (${tool.savingsPct}%)`
     );
   }
+  
+  if (report.adoption) {
+    console.log(formatAdoptionReport(report.adoption));
+  }
 };
 
 export const main = async () => {