smart-context-mcp 1.0.3 → 1.0.4

package/README.md

@@ -5,7 +5,7 @@
 
 **MCP server that reduces AI agent token usage by 90% and improves response quality.**
 
-Instead of reading entire files and repeating context, this MCP provides 7 smart tools that compress, rank, and maintain context efficiently.
+Instead of reading entire files and repeating context, this MCP provides 8 focused tools that compress, rank, and maintain context efficiently.
 
 ## Why use this?
 
@@ -15,7 +15,7 @@ Instead of reading entire files and repeating context, this MCP provides 7 smart
 
 **Real metrics from production use:**
 - 14.5M tokens → 1.6M tokens (89.87% reduction)
-- 3,666 successful calls across 7 tools
+- 3,666 successful calls across the original 7 core tools
 - Compression ratios: 3x to 46x depending on tool
 
 ## Quick Start (2 commands)
@@ -31,13 +31,15 @@ That's it. Restart your AI client (Cursor, Codex, Claude Desktop) and the tools
 
 ## What you get
 
-Seven focused tools that work automatically:
+Eight focused tools that work automatically:
 
 - `smart_read`: compact file summaries instead of full file dumps (3x compression)
 - `smart_read_batch`: read multiple files in one call — reduces round-trip latency
 - `smart_search`: ripgrep-first code search with intent-aware ranking (21x compression)
 - `smart_context`: one-call context planner — search + read + graph expansion
 - `smart_summary`: maintain compressed conversation state across sessions (46x compression)
+- `smart_turn`: one-call turn orchestration for start/end context recovery and checkpointing
+- `smart_metrics`: inspect saved token metrics and recent usage through MCP
 - `smart_shell`: safe diagnostic shell execution with restricted commands (18x compression)
 - `build_index`: lightweight symbol index for faster lookups and smarter ranking
 
@@ -81,14 +83,18 @@ npx smart-context-init --target .
 ```
 
 This installs the MCP server and generates client configs for Cursor, Codex, Qwen, and Claude Code. Open the project with your IDE/agent and the server starts automatically.
+If the target is a git repository, `smart-context-init` also installs an idempotent `pre-commit` hook that blocks commits when `.devctx/state.sqlite` is staged, tracked, or not properly ignored.
+For Claude Code, `smart-context-init` also generates `.claude/settings.json` with native hooks so devctx context recovery and turn-end enforcement run automatically.
 
 ## Binaries
 
-The package exposes three binaries:
+The package exposes five binaries:
 
+- `smart-context-headless`
 - `smart-context-server`
 - `smart-context-init`
 - `smart-context-report`
+- `smart-context-protect`
 
 Start the MCP server against the current project:
 
@@ -127,10 +133,12 @@ smart-context-init --target /path/to/project --command node --args '["./tools/de
 Each tool call persists token metrics to the target repo by default in:
 
 ```bash
-.devctx/metrics.jsonl
+.devctx/state.sqlite
 ```
 
-This makes per-repo usage visible without digging into `node_modules`. Running `smart-context-init` also adds `.devctx/` to the target repo's `.gitignore` idempotently.
+SQLite is now the primary project-local store for persisted context and metrics. Running `smart-context-init` also adds `.devctx/` to the target repo's `.gitignore` idempotently.
+
+When an active session exists, metrics entries automatically inherit its `sessionId`, so you can inspect savings per task with `smart_metrics`.
 
 Show a quick report:
 
@@ -138,7 +146,7 @@ Show a quick report:
 smart-context-report
 ```
 
-Show JSON output or a custom file:
+Show JSON output or inspect a legacy/custom JSONL file explicitly:
 
 ```bash
 smart-context-report --json
@@ -150,8 +158,8 @@ Example output:
 ```text
 devctx metrics report
 
-File:         /path/to/repo/.devctx/metrics.jsonl
-Source:       default
+File:         /path/to/repo/.devctx/state.sqlite
+Source:       sqlite
 Entries:      148
 Raw tokens:   182,340
 Final tokens: 41,920
@@ -163,7 +171,7 @@ By tool:
   smart_search   count=35 raw=33,330 final=7,800 saved=25,530 (76.59%)
 ```
 
-If you want to override the location entirely, set `DEVCTX_METRICS_FILE`.
+If you need JSONL compatibility for external tooling, set `DEVCTX_METRICS_FILE` or pass `--file`.
 
 ## Usage per client
 
@@ -171,7 +179,7 @@ After installing and running `smart-context-init`, each client picks up the serv
 
 ### Cursor
 
-Open the project in Cursor. The MCP server starts automatically. Enable it in **Cursor Settings > MCP** if needed. All seven tools are available in Agent mode.
+Open the project in Cursor. The MCP server starts automatically. Enable it in **Cursor Settings > MCP** if needed. All eight tools are available in Agent mode.
 
 ### Codex CLI
 
@@ -189,7 +197,20 @@ cd /path/to/your-project
 claude
 ```
 
-Claude Code reads `.mcp.json` from the project root.
+Claude Code reads `.mcp.json` from the project root and `.claude/settings.json` for native hook automation.
+
+### Codex/Qwen headless fallback
+
+When a client does not expose native per-turn hooks, use `smart-context-headless` to wrap a headless CLI run and force `smart_turn(start)` plus a closing checkpoint around that invocation.
+
+Examples:
+
+```bash
+smart-context-headless --client codex --prompt "Finish the runtime repo-safety docs" -- codex exec
+smart-context-headless --client qwen --prompt "Review the persisted session and propose the next step" -- qwen -p
+```
+
+This is the current automation path for non-Claude CLI agents. GUI clients without hook support still rely on generated rules plus `smart_turn`.
 
 ### Qwen Code
 
@@ -215,9 +236,9 @@ The `intent` parameter in `smart_search` and `smart_context` adjusts ranking and
 
 - **Cursor**: `.cursor/rules/devctx.mdc` (always-apply rule)
 - **Codex**: `AGENTS.md` (devctx section with sentinel markers)
-- **Claude Code**: `CLAUDE.md` (devctx section with sentinel markers)
+- **Claude Code**: `CLAUDE.md` (devctx section with sentinel markers) and `.claude/settings.json` (native hooks)
 
-The rules are idempotent — running `smart-context-init` again updates the section without duplicating it. Existing content in `AGENTS.md` and `CLAUDE.md` is preserved.
+The generated files are idempotent — running `smart-context-init` again updates the devctx sections and Claude hook entries without duplicating them. Existing content in `AGENTS.md`, `CLAUDE.md`, and `.claude/settings.json` is preserved.
 
 ## Use against another repo
 
@@ -449,16 +470,20 @@ Maintain compressed conversation state across sessions. Solves the context-loss
 
 | Action | Purpose | Returns |
 |--------|---------|---------|
-| `get` | Retrieve current or specified session | Resume summary (≤500 tokens) + compression metadata |
+| `get` | Retrieve current, explicit, or auto-resolved session | Resume summary (≤500 tokens) + compression metadata |
 | `update` | Create or replace session | New session with compressed state |
 | `append` | Add to existing session | Merged session state |
+| `auto_append` | Add only when something meaningful changed | Merged session state or skipped no-op result |
+| `checkpoint` | Event-driven orchestration for persistence decisions | Persisted update or skipped event with decision metadata |
 | `reset` | Clear session | Confirmation |
 | `list_sessions` | Show all available sessions | Array of sessions with metadata |
+| `compact` | Apply retention/compaction to SQLite state | Counts for pruned sessions, events, and metrics |
+| `cleanup_legacy` | Inspect or remove imported JSON/JSONL artifacts | Dry-run or deletion report |
 
 **Parameters:**
 - `action` (required) — one of the actions above
-- `sessionId` (optional) — session identifier; auto-generated from `goal` if omitted
-- `update` (required for update/append) — object with:
+- `sessionId` (optional) — session identifier; auto-generated from `goal` if omitted. Pass `"auto"` to accept the recommended recent session when multiple candidates exist.
+- `update` (required for update/append/auto_append/checkpoint) — object with:
   - `goal`: primary objective
   - `status`: current state (`planning` | `in_progress` | `blocked` | `completed`)
   - `pinnedContext`: critical context that should survive compression when possible
@@ -471,14 +496,27 @@ Maintain compressed conversation state across sessions. Solves the context-loss
   - `nextStep`: immediate next action
   - `touchedFiles`: array of modified files
 - `maxTokens` (optional, default 500) — hard cap on summary size
+- `event` (optional for `checkpoint`) — one of `manual`, `milestone`, `decision`, `blocker`, `status_change`, `file_change`, `task_switch`, `task_complete`, `session_end`, `read_only`, `heartbeat`
+- `force` (optional, default false) — override a suppressed checkpoint event
+- `retentionDays` (optional, default 30) — used by `compact`
+- `keepLatestEventsPerSession` (optional, default 20) — used by `compact`
+- `keepLatestMetrics` (optional, default 1000) — used by `compact`
+- `vacuum` (optional, default false) — run SQLite `VACUUM` after deletions during `compact`
+- `apply` (optional, default false) — required to actually delete files during `cleanup_legacy`
 
-`update` replaces the stored session state for that `sessionId`, so omitted fields are cleared. Use `append` when you want to keep existing state and add progress incrementally.
+`update` replaces the stored session state for that `sessionId`, so omitted fields are cleared. Use `append` when you want to keep existing state and add progress incrementally. Use `auto_append` when the caller may fire checkpoint saves often and you want the tool to skip no-op updates automatically. Use `checkpoint` when the caller has a meaningful event and wants the tool to decide whether that event deserves persistence.
 
 **Storage:**
-- Sessions persist in `.devctx/sessions/<sessionId>.json`
-- Active session tracked in `.devctx/sessions/active.json`
-- 30-day retention for inactive sessions
-- No expiration for active sessions
+- Session state, session events, summary cache, and metrics persist in `.devctx/state.sqlite`
+- Legacy `.devctx/sessions/*.json`, `.devctx/sessions/active.json`, and `.devctx/metrics.jsonl` are imported idempotently when present
+- `compact` enforces retention without deleting the active session
+- `cleanup_legacy` is dry-run by default and only deletes imported legacy artifacts when `apply: true`
+
+**Auto-resume behavior:**
+- `get` returns the active session immediately when `active.json` exists
+- If there is no active session, `get` auto-resumes the best saved session when there is a single clear candidate
+- If multiple recent sessions are plausible, `get` returns ordered `candidates` plus `recommendedSessionId`
+- Passing `sessionId: "auto"` accepts that recommendation and restores it as the active session
 
 **Resume summary fields:**
 - `status` and `nextStep` are preserved with highest priority
@@ -493,6 +531,8 @@ Maintain compressed conversation state across sessions. Solves the context-loss
 - `truncated`: whether the resume summary had to be compressed
 - `compressionLevel`: `none` | `trimmed` | `reduced` | `status_only`
 - `omitted`: fields dropped from the resume summary to fit the token budget
+- `repoSafety`: git hygiene signal for `.devctx/state.sqlite` (`isIgnored`, `isTracked`, `isStaged`, warnings, recommended actions)
+- mutating actions (`update`, `append`, `auto_append`, `checkpoint`, `reset`, `compact`) are blocked at runtime when `.devctx/state.sqlite` is tracked or staged
 
 **Compression strategy:**
 - Keeps the persisted session state intact and compresses only the resume summary
@@ -505,11 +545,12 @@ Maintain compressed conversation state across sessions. Solves the context-loss
 ```javascript
 // Start of work session
 smart_summary({ action: "get" })
-// → retrieves last active session or returns "not found"
+// → retrieves last active session or auto-resumes the best saved session
 
 // After implementing auth middleware
 smart_summary({ 
-  action: "append",
+  action: "checkpoint",
+  event: "milestone",
   update: {
     completed: ["auth middleware"],
     decisions: ["JWT with 1h expiry, refresh tokens in Redis"],
@@ -525,6 +566,77 @@ smart_summary({ action: "get" })
 // List all sessions
 smart_summary({ action: "list_sessions" })
 // → see all available sessions, pick one to resume
+
+// Inspect git safety for project-local state from any smart_summary response
+smart_summary({ action: "get" })
+// → repoSafety warns if .devctx/state.sqlite is tracked or not ignored
+
+// Suppress noisy read-only exploration checkpoints
+smart_summary({
+  action: "checkpoint",
+  event: "read_only",
+  update: { currentFocus: "inspect auth flow" }
+})
+// → skipped=true, no event persisted
+
+// Compact old SQLite events while keeping recent history
+smart_summary({ action: "compact", retentionDays: 30, keepLatestEventsPerSession: 20, keepLatestMetrics: 1000 })
+
+// Inspect what legacy files are safe to remove
+smart_summary({ action: "cleanup_legacy" })
+
+// Remove imported legacy JSON/JSONL artifacts explicitly
+smart_summary({ action: "cleanup_legacy", apply: true })
+```
+
+### `smart_metrics`
+
+Inspect token metrics recorded in project-local SQLite storage without leaving MCP.
+
+- Returns aggregated totals, savings percentage, and per-tool breakdowns
+- Supports `window`: `24h` | `7d` | `30d` | `all`
+- Supports filtering by `tool`
+- Supports filtering by `sessionId`, including `sessionId: "active"`
+- Includes `latestEntries` so an agent can explain recent savings without parsing storage manually
+- Includes `overheadTokens` and `overheadTools` so hook/wrapper context cost stays measurable against the savings
+- When `.devctx/state.sqlite` is tracked or staged, metric writes are skipped and reads fall back to a temporary read-only snapshot with `sideEffectsSuppressed: true`
+
+**Example workflow:**
+
+```javascript
+smart_metrics({ window: "7d", sessionId: "active" })
+// → totals and recent entries for the current task/session
+```
+
+### `smart_turn`
+
+Orchestrate the start or end of a meaningful agent turn with one MCP call.
+
+- `phase: "start"` rehydrates context, classifies whether the current prompt aligns with persisted work, and can auto-create a planning session for a substantial new task
+- `phase: "end"` writes a checkpoint through `smart_summary` and can optionally include compact metrics
+- Designed to make context usage almost mandatory without forcing the agent to chain `smart_summary(get)` and `smart_summary(checkpoint)` manually on every turn
+- Claude Code can invoke this automatically through generated native hooks on `SessionStart`, `UserPromptSubmit`, `PostToolUse`, and `Stop`
+- Non-Claude CLI clients can approximate the same flow with `smart-context-headless`, which wraps one headless agent invocation around `smart_turn(start)` and `smart_turn(end)`
+
+**Example workflow:**
+
+```javascript
+smart_turn({
+  phase: "start",
+  prompt: "Finish runtime repo-safety enforcement for smart metrics",
+  ensureSession: true
+})
+// → summary + continuity classification + repoSafety
+
+smart_turn({
+  phase: "end",
+  event: "milestone",
+  update: {
+    completed: ["Finished smart metrics repo-safety enforcement"],
+    nextStep: "Update docs and run the full suite"
+  }
+})
+// → checkpoint result + optional compact metrics
 ```
 
 ### `build_index`
@@ -575,9 +687,10 @@ Metrics include: P@5, P@10, Recall, wrong-file rate, retrieval honesty, follow-u
 ## Notes
 
 - `@vscode/ripgrep` provides a bundled `rg` binary, so a system install is not required.
-- Metrics are written to `<projectRoot>/.devctx/metrics.jsonl` (override with `DEVCTX_METRICS_FILE` env var).
+- Persistent context and metrics live in `<projectRoot>/.devctx/state.sqlite`.
+- `DEVCTX_METRICS_FILE` is now an explicit compatibility override for JSONL-based workflows and reports.
 - Symbol index stored in `<projectRoot>/.devctx/index.json` when `build_index` is used.
-- Conversation sessions stored in `<projectRoot>/.devctx/sessions/` when `smart_summary` is used.
+- Legacy session JSON files in `<projectRoot>/.devctx/sessions/` are imported idempotently when present.
 - This package is a navigation and diagnostics layer, not a full semantic code intelligence system.
 
 ## Repository

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "smart-context-mcp",
-  "version": "1.0.3",
+  "version": "1.0.4",
   "description": "MCP server that reduces agent token usage and improves response quality with compact file summaries, ranked code search, and curated context.",
   "author": "Francisco Caballero Portero <fcp1978@hotmail.com>",
   "type": "module",
@@ -13,9 +13,11 @@
     "url": "https://github.com/Arrayo/devctx-mcp-mvp/issues"
   },
   "bin": {
+    "smart-context-headless": "scripts/headless-wrapper.js",
     "smart-context-server": "scripts/devctx-server.js",
     "smart-context-init": "scripts/init-clients.js",
-    "smart-context-report": "scripts/report-metrics.js"
+    "smart-context-report": "scripts/report-metrics.js",
+    "smart-context-protect": "scripts/check-repo-safety.js"
   },
   "exports": {
     ".": "./src/server.js",
@@ -23,7 +25,10 @@
   },
   "files": [
     "src/",
+    "scripts/claude-hook.js",
+    "scripts/check-repo-safety.js",
     "scripts/devctx-server.js",
+    "scripts/headless-wrapper.js",
     "scripts/init-clients.js",
     "scripts/report-metrics.js"
   ],
@@ -45,7 +50,7 @@
     "smoke:json": "node ./scripts/smoke-test.js --json",
     "init:clients": "node ./scripts/init-clients.js",
     "smoke:formats": "node ./scripts/format-smoke.js",
-    "test": "node --test ./tests/*.test.js",
+    "test": "node --test --test-concurrency=1 ./tests/*.test.js",
     "eval": "node ./evals/harness.js",
     "eval:context": "node ./evals/harness.js --tool=context",
     "eval:both": "node ./evals/harness.js --tool=both",

package/scripts/check-repo-safety.js

@@ -0,0 +1,84 @@
+#!/usr/bin/env node
+import fs from 'node:fs';
+import path from 'node:path';
+import { enforceRepoSafety } from '../src/repo-safety.js';
+import { setProjectRoot } from '../src/utils/runtime-config.js';
+
+const writeStdout = (text) => {
+  fs.writeSync(process.stdout.fd, text);
+};
+
+const writeStderr = (text) => {
+  fs.writeSync(process.stderr.fd, text);
+};
+
+const parseArgs = (argv) => {
+  const options = {
+    projectRoot: process.cwd(),
+    json: false,
+  };
+
+  for (let index = 0; index < argv.length; index += 1) {
+    const token = argv[index];
+
+    if (token === '--project-root') {
+      const next = argv[index + 1];
+      if (!next || next.startsWith('--')) {
+        throw new Error('Missing value for --project-root');
+      }
+      options.projectRoot = path.resolve(next);
+      index += 1;
+      continue;
+    }
+
+    if (token === '--json') {
+      options.json = true;
+      continue;
+    }
+
+    throw new Error(`Unknown argument: ${token}`);
+  }
+
+  return options;
+};
+
+const printHuman = (result) => {
+  if (result.ok) {
+    writeStdout('devctx repo safety: ok\n');
+    return;
+  }
+
+  writeStderr('devctx repo safety: failed\n');
+  for (const violation of result.violations) {
+    writeStderr(`- ${violation}\n`);
+  }
+  for (const action of result.recommendedActions) {
+    writeStderr(`- ${action}\n`);
+  }
+};
+
+const main = () => {
+  const options = parseArgs(process.argv.slice(2));
+  setProjectRoot(options.projectRoot);
+
+  const result = enforceRepoSafety({ root: options.projectRoot });
+  if (options.json) {
+    const output = `${JSON.stringify(result, null, 2)}\n`;
+    if (result.ok) {
+      writeStdout(output);
+    } else {
+      writeStderr(output);
+    }
+  } else {
+    printHuman(result);
+  }
+
+  process.exitCode = result.ok ? 0 : 1;
+};
+
+try {
+  main();
+} catch (error) {
+  writeStderr(`${error.message}\n`);
+  process.exitCode = 1;
+}

package/scripts/claude-hook.js

@@ -0,0 +1,33 @@
+#!/usr/bin/env node
+import { handleClaudeHookEvent } from '../src/hooks/claude-hooks.js';
+
+const readStdin = async () => {
+  let buffer = '';
+  for await (const chunk of process.stdin) {
+    buffer += chunk;
+  }
+  return buffer.trim();
+};
+
+const main = async () => {
+  const raw = await readStdin();
+  if (!raw) {
+    return;
+  }
+
+  const input = JSON.parse(raw);
+  const result = await handleClaudeHookEvent(input);
+
+  if (!result) {
+    return;
+  }
+
+  process.stdout.write(`${JSON.stringify(result)}\n`);
+};
+
+main().catch((error) => {
+  if (process.env.DEVCTX_DEBUG === '1') {
+    process.stderr.write(`devctx claude hook error: ${error.message}\n`);
+  }
+  process.exit(0);
+});

package/scripts/headless-wrapper.js

@@ -0,0 +1,106 @@
+#!/usr/bin/env node
+import { runHeadlessWrapper } from '../src/orchestration/headless-wrapper.js';
+
+const requireValue = (argv, index, flag) => {
+  const value = argv[index + 1];
+  if (!value || value === '--') {
+    throw new Error(`Missing value for ${flag}`);
+  }
+  return value;
+};
+
+const parseArgs = (argv) => {
+  const options = {
+    client: 'generic',
+    prompt: '',
+    sessionId: undefined,
+    event: undefined,
+    stdinPrompt: false,
+    dryRun: false,
+    json: false,
+    streamOutput: true,
+    command: '',
+    args: [],
+  };
+
+  let commandIndex = argv.indexOf('--');
+  const head = commandIndex === -1 ? argv : argv.slice(0, commandIndex);
+
+  for (let index = 0; index < head.length; index += 1) {
+    const token = head[index];
+
+    if (token === '--client') {
+      options.client = requireValue(head, index, '--client');
+      index += 1;
+      continue;
+    }
+
+    if (token === '--prompt') {
+      options.prompt = requireValue(head, index, '--prompt');
+      index += 1;
+      continue;
+    }
+
+    if (token === '--session-id') {
+      options.sessionId = requireValue(head, index, '--session-id');
+      index += 1;
+      continue;
+    }
+
+    if (token === '--event') {
+      options.event = requireValue(head, index, '--event');
+      index += 1;
+      continue;
+    }
+
+    if (token === '--stdin-prompt') {
+      options.stdinPrompt = true;
+      continue;
+    }
+
+    if (token === '--dry-run') {
+      options.dryRun = true;
+      continue;
+    }
+
+    if (token === '--json') {
+      options.json = true;
+      continue;
+    }
+
+    if (token === '--quiet') {
+      options.streamOutput = false;
+      continue;
+    }
+
+    throw new Error(`Unknown argument: ${token}`);
+  }
+
+  if (commandIndex !== -1) {
+    const commandParts = argv.slice(commandIndex + 1);
+    if (commandParts.length > 0) {
+      [options.command, ...options.args] = commandParts;
+    }
+  }
+
+  return options;
+};
+
+const main = async () => {
+  const options = parseArgs(process.argv.slice(2));
+  if (options.json) {
+    options.streamOutput = false;
+  }
+  const result = await runHeadlessWrapper(options);
+  if (options.dryRun || options.json) {
+    process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
+  }
+  if (!options.dryRun && result.exitCode !== 0) {
+    process.exitCode = result.exitCode;
+  }
+};
+
+main().catch((error) => {
+  console.error(error.message);
+  process.exit(1);
+});