docdex 0.2.24 → 0.2.25

package/assets/agents.md

@@ -1,3 +1,7 @@
+---- START OF DOCDEX INFO V0.2.25 ----
+Docdex URL: http://127.0.0.1:28491
+Use this base URL for Docdex HTTP endpoints.
+
 ## Priority Directives (Mandatory)
 
 - Always use both memory lobes early: docdex_get_profile + docdex_memory_recall; save new facts via docdex_memory_save and new preferences via docdex_save_preference.
@@ -66,6 +70,7 @@ Precision tools for structural analysis. Do not rely on text search for definiti
 | docdex_ast | Specific AST nodes (e.g., "Find all class definitions"). |
 | docdex_impact_diagnostics | Check for broken/dynamic imports. |
 | HTTP /v1/graph/impact | Impact Analysis: "What breaks if I change this?" Returns inbound/outbound dependencies. |
+| HTTP /v1/dag/export | Export the dependency DAG for change ordering and scope. |
 
 ### C. Memory Operations
 
@@ -76,6 +81,30 @@ Precision tools for structural analysis. Do not rely on text search for definiti
 | docdex_save_preference | Store a global user preference (Style, Tooling, Constraint). |
 | docdex_get_profile | Retrieve global preferences. |
 
+### D. Local Delegation (Cheap Models)
+
+Use local delegation for low-complexity, code-generation-oriented tasks to reduce paid-model usage.
+
+| MCP Tool / HTTP | Purpose |
+| --- | --- |
+| docdex_local_completion | Delegate small tasks to a local model with strict output formats. |
+| HTTP /v1/delegate | HTTP endpoint for delegated completions with structured responses. |
+
+Required fields: `task_type`, `instruction`, `context`. Optional: `max_tokens`, `timeout_ms`, `mode` (`draft_only` or `draft_then_refine`), `agent` (local agent id/slug).
+Expensive model library: `docs/expensive_models.json` (match by `agent_id`, `agent_slug`, `model`, or adapter type; case-insensitive).
+
+### E. Index Health + File Access
+
+Use these to verify index coverage, repo binding, and to read precise file slices.
+
+| MCP Tool | Purpose |
+| --- | --- |
+| docdex_repo_inspect | Confirm normalized repo root/identity (resolve missing_repo). |
+| docdex_stats | Index size/last update; detect stale indexes. |
+| docdex_files | Indexed file coverage; confirm a file is in the index. |
+| docdex_index | Reindex full repo or ingest specific files when stale/missing. |
+| docdex_open | Read exact file slices after you identify targets. |
+
 ## Quick Tool Map (Often Missed)
 
 - docdex_files: List indexed docs with rel_path/doc_id/token_estimate; use to verify indexing coverage.
@@ -84,8 +113,14 @@ Precision tools for structural analysis. Do not rely on text search for definiti
 - docdex_index: Reindex the full repo or ingest specific files when stale.
 - docdex_search diff: Limit search to working tree, staged, or ref ranges; filter by paths.
 - docdex_web_research knobs: force_web, skip_local_search, repo_only, no_cache, web_limit, llm_filter_local_results, llm_model.
+- docdex_open: Read narrow file slices after targets are identified.
+- docdex_impact_diagnostics: Scan dynamic imports when imports are unclear or failing.
+- docdex_local_completion: Delegate low-complexity codegen tasks (tests, docstrings, boilerplate, simple refactors).
+- docdex_ast: Use AST queries for precise structure (class/function definitions, call sites, imports).
+- docdex_symbols: Use symbols to confirm exact signatures/locations before edits.
+- HTTP /v1/graph/impact: Mandatory before code changes to review inbound/outbound deps.
+- HTTP /v1/dag/export: Export dependency graph to plan change order.
 - HTTP /v1/initialize: Bind a default repo root for MCP when clients omit project_root.
-- HTTP /v1/dag/export: Export the dependency graph for external analysis.
 
 ## Interaction Patterns
 
@@ -96,9 +131,76 @@ When answering a complex coding query, follow this "Reasoning Trace":
 1. Retrieve Profile: Call docdex_get_profile to load user style/constraints (e.g., "Use functional components").
 2. Search Code: Call docdex_search or docdex_symbols to find the relevant code.
 3. Check Memory: Call docdex_memory_recall for project-specific caveats (e.g., "Auth logic was refactored last week").
-4. Synthesize: Generate code that matches the Repo Truth while adhering to the Profile Style.
+4. Validate structure: Use docdex_ast/docdex_symbols to confirm targets before editing.
+5. Read context: Use docdex_open to fetch minimal file slices after locating targets.
+6. Plan with DAG: Use /v1/dag/export or /v1/graph/impact to order changes by dependencies.
+7. Synthesize: Generate code that matches the Repo Truth while adhering to the Profile Style.
+
+### 2. Memory Capture (Mandatory)
+
+Save more memories for both lobes during the task, not just at the end.
+
+1. Repo memory: After each meaningful discovery or code change, save at least one durable fact (file location, behavior, config, gotcha) via `docdex_memory_save`.
+2. Profile memory: When the user expresses a preference, constraint, or workflow correction, call `docdex_save_preference` immediately with the right category.
+3. Keep it crisp: 1-3 short sentences, include file paths when relevant, avoid raw code blobs.
+4. Safety: Never store secrets, tokens, or sensitive user data. Skip transient or speculative info.
+
+### 3. Index Health + Diff-Aware Search (Mandatory)
+
+Use these when results look incomplete or when the task is about recent changes.
+
+1. Confirm repo binding: Use docdex_repo_inspect or /v1/initialize when repo_id is missing/ambiguous.
+2. Check index coverage: Use docdex_stats + docdex_files before assuming code is missing.
+3. Reindex if needed: Run docdex_index (or advise it) when stale_index/missing files appear.
+4. Use diff search: For change-specific tasks, use docdex_search with diff mode (working tree/staged/range).
+
+### 4. Local Delegation Workflow (Cheap Models)
+
+When you are an expensive/paid model, delegate eligible low-complexity tasks first unless the user opts out.
+Delegation is mandatory for eligible tasks. If you are expensive and delegation is enabled (explicitly or auto-enabled with local candidates), you MUST attempt local delegation before doing the work yourself. Only skip when delegation is disabled, no local candidate exists, or local output fails validation after a retry/fallback.
+
+#### Mandatory Delegation Triggers
+
+- The work fits a delegation task type or can be split into sub-tasks that do.
+- The change is small, localized, or mechanical (boilerplate, formatting, docstrings, small refactors).
+- The task can be fully expressed in a capsule with explicit context and output rules.
+
+#### Reasons to Delegate (Non-Optional)
+
+- Cuts paid-token usage and cost.
+- Lowers latency and avoids rate-limit pressure on premium models.
+- Keeps expensive models focused on complex reasoning and multi-step planning.
+- Uses tuned prompts that reduce drift for repetitive edits.
+- Enforces the product's cost-control policy; skipping delegation is a regression.
+
+1. Determine if you are expensive: load `docs/expensive_models.json` and match your `agent_id`, `agent_slug`, `model`, or adapter type. If you match, continue; if not and you are clearly local/cheap, skip delegation. If unsure, default to expensive.
+2. Check config: Only delegate when `[llm.delegation].enabled` is true or `auto_enable` is true with an eligible local model/agent (and `task_type` is allowed). If uncertain, attempt delegation and handle the error.
+3. Choose task type: Use one of `GENERATE_TESTS`, `WRITE_DOCSTRING`, `SCAFFOLD_BOILERPLATE`, `REFACTOR_SIMPLE`, `FORMAT_CODE`.
+4. Call the tool: `docdex_local_completion` with `task_type`, `instruction`, and minimal `context` (smallest necessary snippet).
+5. Validate output: If the local output is invalid or empty, fall back to the primary agent or handle with the paid model.
+6. Optional refine: If mode is `draft_then_refine`, refine the draft with the primary agent and return a final result.
+
+#### Delegation Handoff Package (Required)
+
+Local models cannot call tools. The leading agent must provide a complete, minimal capsule.
+
+1. Task capsule: `task_type`, goal, success criteria, output format, and constraints (tests to update, style rules).
+2. Context payload: file paths plus the exact snippets from docdex_open; include symbol signatures/AST findings.
+3. Dependency notes: summarize impact analysis and any DAG ordering that affects the change.
+4. Boundaries: explicit files allowed to edit vs read-only; no new dependencies unless allowed.
+5. Guardrails: ask for clarification if context is insufficient; do not invent missing APIs; return only the requested format.
+
+### 5. Graph + AST Usage (Mandatory for Code Changes)
+
+For any code change, use both AST and graph tools to reduce drift and hidden coupling.
+
+1. Use `docdex_ast` or `docdex_symbols` to locate exact definitions and call sites.
+2. Call HTTP `/v1/graph/impact?file=...` before edits and summarize inbound/outbound deps.
+3. For multi-file changes, export the DAG (`/v1/dag/export`) and order edits by dependency direction.
+4. Use docdex_impact_diagnostics when imports are dynamic or unresolved.
+5. If graph endpoints are unavailable, state it and proceed cautiously with extra local search.
 
-### 2. Handling Corrections (Learning)
+### 6. Handling Corrections (Learning)
 
 If the user says: "I told you, we do not use Moment.js here, use date-fns!"
 
@@ -107,21 +209,21 @@ If the user says: "I told you, we do not use Moment.js here, use date-fns!"
 - content: "Do not use Moment.js; prefer date-fns."
 - agent_id: "default" (or active agent ID)
 
-### 3. Impact Analysis
+### 7. Impact Analysis
 
 If the user asks: "Safe to delete getUser?"
 
 - Action: Call GET /v1/graph/impact?file=src/user.ts
 - Output: Analyze the inbound edges. If the list is not empty, it is unsafe.
 
-### 4. Non-Repo Real-World Queries (Web First)
+### 8. Non-Repo Real-World Queries (Web First)
 
 If the user asks a non-repo, real-world question (weather, news, general facts), immediately call docdex_web_research with force_web=true.
 - Resolve relative dates ("yesterday", "last week") using system time by default.
 - Do not run docdex_search unless the user explicitly wants repo-local context.
 - Assume web access is allowed unless the user forbids it; if the web call fails, report the failure and ask for a source or permission.
 
-### 5. Failure Handling (Missing Results or Errors)
+### 9. Failure Handling (Missing Results or Errors)
 
 - Ensure project_root or repo_path is set, or call /v1/initialize to bind a default root.
 - Use docdex_repo_inspect to confirm repo identity and normalized root.
@@ -151,3 +253,4 @@ Docdex adapts to the host.
 
 - Project Mapping: On constrained hardware, docdex uses a "Spotlight Heuristic" to show you only a skeletal file tree based on your role keywords, rather than the full file system.
 - LLM: It may be running a quantized model (e.g., phi3.5) or a heavy model (llama3.1:70b) depending on VRAM. Trust the daemon's token limits; it handles truncation.
+---- END OF DOCDEX INFO -----

package/lib/postinstall_setup.js

@@ -26,6 +26,7 @@ const SETUP_PENDING_MARKER = "setup_pending.json";
 const AGENTS_DOC_FILENAME = "agents.md";
 const DOCDEX_INFO_START_PREFIX = "---- START OF DOCDEX INFO V";
 const DOCDEX_INFO_END = "---- END OF DOCDEX INFO -----";
+const DOCDEX_INFO_END_LEGACY = "---- END OF DOCDEX INFO ----";
 
 function defaultConfigPath() {
   return path.join(os.homedir(), ".docdex", "config.toml");
@@ -353,9 +354,14 @@ function docdexBlockStart(version) {
   return `${DOCDEX_INFO_START_PREFIX}${version} ----`;
 }
 
+function docdexInfoEndPattern() {
+  return `(?:${escapeRegExp(DOCDEX_INFO_END)}|${escapeRegExp(DOCDEX_INFO_END_LEGACY)})`;
+}
+
 function buildDocdexInstructionBlock(instructions) {
   const next = normalizeInstructionText(instructions);
   if (!next) return "";
+  if (hasDocdexBlock(next)) return next;
   const version = resolvePackageVersion();
   return `${docdexBlockStart(version)}\n${next}\n${DOCDEX_INFO_END}`;
 }
@@ -363,9 +369,7 @@ function buildDocdexInstructionBlock(instructions) {
 function extractDocdexBlockBody(text) {
   const match = String(text || "").match(
     new RegExp(
-      `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\r\\n]* ----\\r?\\n([\\s\\S]*?)\\r?\\n${escapeRegExp(
-        DOCDEX_INFO_END
-      )}`
+      `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\r\\n]* ----\\r?\\n([\\s\\S]*?)\\r?\\n${docdexInfoEndPattern()}`
     )
   );
   return match ? normalizeInstructionText(match[1]) : "";
@@ -383,11 +387,17 @@ function hasDocdexBlockVersion(text, version) {
   return String(text || "").includes(docdexBlockStart(version));
 }
 
+function hasDocdexBlock(text) {
+  const source = String(text || "");
+  return (
+    source.includes(DOCDEX_INFO_START_PREFIX) &&
+    (source.includes(DOCDEX_INFO_END) || source.includes(DOCDEX_INFO_END_LEGACY))
+  );
+}
+
 function stripDocdexBlocks(text) {
   const re = new RegExp(
-    `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\r\\n]* ----\\r?\\n[\\s\\S]*?\\r?\\n${escapeRegExp(
-      DOCDEX_INFO_END
-    )}\\r?\\n?`,
+    `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\r\\n]* ----\\r?\\n[\\s\\S]*?\\r?\\n${docdexInfoEndPattern()}\\r?\\n?`,
     "g"
   );
   return String(text || "").replace(re, "").trim();
@@ -397,9 +407,7 @@ function stripDocdexBlocksExcept(text, version) {
   if (!version) return stripDocdexBlocks(text);
   const source = String(text || "");
   const re = new RegExp(
-    `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\r\\n]* ----\\r?\\n[\\s\\S]*?\\r?\\n${escapeRegExp(
-      DOCDEX_INFO_END
-    )}\\r?\\n?`,
+    `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\r\\n]* ----\\r?\\n[\\s\\S]*?\\r?\\n${docdexInfoEndPattern()}\\r?\\n?`,
     "g"
   );
   let result = "";
@@ -432,7 +440,7 @@ function stripLegacyDocdexBody(text, body) {
   if (!body) return String(text || "");
   const source = String(text || "").replace(/\r\n/g, "\n");
   const re = new RegExp(
-    `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\n]* ----\\n[\\s\\S]*?\\n${escapeRegExp(DOCDEX_INFO_END)}\\n?`,
+    `${escapeRegExp(DOCDEX_INFO_START_PREFIX)}[^\\n]* ----\\n[\\s\\S]*?\\n${docdexInfoEndPattern()}\\n?`,
     "g"
   );
   let result = "";
@@ -493,6 +501,18 @@ function upsertPromptFile(pathname, instructions, { prepend = false } = {}) {
   return writeTextFile(pathname, merged);
 }
 
+function removePromptFile(pathname) {
+  if (!fs.existsSync(pathname)) return false;
+  const current = fs.readFileSync(pathname, "utf8");
+  const stripped = stripDocdexBlocks(current);
+  if (normalizeInstructionText(stripped) === normalizeInstructionText(current)) return false;
+  if (!stripped) {
+    fs.unlinkSync(pathname);
+    return true;
+  }
+  return writeTextFile(pathname, stripped);
+}
+
 function escapeRegExp(value) {
   return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
 }
@@ -554,13 +574,7 @@ function upsertYamlInstruction(pathname, key, instructions) {
 }
 
 function upsertClaudeInstructions(pathname, instructions) {
-  const { value } = readJson(pathname);
-  if (typeof value !== "object" || value == null || Array.isArray(value)) return false;
-  const merged = mergeInstructionText(value.instructions, instructions);
-  if (!merged || merged === value.instructions) return false;
-  value.instructions = merged;
-  writeJson(pathname, value);
-  return true;
+  return upsertPromptFile(pathname, instructions);
 }
 
 function upsertContinueJsonInstructions(pathname, instructions) {
@@ -731,7 +745,7 @@ function rewriteContinueYamlRules(source, instructions, addDocdex) {
 
   const keptItems = items.filter((item) => {
     const text = item.join("\n");
-    return !(text.includes(DOCDEX_INFO_START_PREFIX) && text.includes(DOCDEX_INFO_END));
+    return !hasDocdexBlock(text);
   });
 
   if (addDocdex) {
@@ -786,45 +800,43 @@ function upsertVsCodeInstructionKey(value, key, instructions) {
   return true;
 }
 
-function upsertVsCodeInstructionLocations(value, instructionsDir) {
-  const key = "chat.instructionsFilesLocations";
-  const location = String(instructionsDir);
-  if (value[key] && typeof value[key] === "object" && !Array.isArray(value[key])) {
-    if (value[key][location] === true) return false;
-    value[key][location] = true;
-    return true;
-  }
-  if (Array.isArray(value[key])) {
-    if (value[key].some((entry) => entry === location)) return false;
-    value[key].push(location);
+function removeVsCodeInstructionKey(value, key, instructions, { legacyPath } = {}) {
+  if (typeof value[key] !== "string") return false;
+  const current = value[key];
+  const stripped = stripDocdexBlocks(current);
+  if (normalizeInstructionText(stripped) !== normalizeInstructionText(current)) {
+    if (!stripped) {
+      delete value[key];
+    } else {
+      value[key] = stripped;
+    }
     return true;
   }
-  if (typeof value[key] === "string") {
-    if (value[key] === location) return false;
-    value[key] = [value[key], location];
+  const normalized = normalizeInstructionText(instructions);
+  if (current === normalized || (legacyPath && current === legacyPath)) {
+    delete value[key];
     return true;
   }
-  value[key] = { [location]: true };
-  return true;
+  return false;
 }
 
-function upsertVsCodeInstructions(pathname, instructions, instructionsDir) {
+function upsertVsCodeInstructions(pathname, instructions, legacyPath) {
   const { value } = readJson(pathname);
   if (typeof value !== "object" || value == null || Array.isArray(value)) return false;
   const normalized = normalizeInstructionText(instructions);
   if (!normalized) return false;
   let updated = false;
-  if (upsertVsCodeInstructionKey(value, "github.copilot.chat.codeGeneration.instructions", instructions)) {
-    updated = true;
-  }
-  if (upsertVsCodeInstructionKey(value, "copilot.chat.codeGeneration.instructions", instructions)) {
+  if (upsertVsCodeInstructionKey(value, "chat.instructions", instructions)) {
     updated = true;
   }
-  if (value["github.copilot.chat.codeGeneration.useInstructionFiles"] !== true) {
-    value["github.copilot.chat.codeGeneration.useInstructionFiles"] = true;
+  if (removeVsCodeInstructionKey(value, "github.copilot.chat.codeGeneration.instructions", instructions)) {
     updated = true;
   }
-  if (upsertVsCodeInstructionLocations(value, instructionsDir)) {
+  if (
+    removeVsCodeInstructionKey(value, "copilot.chat.codeGeneration.instructions", instructions, {
+      legacyPath
+    })
+  ) {
     updated = true;
   }
   if (!updated) return false;
@@ -1169,11 +1181,18 @@ function clientInstructionPaths() {
   const aiderConfig = path.join(home, ".aider.conf.yml");
   const gooseConfig = path.join(home, ".config", "goose", "config.yaml");
   const openInterpreterConfig = path.join(home, ".openinterpreter", "profiles", "default.yaml");
+  const geminiInstructions = path.join(userProfile, ".gemini", "GEMINI.md");
+  const claudeInstructions = path.join(userProfile, ".claude", "CLAUDE.md");
+  const cursorAgents = path.join(userProfile, ".cursor", "agents.md");
+  const cursorAgentsUpper = path.join(userProfile, ".cursor", "AGENTS.md");
   const codexAgents = path.join(userProfile, ".codex", "AGENTS.md");
   switch (process.platform) {
     case "win32":
       return {
-        claude: path.join(appData, "Claude", "claude_desktop_config.json"),
+        gemini: geminiInstructions,
+        claude: claudeInstructions,
+        cursorAgents,
+        cursorAgentsUpper,
         continue: continueJson,
         continueYaml,
         continueYml,
@@ -1192,7 +1211,10 @@ function clientInstructionPaths() {
       };
     case "darwin":
       return {
-        claude: path.join(home, "Library", "Application Support", "Claude", "claude_desktop_config.json"),
+        gemini: geminiInstructions,
+        claude: claudeInstructions,
+        cursorAgents,
+        cursorAgentsUpper,
         continue: continueJson,
         continueYaml,
         continueYml,
@@ -1211,7 +1233,10 @@ function clientInstructionPaths() {
       };
     default:
       return {
-        claude: path.join(home, ".config", "Claude", "claude_desktop_config.json"),
+        gemini: geminiInstructions,
+        claude: claudeInstructions,
+        cursorAgents,
+        cursorAgentsUpper,
         continue: continueJson,
         continueYaml,
         continueYml,
@@ -1295,18 +1320,22 @@ function applyAgentInstructions({ logger } = {}) {
   };
 
   if (paths.vscodeGlobalInstructions) {
-    safeApply("vscode-global", () =>
-      upsertPromptFile(paths.vscodeGlobalInstructions, instructions, { prepend: true })
-    );
+    safeApply("vscode-global-cleanup", () => removePromptFile(paths.vscodeGlobalInstructions));
   }
   if (paths.vscodeInstructionsFile) {
-    safeApply("vscode-instructions-file", () =>
-      upsertPromptFile(paths.vscodeInstructionsFile, instructions, { prepend: true })
+    safeApply("vscode-instructions-file-cleanup", () =>
+      removePromptFile(paths.vscodeInstructionsFile)
     );
   }
-  if (paths.vscodeSettings && paths.vscodeInstructionsDir) {
+  if (paths.cursorAgents) {
+    safeApply("cursor-legacy-cleanup", () => removePromptFile(paths.cursorAgents));
+  }
+  if (paths.cursorAgentsUpper) {
+    safeApply("cursor-legacy-upper-cleanup", () => removePromptFile(paths.cursorAgentsUpper));
+  }
+  if (paths.vscodeSettings) {
     safeApply("vscode-settings", () =>
-      upsertVsCodeInstructions(paths.vscodeSettings, instructions, paths.vscodeInstructionsDir)
+      upsertVsCodeInstructions(paths.vscodeSettings, instructions, paths.vscodeGlobalInstructions)
     );
   }
   if (paths.windsurfGlobalRules) {
@@ -1321,6 +1350,9 @@ function applyAgentInstructions({ logger } = {}) {
   if (paths.claude) {
     safeApply("claude", () => upsertClaudeInstructions(paths.claude, instructions));
   }
+  if (paths.gemini) {
+    safeApply("gemini", () => upsertPromptFile(paths.gemini, instructions));
+  }
   const continueYamlExists =
     (paths.continueYaml && fs.existsSync(paths.continueYaml)) ||
     (paths.continueYml && fs.existsSync(paths.continueYml));

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "docdex",
-  "version": "0.2.24",
+  "version": "0.2.25",
   "mcpName": "io.github.bekirdag/docdex",
   "description": "Local-first documentation and code indexer with HTTP/MCP search, AST, and agent memory.",
   "bin": {