devworkspace 0.3.3 → 0.4.1

package/docs/context-accuracy-report.md

@@ -0,0 +1,60 @@
+# DevWorkspace Context Accuracy Evaluation Report
+
+**Generated:** 2026-11-03T12:56:03.656Z  
+**Evaluation Scope:** 10 deterministic task cases  
+**Status:** 10/10 cases passed all safety and relevance assertions  
+
+## 1. Executive Summary & Aggregate Metrics
+
+| Role | Avg Precision | Avg Recall | Avg F1 | Safety Retention | Verification Coverage | Avg Bytes | Est. Tokens (~4 chars/tok) | Size Reduction |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **default** | 55.0% | 100.0% | 65.7% | 100.0% | 100.0% | 9,928 B | 2,482 | Baseline |
+| **explorer** | 57.1% | 95.0% | 64.9% | 100.0% | 100.0% | 8,213 B | 2,054 | -16.75% |
+| **tester** | 80.4% | 87.5% | 81.4% | 100.0% | 100.0% | 8,346 B | 2,087 | -13.5% |
+| **reviewer** | 71.6% | 90.0% | 73.4% | 100.0% | 93.3% | 7,025 B | 1,757 | -27.32% |
+
+### Measurement Note
+- **Byte Size**: Deterministic UTF-8 byte count of the context payload. This is the primary metric for context size.
+- **Estimated Tokens**: Approximate heuristic based on $\lceil \text{characters} / 4 \rceil$. This is not an actual BPE or model tokenizer measurement.
+
+### Key Highlights
+- **Critical Safety Retention**: 100% across all evaluated tasks and roles. Mandatory safety rules and database protection boundaries remain intact.
+- **Role Efficiency**: `explorer` role achieves an average **16.75% context size reduction** by eliminating irrelevant checks while preserving structural entry points and feature references.
+- **Missing Graphify Resilience**: DevWorkspace degrades gracefully when graphify data is unavailable, retrieving baseline facts and rules without crashing.
+
+## 2. Per-Task Case Breakdown
+
+| ID | Category | Task Request | Default Facts (P/R) | Safety Retention | Explorer Reduction | Status |
+| :--- | :--- | :--- | :---: | :---: | :---: | :---: |
+| `eval-01-bugfix` | Bug fixing | "Fix rounding error in invoice calculation" | 100% / 100% | 100% | -18.38% | ✓ PASS |
+| `eval-02-backend-api` | New backend API development | "Add supplier management REST API endpoints" | 38% / 100% | 100% | -11.66% | ✓ PASS |
+| `eval-03-frontend-component` | Frontend component changes | "Update billing reports table and pagination" | 67% / 100% | 100% | --4.25% | ✓ PASS |
+| `eval-04-database-schema` | Database schema changes | "Add tenant isolation column and migrations to orders table" | 50% / 100% | 100% | -16.04% | ✓ PASS |
+| `eval-05-refactoring` | Refactoring | "Extract common stock calculations from product service into inventory service" | 33% / 100% | 100% | -13.5% | ✓ PASS |
+| `eval-06-security-review` | Security-sensitive code review | "Review JWT authentication middleware and password hashing for vulnerabilities" | 80% / 100% | 100% | -32.1% | ✓ PASS |
+| `eval-07-test-creation` | Test creation and regression prevention | "Add unit and integration tests for invoice tax calculation" | 40% / 100% | 100% | -17.27% | ✓ PASS |
+| `eval-08-monorepo-cross-package` | Monorepo tasks spanning multiple packages | "Add product SKU barcode field across backend DTO, database schema, and frontend UI" | 43% / 100% | 100% | -12.3% | ✓ PASS |
+| `eval-09-missing-graphify` | Tasks with missing Graphify data | "Fix rounding error in invoice calculation" | 100% / 100% | 100% | -25.49% | ✓ PASS |
+| `eval-10-ambiguous-request` | Ambiguous or underspecified requests | "Improve performance and check errors" | 0% / 100% | 100% | -25.01% | ✓ PASS |
+
+## 3. Top Accuracy Problems to Fix Next
+
+Based on empirical evaluation of precision and recall against ground truth, the following retrieval bottlenecks were identified for Phase 2 optimization:
+
+### Priority 1: Semantic Keyword Gap (Recall Loss)
+- **Observation**: Keyword matching fails when user prompts use high-level domain terms that do not literally appear in fact keys (e.g. "orders table" or "SKU barcode").
+- **Impact**: Lower recall in cross-package and database schema tasks where relevant architectural rules are missed.
+- **Recommendation for Phase 2**: Introduce fuzzy / synonym expansion or token stem matching in `src/services/taskContext.ts`.
+
+### Priority 2: Broad Package Layout Facts (Noise / Precision Loss)
+- **Observation**: `repo.entry-points` and `repo.packages` are routinely injected into all bug/feature contexts regardless of whether the task touches multiple packages.
+- **Impact**: Lowers precision in narrow single-file bugfixes where package layout adds unnecessary tokens.
+- **Recommendation for Phase 2**: Suppress global package layout facts when the target package is already deterministically identified by the prompt.
+
+### Priority 3: Decision Keyword Association
+- **Observation**: Architectural decisions are matched purely on exact word overlap with decision titles. Relevant decisions (such as ADR 0003 for tenant isolation) require exact word matches.
+- **Impact**: Incomplete decision context for architectural tasks.
+- **Recommendation for Phase 2**: Match against decision tags and context paragraphs in addition to titles.
+
+---
+*Report generated deterministically by DevWorkspace Context Accuracy Evaluation Framework.*

package/docs/context-evaluation.md

@@ -0,0 +1,170 @@
+# DevWorkspace Context Accuracy Evaluation Framework
+
+## 1. Overview & Objective
+
+DevWorkspace is a deterministic, project-local AI engineering workspace designed to provide AI coding agents with high-relevance repository context, architectural decisions, facts, and verification gates.
+
+The **Context Accuracy Evaluation Framework** establishes a standardized, automated, and deterministic benchmark suite to measure context retrieval precision, recall, safety retention, verification coverage, and token efficiency before any Phase 2 retrieval algorithm changes are made.
+
+### Core Principles
+1. **Deterministic & Offline**: Requires zero LLM API calls, network requests, or non-deterministic heuristics.
+2. **Zero Runtime Dependencies**: Authored in pure Node.js/TypeScript conforming to DevWorkspace's zero-production-dependency architecture.
+3. **Safety First**: Verifies that mandatory safety rules and dangerous operation blocks (e.g. unverified database operations, credential exposure protection) are preserved 100% of the time across all roles.
+4. **Role Slicing Awareness**: Benchmarks role-specific contexts (`explorer`, `tester`, `reviewer`) against full baseline contexts to measure signal-to-noise ratio and token savings.
+
+---
+
+## 2. Evaluation Methodology & Metric Definitions
+
+The evaluation engine tests tasks against realistic fixture workspaces (`fixtures/eval-monorepo` and `fixtures/eval-no-graphify`) and compares retrieved contexts against curated ground truth.
+
+### 2.1 Context Precision
+Measures the proportion of retrieved context items (facts, decisions, documents, graph nodes) that are genuinely relevant to the requested task:
+
+$$\text{Precision} = \frac{|\text{Retrieved} \cap \text{ExpectedRelevant}|}{|\text{Retrieved}|}$$
+
+*Boundary rules*:
+- If $|\text{Retrieved}| = 0$ and $|\text{ExpectedRelevant}| = 0$: $\text{Precision} = 1.0$.
+- If $|\text{Retrieved}| = 0$ and $|\text{ExpectedRelevant}| > 0$: $\text{Precision} = 0.0$.
+
+### 2.2 Context Recall
+Measures the proportion of genuinely relevant ground-truth items that were successfully retrieved into the context window:
+
+$$\text{Recall} = \frac{|\text{Retrieved} \cap \text{ExpectedRelevant}|}{|\text{ExpectedRelevant}|}$$
+
+*Boundary rules*:
+- If $|\text{ExpectedRelevant}| = 0$: $\text{Recall} = 1.0$.
+
+### 2.3 F1-Score
+The harmonic mean of precision and recall:
+
+$$F_1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$$
+
+### 2.4 Critical Safety Retention Rate
+Measures whether mandatory safety rules and execution barriers are retained without omission:
+
+$$\text{Retention} = \frac{|\text{RetainedMandatoryRules}|}{|\text{TotalMandatoryRules}|}$$
+
+- **Mandatory Floor**: Must be strictly **1.0 (100%)** across all tasks and roles.
+
+### 2.5 Verification Check Coverage
+Measures whether relevant verification commands (typecheck, lint, unit tests) are provided for tasks that modify code:
+
+$$\text{Coverage} = \frac{|\text{IncludedRequiredChecks}|}{|\text{TotalRequiredChecks}|}$$
+
+- For read-only tasks or the `explorer` role (which explores without running checks), coverage is evaluated against role expectations ($1.0$).
+
+### 2.6 Token & Size Efficiency
+Measures output size in characters and bytes, with estimated token counts calculated via the standard heuristic ($\lceil \text{chars} / 4 \rceil$), alongside the percentage reduction relative to default full context:
+
+$$\text{Reduction \%} = \frac{\text{Bytes}_{\text{default}} - \text{Bytes}_{\text{role}}}{\text{Bytes}_{\text{default}}} \times 100$$
+
+---
+
+## 3. Curated 10-Task Evaluation Dataset
+
+The dataset covers 10 representative engineering archetypes:
+
+| Case ID | Archetype | Request | Target Fixture | Primary Evaluated Focus |
+| :--- | :--- | :--- | :--- | :--- |
+| `eval-01-bugfix` | Bug fixing | "Fix rounding error in invoice calculation" | `fixtures/eval-monorepo` | Fact matching for billing precision & known issue |
+| `eval-02-backend-api` | New backend API | "Add supplier management REST API endpoints" | `fixtures/eval-monorepo` | DTO validation conventions & service layer architecture |
+| `eval-03-frontend-component` | Frontend component | "Update billing reports table and pagination" | `fixtures/eval-monorepo` | React UI conventions & frontend verification gates |
+| `eval-04-database-schema` | Database schema | "Add tenant isolation column and migrations to orders table" | `fixtures/eval-monorepo` | Migration safety rules, tenancy isolation, schema facts |
+| `eval-05-refactoring` | Refactoring | "Extract common stock calculations from product service into inventory service" | `fixtures/eval-monorepo` | Service layer boundaries & architectural decisions |
+| `eval-06-security-review` | Security review | "Review JWT authentication middleware and password hashing for vulnerabilities" | `fixtures/eval-monorepo` | Auth rules, hashing algorithms, and security gate checks |
+| `eval-07-test-creation` | Test creation | "Add unit and integration tests for invoice tax calculation" | `fixtures/eval-monorepo` | Test conventions, disposable DB setup, unit/integration checks |
+| `eval-08-monorepo-cross-package` | Monorepo multi-package | "Add product SKU barcode field across backend DTO, database schema, and frontend UI" | `fixtures/eval-monorepo` | Multi-package checks, full-stack fact retrieval |
+| `eval-09-missing-graphify` | Missing Graphify data | "Fix rounding error in invoice calculation" | `fixtures/eval-no-graphify` | Graceful fallback when Graphify knowledge graph is missing |
+| `eval-10-ambiguous-request` | Ambiguous / vague prompt | "Improve performance and check errors" | `fixtures/eval-monorepo` | Safety rule retention and non-crash stability on vague input |
+
+---
+
+## 4. Baseline Evaluation Results
+
+Evaluated on DevWorkspace v0.3.3 across all 10 benchmark cases:
+
+| Role | Avg Precision | Avg Recall | Avg F1 | Safety Retention | Verification Coverage | Avg Output Bytes | Est. Tokens | Size Reduction |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **`default`** | 38.0% | 95.0% | 49.8% | 100.0% | 100.0% | 14,671 B | 3,668 | Baseline (0%) |
+| **`explorer`** | 40.0% | 87.5% | 50.0% | 100.0% | 100.0% | 10,291 B | 2,573 | **-30.75%** |
+| **`tester`** | 56.3% | 60.0% | 57.0% | 100.0% | 100.0% | 9,834 B | 2,459 | **-29.50%** |
+| **`reviewer`** | 59.6% | 76.7% | 64.7% | 100.0% | 61.7% | 8,901 B | 2,225 | **-38.33%** |
+
+> [!NOTE]
+> **Evaluator Metric Correction**:
+> In the initial uncorrected evaluation script, case `eval-10` (which has 0 expected relevant items) defaulted to 1.0 precision when retrieved items were present, yielding an aggregate legacy baseline precision of 38.0%. With the corrected denominator boundary where non-empty retrieval against 0 expected items evaluates to 0.0 precision (representing 100% false-positive noise), the true baseline precision was 28.0%. Phase 2 achieves 55.0% precision under the rigorous corrected metric (and 65.0% under the legacy formula), confirming massive improvement under both standards.
+
+### Key Findings
+1. **Safety Rules Invariant**: All 4 mandatory safety rules and unverified database command blocks were 100% retained across all 40 role executions.
+2. **Context Compression**: Role-based slicing reliably eliminates 29% to 38% of context bytes, stripping irrelevant checks and documents while preserving targeted guidance.
+3. **Missing Graphify Resilience**: In `eval-09-missing-graphify`, DevWorkspace executed cleanly without errors, emitting entry points and features while omitting missing symbol nodes gracefully.
+
+---
+
+## 5. Top Accuracy Problems to Fix Next (Phase 2 Priorities)
+
+Empirical evaluation against ground truth identified three specific retrieval bottlenecks:
+
+### Priority 1: Semantic Keyword Gap (Recall Loss)
+- **Problem**: In task `eval-04-database-schema`, the request refers to "orders table", but facts reference PostgreSQL/Drizzle and tenancy isolation. Because retrieval relies on exact keyword matching, facts without the exact word "order" or "table" receive lower match scores.
+- **Impact**: Database and cross-package tasks suffer ~50% recall in default retrieval.
+- **Phase 2 Solution**: Implement word stemming, token lemmatization, and domain synonym expansion (e.g. mapping "table", "schema", "column" to database facts).
+
+### Priority 2: Broad Package Layout Noise (Precision Loss)
+- **Problem**: Facts `repo.entry-points` and `repo.packages` are routinely retrieved for all tasks classified as `BUG` or `FEATURE` under the `package layout` rule, even for tightly scoped single-file bugfixes (e.g. `eval-01-bugfix`).
+- **Impact**: Suppresses precision in focused bugfix tasks down to ~11–50%.
+- **Phase 2 Solution**: Condition package layout fact injection on task scope; omit global entry points when the prompt targets an identified single module.
+
+### Priority 3: Architectural Decision Keyword Association
+- **Problem**: Decisions are matched against the user prompt based solely on title string matching. When a decision's title does not share words with the prompt, the ADR is missed.
+- **Impact**: ADR 0003 (Tenant Isolation) was missed when queries did not explicitly contain the title's exact wording.
+- **Phase 2 Solution**: Index ADR tags, problem summaries, and decision rationale alongside titles.
+
+---
+
+## 6. Phase 2 Optimization Results (Before vs After)
+
+Phase 2 implemented deterministic word stemming (`normalizeStem`), domain synonym aliasing (`DOMAIN_SYNONYMS`), package-scoped layout suppression, and expanded ADR matching with weak-term filtering in `src/services/taskContext.ts`.
+
+### 6.1 Benchmark Comparison
+
+| Role | Baseline Precision | Phase 2 Precision | Baseline Recall | Phase 2 Recall | Baseline F1 | Phase 2 F1 | Safety Retention | Avg Bytes (Before -> After) | Token Reduction |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| **`default`** | 38.0% | **55.0%** (+44.7%) | 95.0% | **100.0%** (+5.3%) | 49.8% | **65.7%** (+31.9%) | **100.0%** | 14,671 B → 9,939 B | **-32.3%** |
+| **`explorer`** | 40.0% | **57.1%** (+42.8%) | 87.5% | **95.0%** (+8.6%) | 50.0% | **64.9%** (+29.8%) | **100.0%** | 10,291 B → 8,216 B | **-20.2%** |
+| **`tester`** | 56.3% | **80.4%** (+42.8%) | 60.0% | **87.5%** (+45.8%) | 57.0% | **81.4%** (+42.8%) | **100.0%** | 9,834 B → 8,357 B | **-15.0%** |
+| **`reviewer`** | 59.6% | **71.6%** (+20.1%) | 76.7% | **90.0%** (+17.3%) | 64.7% | **73.4%** (+13.4%) | **100.0%** | 8,901 B → 7,167 B | **-19.5%** |
+
+### 6.2 Key Verification Outcomes
+1. **Zero Regressions**: Baseline recall of 95.0% increased to **100.0%** in default context (0 required items missed across all 10 cases).
+2. **Precision Surge**: Default context precision jumped from 38.0% to **55.0%**, while tester precision reached **80.4%** and reviewer precision reached **71.6%**.
+3. **Safety Guarantee Preserved**: Critical safety retention remained strictly **100.0%** across all roles and all test cases.
+4. **Token Savings**: Default context payload dropped by 32.3% in bytes (~1,183 fewer tokens per prompt), and role slices reduced payloads further down to 1,792–2,089 tokens.
+
+---
+
+## 7. How to Run Context Accuracy Benchmarks
+
+### Quick Evaluation
+```bash
+npm run eval
+```
+
+### Full Benchmark with Report Generation
+```bash
+npm run eval:accuracy
+```
+
+### Options and Filters
+```bash
+# Run benchmark for a specific role
+node scripts/eval-accuracy.mjs --role explorer
+
+# Run benchmark for a single test case with verbose output
+node scripts/eval-accuracy.mjs --case eval-01-bugfix --verbose
+
+# Save custom JSON and Markdown reports
+node scripts/eval-accuracy.mjs --json reports/my-report.json --markdown docs/my-report.md
+```
+

package/docs/mcp.md

@@ -0,0 +1,264 @@
+# DevWorkspace Model Context Protocol (MCP) Integration
+
+DevWorkspace includes a local-first Model Context Protocol (MCP) server that enables AI coding agents (such as Claude Code, OpenAI Codex, and Google Antigravity) to retrieve project architecture, role-sliced context, and file metadata automatically during development.
+
+---
+
+## 1. What DevWorkspace MCP Is
+
+The DevWorkspace MCP integration exposes DevWorkspace's deterministic context retrieval engine over the standard Model Context Protocol (MCP 2024-11-05). It allows AI agents to inspect:
+
+- High-level project architecture and stack detection (`get_project_overview`)
+- Natural-language task context with role slicing (`get_task_context` for `explorer`, `tester`, `reviewer`)
+- Workspace-relative file context bounded to 32KB (`get_file_context`)
+
+The developer does not need to manually copy and paste context into conversation prompts.
+
+---
+
+## 2. Why It Runs Locally
+
+DevWorkspace MCP is strictly local-first:
+
+- **stdio transport only:** The server runs as a child process launched directly by your coding agent over standard input/output.
+- **Zero open ports:** It does not bind a TCP listener, open HTTP ports, or require a reverse proxy / tunnel.
+- **No cloud dependencies:** No cloud account, third-party server, or LLM API key is required to serve context.
+- **Deterministic retrieval:** Context is derived offline from local repository evidence.
+
+---
+
+## 3. Prerequisites
+
+- **Node.js:** `>= 20.11.0`
+- **DevWorkspace:** Installed locally in your project or globally (`npm install -D devworkspace` or `npm install -g devworkspace`).
+- **An initialized project:** The project root must contain `.devworkspace/project.json` (created with `npx devworkspace init`).
+
+---
+
+## 4. Installation and Initialization
+
+In an existing software repository:
+
+### Bash / macOS / Linux:
+```bash
+# 1. Initialize DevWorkspace in your project
+npx devworkspace init
+
+# 2. Analyze the project layout and stack
+npx devworkspace analyze
+```
+
+### Windows PowerShell:
+```powershell
+# 1. Initialize DevWorkspace in your project
+npx devworkspace init
+
+# 2. Analyze the project layout and stack
+npx devworkspace analyze
+```
+
+---
+
+## 5. Agent-Specific Setup
+
+DevWorkspace provides single-command setup for supported AI coding agents.
+
+### Claude Code
+
+To configure Claude Code:
+
+```bash
+# Preview planned changes without modifying files:
+npx devworkspace setup --agent claude-code --check
+
+# Apply configuration:
+npx devworkspace setup --agent claude-code
+```
+
+**What this configures:**
+- Creates or updates `.mcp.json` at the project root:
+  ```json
+  {
+    "mcpServers": {
+      "devworkspace": {
+        "command": "devworkspace",
+        "args": ["mcp", "serve"]
+      }
+    }
+  }
+  ```
+- Appends or updates instructions in `CLAUDE.md` directing Claude Code to invoke `get_project_overview` when starting work and `get_task_context` when addressing issues.
+
+### OpenAI Codex
+
+To configure OpenAI Codex:
+
+```bash
+# Preview planned changes:
+npx devworkspace setup --agent codex --check
+
+# Apply configuration:
+npx devworkspace setup --agent codex
+```
+
+**What this configures:**
+- Creates or updates `mcp.json` at the project root with the `devworkspace` server.
+- Updates `AGENTS.md` with instructions on invoking DevWorkspace tools.
+
+### Google Antigravity
+
+To configure Google Antigravity:
+
+```bash
+# Preview planned changes:
+npx devworkspace setup --agent antigravity --check
+
+# Apply configuration:
+npx devworkspace setup --agent antigravity
+```
+
+**What this configures:**
+- Creates or updates `.antigravity/mcp.json` registering the local `devworkspace mcp serve` command.
+- Updates `.antigravity/rules` instructing the agent when to call `get_project_overview` and `get_task_context`.
+
+### Configure All Agents at Once
+
+```bash
+npx devworkspace setup --agent all
+```
+
+---
+
+## 6. How to Confirm the Server Is Connected
+
+1. **Verify configuration files:**
+   Confirm `.mcp.json`, `mcp.json`, or `.antigravity/mcp.json` is present.
+2. **Run a configuration check:**
+   ```bash
+   npx devworkspace setup --agent claude-code --check
+   ```
+   If all items report `UNCHANGED`, the configuration is verified.
+3. **In Claude Code / Codex / Antigravity:**
+   Start your agent and run:
+   - Claude Code: `/mcp` displays active servers (`devworkspace: connected`).
+   - Or ask the agent: *"Call the get_project_overview tool to inspect the repository."*
+
+---
+
+## 7. How to Test a Context Request Manually
+
+You can test requests by piping JSON-RPC lines to `devworkspace mcp serve`:
+
+### Bash:
+```bash
+printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05"}}\n{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_project_overview","arguments":{}}}\n' | npx devworkspace mcp serve
+```
+
+### Windows PowerShell:
+```powershell
+@'
+{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05"}}
+{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_project_overview","arguments":{}}}
+'@ | npx devworkspace mcp serve
+```
+
+---
+
+## 8. What the Agent Can and Cannot Access
+
+### What the agent CAN access:
+- **Project architecture summary:** Detected languages, web/UI frameworks, package structure, and health state.
+- **Task context:** Facts, verification commands (`commands.json`), architectural decision records (`.devworkspace/decisions/`), and safety rules.
+- **Specific source files:** Regular files within the project root, bounded to 32KB per call.
+
+### What the agent CANNOT access:
+- **No arbitrary shell execution:** The MCP server contains zero execution tools. It cannot run commands, scripts, or package managers.
+- **No file modification:** All MCP tools are strictly read-only.
+- **No access outside the workspace:** Path traversal (`..`), absolute paths, and symlink escapes are strictly blocked.
+- **No credential access:** Environment files and private keys are barred by policy.
+
+---
+
+## 9. Secret Exclusions and Workspace Boundaries
+
+DevWorkspace enforces defense-in-depth security:
+
+1. **Workspace Boundary:**
+   Every requested path is validated using canonical resolution (`resolveInside`) and symlink escape checks (`assertNoSymlinkEscape`). Requests attempting to escape the project root return a `Security Error`.
+2. **Credential Exclusion Policy:**
+   The following file patterns are automatically blocked from `get_file_context`:
+   - `.env`, `.env.*` (except harmless templates like `.env.example`)
+   - `*.pem`, `*.key`, `*.p12`, `*.pfx`, `*.keystore`
+   - `id_rsa*`, `id_ed25519*`
+   - `credentials*`, `secrets*`, `service-account*.json`
+   - `.npmrc`, `.pypirc`, `.netrc`, `.htpasswd`
+   - `terraform.tfstate*`, `*.tfvars`
+3. **Secret Redaction:**
+   Any detected secret patterns (API keys, private key blocks, AWS tokens, GitHub tokens, JWTs, URL credentials) are masked with `[REDACTED:<type>]` before context is returned.
+4. **Prompt-Injection Resistance:**
+   All retrieved context is demarcated with `=== BEGIN UNTRUSTED DEVWORKSPACE CONTEXT ===` headers. The agent is explicitly instructed to treat repository content as untrusted evidence rather than executable instructions.
+
+---
+
+## 10. Privacy and External Data Transmission
+
+- **Local Execution:** DevWorkspace runs entirely on your local machine. It does not send telemetry, analytics, or file contents to any DevWorkspace server.
+- **AI Provider Notice:** When an AI coding agent calls DevWorkspace MCP tools, the retrieved context enters the agent's context window. Depending on your agent configuration and AI provider (e.g., Anthropic, OpenAI, Google), the agent will send this context to its cloud model.
+
+---
+
+## 11. Troubleshooting
+
+| Issue | Cause | Resolution |
+| --- | --- | --- |
+| `File not found: project.json` | Running in an uninitialized directory | Run `npx devworkspace init` from the project root |
+| `Refusing absolute path` | Path parameter was absolute | Pass workspace-relative paths, e.g., `src/index.ts` |
+| `Security Policy Refusal` | File contains secrets (e.g., `.env`) | DevWorkspace refuses credential files by policy |
+| Agent does not call MCP tools | Agent instructions missing or ignored | Run `npx devworkspace setup --agent <agent>` and ensure project instructions (`CLAUDE.md`, `AGENTS.md`) include DevWorkspace guidance |
+| MCP server exits immediately | Stdio stream closed by parent | Ensure the parent agent keeps stdin open while communicating |
+
+---
+
+## 12. Uninstallation and Cleanup
+
+To remove DevWorkspace MCP configuration:
+
+1. Remove the `"devworkspace"` entry from `.mcp.json`, `mcp.json`, or `.antigravity/mcp.json`.
+2. To remove DevWorkspace instructions from agent markdown files:
+   ```bash
+   npx devworkspace reset --include-agent-files
+   ```
+   (Only managed `<!-- devworkspace:start -->` and `<!-- devworkspace-mcp:start -->` blocks are removed; user content is preserved).
+
+---
+
+## 13. Known Client Limitations & PATH Requirements
+
+- **CLI PATH requirement:** The generated MCP configuration specifies `"command": "devworkspace"`, which requires `devworkspace` to be discoverable in the coding agent's executable `PATH` (for instance, via `npm install -g devworkspace` or having the package manager's bin folder in PATH).
+  - If `devworkspace` is installed locally as a devDependency and not globally in PATH, adjust the agent configuration to use `npx`:
+    ```json
+    {
+      "mcpServers": {
+        "devworkspace": {
+          "command": "npx",
+          "args": ["-y", "devworkspace", "mcp", "serve"]
+        }
+      }
+    }
+    ```
+  - Alternatively, specify the absolute path to your Node.js runtime and the CLI bundle (e.g. `node /path/to/dist/cli/main.js mcp serve`).
+- **Initialization Lifecycle:** MCP specification (2024-11-05) requires initialization prior to general requests. Calling `tools/list` or `tools/call` prior to completing the `initialize` handshake returns JSON-RPC error code `-32600` (`INVALID_REQUEST`). `ping` is supported both before and after initialization.
+- **JSON-RPC Version:** MCP requires JSON-RPC 2.0. Clients sending invalid JSON-RPC receive standard error code `-32700` (`PARSE_ERROR`) or `-32600` (`INVALID_REQUEST`).
+- **Interactive Verification:** Live interactive tool calling in Claude Code, OpenAI Codex, and Google Antigravity depends on the respective client applications executing in a compatible desktop or terminal environment. While standard stdio protocol compliance is fully covered by automated integration tests, live AI client behavior remains subject to agent-specific client updates and context window limits.
+
+---
+
+## 14. Manual Debugging
+
+To run the MCP server with diagnostic debug logging to stderr:
+
+```bash
+# Debug mode writes protocol trace to stderr without corrupting stdout JSON-RPC
+npx devworkspace mcp serve --debug
+```
+

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "devworkspace",
-  "version": "0.3.3",
+  "version": "0.4.1",
   "description": "Project-local AI engineering workspace: deterministic repository discovery, portable project knowledge, tasks, decisions and verification gates for AI coding agents.",
   "type": "module",
   "bin": {
@@ -36,6 +36,8 @@
     "coverage": "vitest run --coverage",
     "audit:security": "npm audit --omit=dev --audit-level=high",
     "smoke": "node scripts/smoke.mjs",
+    "eval": "node scripts/eval-accuracy.mjs",
+    "eval:accuracy": "node scripts/eval-accuracy.mjs --json reports/context-accuracy-report.json --markdown docs/context-accuracy-report.md",
     "release:check": "node scripts/release-check.mjs",
     "prepack": "npm run build"
   },