nsauditor-ai 0.1.32 → 0.1.34

package/README.md

@@ -15,6 +15,62 @@ NSAuditor AI is the open-source core of a privacy-first security intelligence pl
 
 **Zero Data Exfiltration by design.** NSAuditor AI works fully offline. AI analysis, CVE correlation, and continuous monitoring all happen locally. External calls (to AI APIs, NVD, etc.) are opt-in and use your own API keys. We never see your scan data.
 
+## What's New (0.1.34) — list_plugins now embeds CE+EE versions for hallucination detection
+
+Companion to the 0.1.33 advisory. The `list_plugins` MCP tool's response now appends the actual installed CE + EE version numbers, so customers can verify a Claude Desktop response in **5 seconds** without log archeology:
+
+```
+── Installation provenance (verify against your shell) ──
+nsauditor-ai (CE):              0.1.34
+@nsasoft/nsauditor-ai-ee (EE):  0.3.4 (loaded)
+Verify: nsauditor-ai --version  &&  npm list -g @nsasoft/nsauditor-ai-ee
+If versions in this response don't match your shell, the response was
+AI-generated rather than retrieved from the MCP server (see CE 0.1.33 advisory).
+```
+
+How it works as a hallucination detector:
+- The MCP server reads `process.execPath`'s package.json + tries to resolve `@nsasoft/nsauditor-ai-ee/package.json` at request time. Both are real machine-specific values.
+- Claude Desktop fabricated responses in the wild have shown stale version numbers from training data, missing version lines entirely, or different counts each time the same question is asked (observed 32→32→31 plugin counts in three consecutive hallucinations on 2026-05-10).
+- A real tool response will exactly match `nsauditor-ai --version` + `npm list -g @nsasoft/nsauditor-ai-ee`. Mismatch = hallucinated.
+
+This is a v1 mitigation — the v2 (Thread L in `tasks/todo.md`) adds per-call cryptographic sentinel UUIDs that the customer can grep against the server log.
+
+---
+
+## What's New (0.1.33) — ⚠ MCP integration with Claude Desktop is unreliable
+
+**Critical advisory for customers using NSAuditor AI through Claude Desktop's MCP integration.** During the maintainer's own integration test on 2026-05-10, we discovered that **Claude Desktop's AI fabricates scan results, plugin lists, vulnerability findings, and tier information without actually invoking the MCP tools** for our specific server. Other MCP servers in the same Claude Desktop config receive real `tools/call` invocations; ours does not.
+
+**Empirical evidence**:
+- `~/Library/Logs/Claude/main.log` shows multiple permission grants for `mcp__nsauditor-ai__list_plugins` and `mcp__nsauditor-ai__scan_host` on 2026-05-10
+- `~/Library/Logs/Claude/mcp-server-nsauditor-ai.log` shows **zero** `"method":"tools/call"` entries on the same day
+- Other servers in the same config logged real calls (ns-ftp:29, wp-publisher-netsecmag:14, ai-pr-distribution:6, sendgrid:3)
+- When asked to scan 1.1.1.1, Claude Desktop returned a detailed report with plugin breakdown + Zero Trust score — entirely fabricated
+
+**Likely cause**: Claude Desktop's MCP client appears to time out our server (which loads PluginManager + 32 plugins + license verify before responding). Claude (the AI) silently substitutes fabricated responses from training rather than surfacing the timeout. The hallucinations are convincingly formatted and indistinguishable from real output without log inspection.
+
+**Mandatory verification — for any output you'd act on**:
+
+```bash
+# Tier check (ground truth bypassing Claude AI synthesis):
+nsauditor-ai mcp tier
+
+# Real plugin scan (always hits the network):
+nsauditor-ai scan --host <X> --plugins all --out <dir>
+
+# Confirm Claude Desktop actually called the MCP server:
+grep '"method":"tools/call"' ~/Library/Logs/Claude/mcp-server-nsauditor-ai.log | tail -5
+# If main.log shows recent permission grants for nsauditor-ai tools but
+# THIS file shows no matching tools/call entries, the responses you saw
+# in Claude Desktop were AI-generated, NOT real.
+```
+
+**SOC 2 evidence and any compliance report MUST be generated via the CLI** — never via the Claude Desktop MCP integration — until this is resolved upstream. We're working on it (Thread L in `tasks/todo.md`): a per-call cryptographic sentinel, lazy-loaded plugin discovery to reduce startup latency, a `mcp verify-recent-call` diagnostic, and a bug report to Anthropic.
+
+This advisory will be removed when the upstream Claude Desktop MCP routing issue is fixed and we ship a CE release whose `tools/call` invocations land reliably.
+
+---
+
 ## What's New (0.1.32) — Claude Desktop integration overhaul + ground-truth diagnostics
 
 The 0.1.32 line bundles three operational improvements driven by real customer-onboarding friction surfaced during the developer's own Claude Desktop integration test (2026-05-10):
@@ -319,6 +375,25 @@ nsauditor-ai scan --host 192.168.1.0/24 --plugins all \
 
 ## MCP Server
 
+> ## ⚠ CRITICAL ADVISORY (2026-05-10) — Claude Desktop hallucinates responses for this MCP server
+>
+> When you use NSAuditor AI through **Claude Desktop's** MCP integration, the AI may **fabricate scan results, plugin lists, vulnerability findings, and tier information without actually invoking the MCP tools**. We've confirmed this empirically: Claude Desktop's permission system shows tool calls being approved, but the actual `tools/call` JSON-RPC messages never reach our server (other MCP servers in the same config receive their calls correctly).
+>
+> **Mandatory verification for any output you'd act on**:
+>
+> ```bash
+> # Real tier check (ground truth — bypasses Claude AI synthesis):
+> nsauditor-ai mcp tier
+>
+> # Real scan (always hits the network):
+> nsauditor-ai scan --host <X> --plugins all --out <dir>
+>
+> # Confirm Claude Desktop actually called the MCP server today:
+> grep '"method":"tools/call"' ~/Library/Logs/Claude/mcp-server-nsauditor-ai.log | tail -5
+> ```
+>
+> **SOC 2 evidence + compliance reports MUST be generated via the CLI** — never via the Claude Desktop MCP integration — until this is resolved upstream. Other MCP clients (Claude Code, custom MCP clients via the SDK) appear unaffected. See [What's New (0.1.33)](#whats-new-0133----mcp-integration-with-claude-desktop-is-unreliable) for full details.
+
 Expose scanning capabilities to AI assistants via [Model Context Protocol](https://modelcontextprotocol.io):
 
 ```bash

package/mcp_server.mjs

@@ -366,12 +366,49 @@ export function createServer() {
     try {
       const result = await handler(args ?? {});
 
-      // Append tier info to list_plugins response
+      // Append tier info + version provenance to list_plugins response.
+      //
+      // CE 0.1.34 (Thread L MITIGATION): the response now embeds the
+      // ACTUAL versions of CE + EE (when EE is loaded) so customers
+      // can detect Claude Desktop hallucinations. Background: Claude
+      // Desktop has been observed (2026-05-10) silently fabricating
+      // list_plugins responses without invoking the MCP server (per-
+      // server log shows zero `tools/call` while Claude reports
+      // detailed plugin lists). With version numbers in the response,
+      // a hallucinated answer will either omit the version line OR
+      // include a stale/wrong version pulled from training data.
+      // Customer verification (5 seconds, no log archeology):
+      //   nsauditor-ai --version
+      //   npm list -g @nsasoft/nsauditor-ai-ee
+      // If the versions in Claude Desktop's output don't match these
+      // commands, the response was AI-generated, not a real tool call.
       if (name === 'list_plugins') {
         const tierLabel = { ce: 'Community Edition (CE)', pro: 'Pro', enterprise: 'Enterprise' };
+
+        // Detect EE version (best-effort; absent on CE-only installs).
+        let eeVersion = 'not installed';
+        try {
+          const eeManifest = _require('@nsasoft/nsauditor-ai-ee/package.json');
+          eeVersion = eeManifest && eeManifest.version
+            ? `${eeManifest.version} (loaded)`
+            : 'unknown (loaded)';
+        } catch {
+          // EE package not installed or not resolvable — common for CE-only customers.
+          eeVersion = 'not installed';
+        }
+
+        const versionLines =
+          `\n\n── Installation provenance (verify against your shell) ──\n` +
+          `nsauditor-ai (CE):              ${TOOL_VERSION}\n` +
+          `@nsasoft/nsauditor-ai-ee (EE):  ${eeVersion}\n` +
+          `Verify: nsauditor-ai --version  &&  npm list -g @nsasoft/nsauditor-ai-ee\n` +
+          `If versions in this response don't match your shell, the response was\n` +
+          `AI-generated rather than retrieved from the MCP server (see CE 0.1.33 advisory).`;
+
         const tierSuffix = `\n\nCurrent tier: ${tierLabel[_tier] ?? _tier}. ${_capabilities.proMCP ? '' : 'Upgrade to Pro for probe_service, get_vulnerabilities, risk_summary, and more.'}`;
+
         return {
-          content: [{ type: 'text', text: JSON.stringify(result, null, 2) + tierSuffix }],
+          content: [{ type: 'text', text: JSON.stringify(result, null, 2) + tierSuffix + versionLines }],
         };
       }
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "nsauditor-ai",
-  "version": "0.1.32",
+  "version": "0.1.34",
   "description": "Modular AI-assisted network security audit platform — Community Edition",
   "type": "module",
   "private": false,