nsauditor-ai 0.1.34 → 0.1.36

package/README.md

@@ -15,6 +15,68 @@ NSAuditor AI is the open-source core of a privacy-first security intelligence pl
 
 **Zero Data Exfiltration by design.** NSAuditor AI works fully offline. AI analysis, CVE correlation, and continuous monitoring all happen locally. External calls (to AI APIs, NVD, etc.) are opt-in and use your own API keys. We never see your scan data.
 
+## What's New (0.1.36) — cryptographic per-call sentinel UUID (hallucination becomes mathematically detectable)
+
+The version-block comparison shipped in 0.1.34/0.1.35 catches lazy hallucinations, but a sufficiently capable AI client can still copy a previously-seen version block from chat context and pass off a fabricated response. **0.1.36 closes that gap** with a per-call cryptographic sentinel that the AI cannot fake.
+
+**How it works:**
+- Each `tools/call` invocation mints a fresh server-side UUID via Node's `crypto.randomUUID()`.
+- The UUID is appended to the response text under a `── Verified MCP call ──` footer.
+- The same UUID is persisted to `~/.nsauditor/mcp-calls.log` (mode 0600, JSON-per-line) **before** the response is returned.
+- A new CLI subcommand `nsauditor-ai mcp verify-call <uuid>` greps the log:
+  - **Found** → the UUID was issued by your local MCP server, so the response bearing it is genuine.
+  - **Not found** → the UUID was never issued, so the entire response was fabricated by the AI client.
+
+**Customer verification workflow (10 seconds):**
+
+```bash
+# 1. In Claude Desktop, ask Claude to use any MCP tool (e.g., list_plugins).
+# 2. The response ends with:
+#       ── Verified MCP call ──
+#       call_id: 3f8a1b22-7e44-4c91-9d62-12bd0a4f5e91
+#       Verify: nsauditor-ai mcp verify-call 3f8a1b22-7e44-4c91-9d62-12bd0a4f5e91
+# 3. Run that exact verify command in your terminal:
+nsauditor-ai mcp verify-call 3f8a1b22-7e44-4c91-9d62-12bd0a4f5e91
+# ✓ Verified MCP call → genuine
+# ✗ call_id not found  → fabricated (response was AI-generated, not from the MCP server)
+```
+
+This makes the hallucination detection unfakeable in principle: the AI client has no access to your local Node `crypto.randomUUID()` output, and the sentinel is generated **at the moment the call hits the server** — there's no way to forge a UUID that will appear in a log file the client cannot read or write.
+
+The 0.1.34/0.1.35 version-block check remains as the first line of defense (instant visual mismatch). The 0.1.36 UUID is the cryptographic ground truth for any response you'd act on.
+
+`scan_host`, `probe_service`, `get_vulnerabilities`, and `list_plugins` all mint sentinels — even Pro-tier denials carry a UUID so customers can prove the call reached the server.
+
+```bash
+npm install -g nsauditor-ai@0.1.36
+```
+
+---
+
+## What's New (0.1.35) — CLI provenance footer matches MCP response (so the comparison actually works)
+
+0.1.34 added the version-provenance block to the MCP server's `list_plugins` response, but **the CLI baseline (`license --plugins` / `license --status`) didn't show versions** — so customers couldn't easily compare. 0.1.35 fixes that asymmetry.
+
+Both CLI commands now emit an identical provenance block:
+
+```
+── Installation provenance ──
+  nsauditor-ai (CE):              0.1.35
+  @nsasoft/nsauditor-ai-ee (EE):  0.3.4 (loaded)
+```
+
+**Customer hallucination-detection workflow (5 seconds, no log archeology):**
+
+1. In Claude Desktop: ask "list plugins" → receive a response that should end with the provenance block
+2. In your terminal: run `nsauditor-ai license --plugins`
+3. Compare the two `── Installation provenance ──` blocks character-for-character
+4. **Match** → real MCP `tools/call` happened, response is trustworthy
+5. **Mismatch / missing block** → Claude fabricated the response (see CE 0.1.33 advisory)
+
+This is the v1 mitigation; the v2 (Thread L Phase 2) adds per-call cryptographic sentinel UUIDs that the customer can grep against the server log directly. v1 catches the common case where Claude either omits the block entirely (unlikely to fabricate the new structure verbatim) or includes a stale version pulled from training data.
+
+---
+
 ## What's New (0.1.34) — list_plugins now embeds CE+EE versions for hallucination detection
 
 Companion to the 0.1.33 advisory. The `list_plugins` MCP tool's response now appends the actual installed CE + EE version numbers, so customers can verify a Claude Desktop response in **5 seconds** without log archeology:
@@ -379,20 +441,23 @@ nsauditor-ai scan --host 192.168.1.0/24 --plugins all \
 >
 > When you use NSAuditor AI through **Claude Desktop's** MCP integration, the AI may **fabricate scan results, plugin lists, vulnerability findings, and tier information without actually invoking the MCP tools**. We've confirmed this empirically: Claude Desktop's permission system shows tool calls being approved, but the actual `tools/call` JSON-RPC messages never reach our server (other MCP servers in the same config receive their calls correctly).
 >
-> **Mandatory verification for any output you'd act on**:
+> **Mandatory verification for any output you'd act on (NEW in 0.1.36 — works for any MCP client):**
 >
 > ```bash
-> # Real tier check (ground truth — bypasses Claude AI synthesis):
+> # Cryptographic ground truth: copy the call_id from the response footer
+> # ("── Verified MCP call ──") and run:
+> nsauditor-ai mcp verify-call <call_id>
+> # ✓ Verified MCP call → genuine, response is trustworthy
+> # ✗ call_id not found → fabricated, IGNORE the response
+>
+> # Real tier check (bypasses Claude AI synthesis):
 > nsauditor-ai mcp tier
 >
-> # Real scan (always hits the network):
+> # Real scan (always hits the network, no MCP client involved):
 > nsauditor-ai scan --host <X> --plugins all --out <dir>
->
-> # Confirm Claude Desktop actually called the MCP server today:
-> grep '"method":"tools/call"' ~/Library/Logs/Claude/mcp-server-nsauditor-ai.log | tail -5
 > ```
 >
-> **SOC 2 evidence + compliance reports MUST be generated via the CLI** — never via the Claude Desktop MCP integration — until this is resolved upstream. Other MCP clients (Claude Code, custom MCP clients via the SDK) appear unaffected. See [What's New (0.1.33)](#whats-new-0133----mcp-integration-with-claude-desktop-is-unreliable) for full details.
+> **SOC 2 evidence + compliance reports MUST be generated via the CLI** — never via the Claude Desktop MCP integration — unless every response you act on has a verified call_id. Other MCP clients (Claude Code, custom MCP clients via the SDK) appear unaffected. See [What's New (0.1.36)](#whats-new-0136--cryptographic-per-call-sentinel-uuid-hallucination-becomes-mathematically-detectable) for the cryptographic mitigation and [What's New (0.1.33)](#whats-new-0133----mcp-integration-with-claude-desktop-is-unreliable) for the original advisory.
 
 Expose scanning capabilities to AI assistants via [Model Context Protocol](https://modelcontextprotocol.io):
 
@@ -543,7 +608,16 @@ claude mcp add nsauditor-ai \
 
 > Use the `list_plugins` MCP tool right now and show me the raw tool response verbatim, including the exact text after the JSON.
 
-Then check the MCP log to verify a real call happened:
+Then verify a real call happened. The 0.1.36+ way (works for any client, not just Claude Desktop):
+
+```bash
+# Copy the call_id from the response footer that Claude returned, then:
+nsauditor-ai mcp verify-call <call_id>
+# ✓ Verified MCP call → genuine, response is trustworthy
+# ✗ call_id not found → fabricated, IGNORE the response
+```
+
+The pre-0.1.36 fallback (Claude Desktop log archeology):
 
 ```bash
 grep '"method":"tools/call"' ~/Library/Logs/Claude/mcp-server-nsauditor-ai.log | tail -5

package/cli.mjs

@@ -5,8 +5,10 @@ import { buildHtmlReport } from './utils/report_html.mjs';
 import fsp from 'node:fs/promises';
 import { dirname } from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { createRequire } from 'node:module';
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
+const _require = createRequire(import.meta.url);
 import path from 'node:path';
 import { platform } from 'node:os';
 import { openaiSimplePrompt, openaiPrompt as openaiProPrompt, openaiPromptOptimized } from './utils/prompts.mjs';
@@ -931,6 +933,20 @@ Docs: https://www.nsauditor.com/ai/   |   Pricing: https://www.nsauditor.com/ai/
           console.log('\n→ Start a free 14-day Pro trial: https://www.nsauditor.com/ai/trial');
         }
       }
+      // CE 0.1.35 (Thread L mitigation v2): version provenance footer
+      // matches the MCP server's list_plugins suffix exactly. Customer
+      // verification flow: read versions in Claude Desktop's MCP
+      // response → compare against `license --status` output here.
+      // Mismatch ⇒ Claude hallucinated.
+      let _eeVersion = 'not installed';
+      try {
+        const ee = _require('@nsasoft/nsauditor-ai-ee/package.json');
+        _eeVersion = ee && ee.version ? `${ee.version} (loaded)` : 'unknown (loaded)';
+      } catch { /* CE-only — fine */ }
+      console.log('');
+      console.log('── Installation provenance ──');
+      console.log(`  nsauditor-ai (CE):              ${TOOL_VERSION}`);
+      console.log(`  @nsasoft/nsauditor-ai-ee (EE):  ${_eeVersion}`);
     } else if (rawArgs.includes('--capabilities')) {
       const tier = getTierFromEnv();
       const caps = resolveCapabilities(tier);
@@ -1004,6 +1020,24 @@ Docs: https://www.nsauditor.com/ai/   |   Pricing: https://www.nsauditor.com/ai/
         console.log('');
         console.log(`  ${totalRendered} plugin${totalRendered === 1 ? '' : 's'} total · current tier: ${tier}`);
       }
+
+      // CE 0.1.35 (Thread L mitigation v2): emit installation provenance
+      // identical in shape to the MCP server's list_plugins suffix.
+      // Customers comparing Claude Desktop's MCP response against the
+      // CLI baseline now see the SAME version block in both places.
+      // Mismatch → Claude hallucinated. Match → real tool call.
+      const ceVersion = TOOL_VERSION;
+      let eeVersion = 'not installed';
+      try {
+        const eeManifest = _require('@nsasoft/nsauditor-ai-ee/package.json');
+        eeVersion = eeManifest && eeManifest.version
+          ? `${eeManifest.version} (loaded)`
+          : 'unknown (loaded)';
+      } catch { /* CE-only install — fine */ }
+      console.log('');
+      console.log('── Installation provenance ──');
+      console.log(`  nsauditor-ai (CE):              ${ceVersion}`);
+      console.log(`  @nsasoft/nsauditor-ai-ee (EE):  ${eeVersion}`);
     } else if (rawArgs.includes('install')) {
       // CE-0.1.30.4 — install command. Verify the JWT FIRST, then persist
       // to a platform-appropriate location (macOS Keychain / file).
@@ -1505,6 +1539,70 @@ Docs: https://www.nsauditor.com/ai/   |   Pricing: https://www.nsauditor.com/ai/
           console.log(`  ⚠ ${MCP_AUTH_DISABLE_ENV_VAR}=1 is set — server will start without auth.`);
         }
       }
+    } else if (subCmd === 'verify-call') {
+      // CE 0.1.36 (Thread L Phase 2): cryptographic ground truth for
+      // "did Claude actually call the MCP server, or hallucinate a
+      // response?" Server mints a fresh UUID per tools/call, embeds it
+      // in the response text, AND appends to ~/.nsauditor/mcp-calls.log.
+      // Customer pastes the UUID here; we grep the log. UUID present →
+      // proven real call. UUID absent → fabricated (or log was rotated/
+      // deleted; we say "unverifiable" rather than "fake").
+      const { readFile, stat } = await import('node:fs/promises');
+      const { join: _join } = await import('node:path');
+      const { homedir: _homedir } = await import('node:os');
+      const logPath = _join(_homedir(), '.nsauditor', 'mcp-calls.log');
+      const uuid = rawArgs[2];
+      if (!uuid) {
+        console.error('Usage: nsauditor-ai mcp verify-call <uuid>');
+        console.error('  Paste the call_id from the MCP tool response footer.');
+        process.exit(2);
+      }
+      // Conservative UUID v4 shape check — avoid grepping the log with
+      // arbitrary user input.
+      if (!/^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$/.test(uuid)) {
+        console.error(`✗ Not a valid UUID: ${uuid}`);
+        console.error('  Expected format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx');
+        process.exit(2);
+      }
+      let logExists = false;
+      try { await stat(logPath); logExists = true; } catch { /* missing */ }
+      if (!logExists) {
+        console.log(`✗ No MCP call log at ${logPath}`);
+        console.log('  Either: (a) MCP server has never been invoked from this account, or');
+        console.log('          (b) the log was deleted/rotated.');
+        console.log('  Trigger one real call (e.g., ask Claude "use list_plugins") then retry.');
+        process.exit(1);
+      }
+      const raw = await readFile(logPath, 'utf8');
+      // Look for an exact JSON-string match on call_id to avoid prefix collisions.
+      const needle = `"call_id":"${uuid.toLowerCase()}"`;
+      const lines = raw.split('\n').filter((l) => l.includes(needle));
+      if (lines.length === 0) {
+        console.log(`✗ call_id not found in ${logPath}`);
+        console.log('');
+        console.log(`  ${uuid}`);
+        console.log('');
+        console.log('  This UUID was NOT issued by this MCP server. Most likely cause:');
+        console.log('  Claude Desktop fabricated the response without invoking the server.');
+        console.log('  (See README §"Verifying that Claude actually called the MCP server".)');
+        process.exit(1);
+      }
+      try {
+        const entry = JSON.parse(lines[lines.length - 1]);
+        console.log(`✓ Verified MCP call`);
+        console.log(`  call_id: ${entry.call_id}`);
+        console.log(`  tool:    ${entry.tool}`);
+        console.log(`  ts:      ${entry.ts}`);
+        console.log(`  log:     ${logPath}`);
+        console.log('');
+        console.log('  This UUID was issued by the local MCP server, so the response');
+        console.log('  bearing it was a genuine tool call (not a hallucination).');
+        process.exit(0);
+      } catch {
+        console.log(`✓ Verified MCP call (matched ${lines.length} log line(s) for this UUID)`);
+        console.log(`  log: ${logPath}`);
+        process.exit(0);
+      }
     } else {
       console.log('Usage:');
       console.log('  nsauditor-ai mcp install-key            Generate a new key, persist, print Claude config');
@@ -1513,6 +1611,7 @@ Docs: https://www.nsauditor.com/ai/   |   Pricing: https://www.nsauditor.com/ai/
       console.log('  nsauditor-ai mcp rotate-key             Replace the stored key with a fresh one');
       console.log('  nsauditor-ai mcp status                 Show storage source without revealing the key');
       console.log('  nsauditor-ai mcp tier                   Print actual MCP server tier (ground truth, bypasses Claude AI synthesis)');
+      console.log('  nsauditor-ai mcp verify-call <uuid>     Prove a tool response came from the real MCP server (not Claude hallucination)');
       console.log('');
       console.log('Environment variables:');
       console.log(`  ${MCP_AUTH_ENV_VAR}        Read by mcp_server.mjs at startup; client supplies via Claude config`);

package/mcp_server.mjs

@@ -8,8 +8,11 @@
 //   import { createServer, toolHandlers } from './mcp_server.mjs'  — for testing
 
 import { createRequire } from 'node:module';
-import { dirname } from 'node:path';
+import { dirname, join } from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { homedir } from 'node:os';
+import { randomUUID } from 'node:crypto';
+import { appendFile, mkdir, chmod } from 'node:fs/promises';
 
 const __dirname = dirname(fileURLToPath(import.meta.url));
 import { Server } from '@modelcontextprotocol/sdk/server/index.js';
@@ -99,6 +102,56 @@ export function _setValidateHost(fn) {
   _validateHostFn = fn ?? validateHost;
 }
 
+// ---------------------------------------------------------------------------
+// Per-call cryptographic sentinel (CE 0.1.36 — Thread L Phase 2)
+// ---------------------------------------------------------------------------
+//
+// Why this exists: even after CE 0.1.34 embedded the resolved tier and CE 0.1.35
+// added a CLI provenance footer, Claude Desktop was empirically observed
+// (2026-05-09, kankanyan@gmail.com) fabricating list_plugins responses
+// WITHOUT routing to this server (per-server log: 0 tools/call entries
+// while other configured MCP servers received 50+ in the same session).
+// A fabricated response can copy any text it has seen — including version
+// numbers from the CLI footer. The only thing it cannot copy is a
+// random UUID generated server-side AT THE MOMENT of the call.
+//
+// Each tool invocation gets a fresh UUID. The UUID is:
+//   1. Embedded in the response text (Claude cannot omit; it's the payload)
+//   2. Persisted to ~/.nsauditor/mcp-calls.log (append-only, mode 0600)
+//
+// Customer verification: paste the UUID from Claude into
+//   nsauditor-ai mcp verify-call <uuid>
+// If it appears in the local log → real call. If not → hallucinated.
+// (See cli.mjs `verify-call` subcommand.)
+const MCP_CALL_LOG_PATH = join(homedir(), '.nsauditor', 'mcp-calls.log');
+
+async function recordToolCall(toolName) {
+  const callId = randomUUID();
+  const ts = new Date().toISOString();
+  // Best-effort: a log-write failure must not break the customer's tool
+  // call. Verification will simply fail-closed (UUID-not-found → treat as
+  // unverifiable rather than as proof-of-fake).
+  try {
+    await mkdir(dirname(MCP_CALL_LOG_PATH), { recursive: true });
+    const line = JSON.stringify({ call_id: callId, tool: toolName, ts }) + '\n';
+    await appendFile(MCP_CALL_LOG_PATH, line, { encoding: 'utf8' });
+    // Tighten on first write; chmod is idempotent so cheap to repeat.
+    try { await chmod(MCP_CALL_LOG_PATH, 0o600); } catch { /* non-fatal on Windows */ }
+  } catch (err) {
+    process.stderr.write(`[nsauditor-mcp] call-log write failed: ${err.message}\n`);
+  }
+  return callId;
+}
+
+function appendCallSentinel(text, callId) {
+  return (
+    `${text}\n\n── Verified MCP call ──\n` +
+    `call_id: ${callId}\n` +
+    `Verify (proves Claude actually called this server, not hallucinated):\n` +
+    `  nsauditor-ai mcp verify-call ${callId}`
+  );
+}
+
 // ---------------------------------------------------------------------------
 // Tool definitions (JSON Schema for input validation)
 // ---------------------------------------------------------------------------
@@ -357,10 +410,23 @@ export function createServer() {
       };
     }
 
+    // CE 0.1.36 (Thread L Phase 2): mint a per-call sentinel UUID and
+    // log it BEFORE the Pro-gate so even denials prove the call hit
+    // the server. Fabricated responses cannot include a UUID that
+    // exists in the customer's local log file.
+    const callId = await recordToolCall(name);
+
     // Gate Pro-tier tools at the MCP dispatch layer
     if (name === 'probe_service' || name === 'get_vulnerabilities') {
       const denied = requireProCapability(name);
-      if (denied) return denied;
+      if (denied) {
+        return {
+          ...denied,
+          content: denied.content.map((c) =>
+            c.type === 'text' ? { ...c, text: appendCallSentinel(c.text, callId) } : c,
+          ),
+        };
+      }
     }
 
     try {
@@ -408,16 +474,16 @@ export function createServer() {
         const tierSuffix = `\n\nCurrent tier: ${tierLabel[_tier] ?? _tier}. ${_capabilities.proMCP ? '' : 'Upgrade to Pro for probe_service, get_vulnerabilities, risk_summary, and more.'}`;
 
         return {
-          content: [{ type: 'text', text: JSON.stringify(result, null, 2) + tierSuffix + versionLines }],
+          content: [{ type: 'text', text: appendCallSentinel(JSON.stringify(result, null, 2) + tierSuffix + versionLines, callId) }],
         };
       }
 
       return {
-        content: [{ type: 'text', text: JSON.stringify(result, null, 2) }],
+        content: [{ type: 'text', text: appendCallSentinel(JSON.stringify(result, null, 2), callId) }],
       };
     } catch (err) {
       return {
-        content: [{ type: 'text', text: JSON.stringify({ error: err.message }) }],
+        content: [{ type: 'text', text: appendCallSentinel(JSON.stringify({ error: err.message }), callId) }],
         isError: true,
       };
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "nsauditor-ai",
-  "version": "0.1.34",
+  "version": "0.1.36",
   "description": "Modular AI-assisted network security audit platform — Community Edition",
   "type": "module",
   "private": false,