mcp-prompt-optimizer 3.7.5 → 3.8.2

package/CHANGELOG.md

@@ -5,6 +5,75 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.8.2] - 2026-08-27
+
+### Added
+- **Surface CE structural warnings in tool output.** The backend `/context-engineer/transform`
+  and `/context-engineer/generate-skill-package` responses now carry a non-fatal `warnings`
+  array (dropped SOP stages, duplicated tables, fabricated headings). `transform_for_framework`
+  and `generate_skill_package` previously discarded it, so a structurally degraded artifact
+  shipped silently. Both now render a `## ⚠️ Structural Warnings` section when the array is
+  non-empty. No change when it is empty (the normal case) — output is byte-identical.
+
+## [3.8.1] - 2026-07-29
+
+### Fixed
+- **Security: the validation cache let any API key inherit a different key's cached identity.**
+  `~/.mcp-cloud-api-cache.json` is a single file per machine, not namespaced per key. Found live
+  while testing against a real registered key: validating key A succeeds and caches its result;
+  minutes later, validating a completely different, invalid key B gets a clean 401 from the
+  backend — but the client's fallback logic didn't check *which* key the cached entry belonged to,
+  and didn't check whether the error was a definitive rejection versus a network outage. It
+  silently returned key A's cached "valid" data and started the server authenticated as A's
+  account, having never actually validated B. Any string passed as `OPTIMIZER_API_KEY` within the
+  cache's 1-hour window (2 hours via the short-term fallback tier) would have worked, as long as
+  *some* real key had been validated on that machine recently.
+  Fixed two ways: (1) a cached entry is only used if its stored `apiKeyPrefix` matches the key
+  currently being validated, and (2) cache fallback is skipped entirely on a 4xx rejection —
+  it's reserved for genuine network/5xx outages of the *same* already-cached key, which was
+  always the intent (the surrounding code already distinguished 4xx from network errors for its
+  error-message hint text; the fallback branch above it just never used that distinction).
+  Added a network-free regression test (`tests/quick-test.js`) that seeds the cache with one
+  key's data, forces a 401 for a different key, and asserts the second key never inherits the
+  first's identity — and a second check confirming the legitimate same-key network-outage
+  fallback still works unchanged.
+
+## [3.8.0] - 2026-07-29
+
+### Added
+- **`optimize_prompt` gains `reasoning_effort` and `execution_shape`**, the two Track C1 controls
+  that previously only reached `/api/v1/optimize` (the WebUI's endpoint) — `/api/v1/mcp/optimize`
+  (what this tool actually calls) never had them wired in at all, a pre-existing architectural
+  gap, not a regression. These matter *more* for a programmatic/agentic caller than a WebUI user:
+  there's no human watching each call to decide whether a prompt is worth paying for deeper
+  reasoning, or to notice a `multi_agent` request quietly running the LLM engine regardless of
+  the prompt's complexity. `multi_agent` is gated the same way on this endpoint as it is on
+  `/optimize` (`_gate_execution_shape`, downgraded to `direct` on tiers without repair access);
+  the response echoes back the *effective* shape that ran, and the formatter now surfaces a
+  downgrade when one happens. `stop_rule` (the third Track C1 control) is deliberately **not**
+  exposed here: this endpoint never runs the quick-eval-then-repair flow that gives `stop_rule`
+  any effect, so accepting it would advertise a guardrail that does nothing. Backend change
+  landed alongside this (`app/api/mcp_router.py`), verified with new endpoint tests covering the
+  free-tier downgrade, the pro-tier non-downgrade, and the team/enterprise-default non-downgrade
+  path (the last one guards against team keys getting silently capped if their tier lookup ever
+  comes back empty) — full backend suite (1740 passed) confirmed no regressions.
+- **`generate_harness_bundle` gains `agent_read_only` and `agent_harness`**, mirroring the two
+  optional fields the backend's `HarnessBundleRequest` already accepts. `agent_harness` lets the
+  caller pick the generated agent.yaml's execution backend (`claude-sdk`/`codex`/`pi`) to match
+  whichever API key they actually have available wherever the bundle runs — without it, the
+  bundle silently defaults per-deploy-target (usually `claude-sdk`) and fails at runtime with a
+  missing-credential error if that's not the key the user holds. `agent_read_only` narrows the
+  generated subagent's tools to Read/Grep/Glob, for audit/review workflows that should never be
+  able to edit or execute anything. Both optional, both already validated server-side (a bad
+  `agent_harness` value gets a clear 422 listing valid choices).
+- **`tests/e2e-stdio-smoke.js`**: black-box test that spawns the published binary as a real
+  subprocess and speaks JSON-RPC over stdio, exactly as an external MCP client would. Confirms
+  clean startup/shutdown behavior for missing/malformed/unregistered keys, including a live round
+  trip to the deployed backend. Full `tools/call` coverage requires `OPTIMIZER_API_KEY` set to a
+  real, backend-registered key — this package has no reachable mock/dev bypass (`developmentMode`
+  is hardcoded `false` in `index.js`; see the `dev`/`dev:mock` npm scripts, which are currently
+  dead for the same reason).
+
 ## [3.7.5] - 2026-07-18
 
 ### Fixed

package/index.js

@@ -187,6 +187,16 @@ class MCPPromptOptimizer {
                     description: "Narrative description of what a successful optimized output achieves (e.g. 'reader understands why churn drives flat revenue even with user growth')."
                   }
                 }
+              },
+              reasoning_effort: {
+                type: "string",
+                enum: ["minimal", "standard", "deep"],
+                description: "How much reasoning to apply: 'minimal' biases toward faster/cheaper routing, 'standard' is the default, 'deep' biases toward the LLM tier for maximum analysis. Most useful when calling this tool programmatically in a loop or pipeline, where there's no human watching each call to decide whether it's worth paying for more depth."
+              },
+              execution_shape: {
+                type: "string",
+                enum: ["direct", "hybrid", "multi_agent"],
+                description: "Execution style, independent of the tier the prompt would normally route to: 'direct' is single-pass, 'hybrid' adds rules+LLM verification, 'multi_agent' forces plan-and-execute with sub-agents regardless of the prompt's complexity. 'multi_agent' is downgraded to 'direct' on tiers without repair access — the response echoes back whichever one actually ran, so you can detect a downgrade."
               }
             },
             required: ["prompt"]
@@ -442,6 +452,24 @@ class MCPPromptOptimizer {
               sop_content: {
                 type: "string",
                 description: "The SOP content to base the harness on (required if no session_id)."
+              },
+              agent_read_only: {
+                type: "boolean",
+                description: (
+                  "Narrow the generated Claude Code subagent's tools to Read/Grep/Glob only. "
+                  + "Set this for audit/review workflows that should never edit or execute anything. "
+                  + "Default: false (full capability)."
+                )
+              },
+              agent_harness: {
+                type: "string",
+                enum: ["claude-sdk", "codex", "pi"],
+                description: (
+                  "Execution backend for the generated agent.yaml: 'claude-sdk' (needs ANTHROPIC_API_KEY), "
+                  + "'codex' (needs OPENROUTER_API_KEY or OPENAI_API_KEY), or 'pi' (needs PI_API_KEY). "
+                  + "Set this to match the API key available wherever the bundle will actually run — "
+                  + "the default is per-deploy-target (usually claude-sdk) and won't know which key you have."
+                )
               }
             },
             required: ["goal"]
@@ -931,10 +959,17 @@ class MCPPromptOptimizer {
         }
       }
 
+      if (args.reasoning_effort) {
+        optimizationPayload.reasoning_effort = args.reasoning_effort;
+      }
+      if (args.execution_shape) {
+        optimizationPayload.execution_shape = args.execution_shape;
+      }
+
       const result = await this.callBackendAPI(ENDPOINTS.OPTIMIZE, optimizationPayload);
-      
+
       const enableBayesian = args.enable_bayesian !== false && this.bayesianOptimizationEnabled;
-      return { content: [{ type: "text", text: this.formatOptimizationResult(result, { detectedContext, enableBayesian }) }] };
+      return { content: [{ type: "text", text: this.formatOptimizationResult(result, { detectedContext, enableBayesian, requestedExecutionShape: args.execution_shape }) }] };
       
     } catch (error) {
       if (error.message.includes('Network') || error.message.includes('DNS') || error.message.includes('timeout') || error.message.includes('Connection')) {
@@ -1254,7 +1289,7 @@ class MCPPromptOptimizer {
         sop_content: args.sop_content, goal: args.goal, framework: args.framework
       });
       const code = result.code || result.content || result.result || JSON.stringify(result, null, 2);
-      return { content: [{ type: "text", text: `# ${args.framework} Implementation\n\n\`\`\`python\n${code}\n\`\`\`\n\n---\n*Transformed by MCP Prompt Optimizer CE*` }] };
+      return { content: [{ type: "text", text: `# ${args.framework} Implementation\n\n\`\`\`python\n${code}\n\`\`\`${this._formatWarnings(result)}\n\n---\n*Transformed by MCP Prompt Optimizer CE*` }] };
     } catch (error) {
       throw new Error(`Failed to transform: ${error.message}`);
     }
@@ -1307,6 +1342,8 @@ class MCPPromptOptimizer {
       user_goal: args.goal,
       sop_content: args.sop_content || "",
     };
+    if (args.agent_read_only) payload.agent_read_only = true;
+    if (args.agent_harness) payload.agent_harness = args.agent_harness;
 
     // If session_id provided, first fetch session artifacts for sop_content
     if (args.session_id) {
@@ -1529,10 +1566,23 @@ class MCPPromptOptimizer {
     } else {
       sections.push('\n```json\n' + JSON.stringify(result, null, 2) + '\n```');
     }
+    const warn = this._formatWarnings(result);
+    if (warn) sections.push(warn);
     sections.push('\n---\n*Generated by MCP Prompt Optimizer CE*');
     return sections.join('\n');
   }
 
+  // Backend CE responses carry a non-fatal `warnings` array (dropped SOP
+  // stages, duplicated tables, fabricated headings). Surface it so a degraded
+  // artifact travels with the reason instead of shipping silently. Returns ''
+  // when there is nothing to report.
+  _formatWarnings(result) {
+    const w = result && (result.warnings || result.validation_warnings);
+    if (!Array.isArray(w) || w.length === 0) return '';
+    return `\n\n## ⚠️ Structural Warnings\n\n${w.map(x => `- ${x}`).join('\n')}\n\n` +
+      `_The artifact is delivered as-is; review these before use._`;
+  }
+
   _buildUrl(path) {
     return `${this.backendUrl}${path}`;
   }
@@ -1655,6 +1705,11 @@ class MCPPromptOptimizer {
     if (result.metadata?.routing_score != null) {
       output += `**Routing Score:** ${result.metadata.routing_score.toFixed(3)} (${result.metadata?.routing_tier || 'unknown'})\n`;
     }
+    if (result.execution_shape && context.requestedExecutionShape && result.execution_shape !== context.requestedExecutionShape) {
+      output += `**Execution Shape:** \`${result.execution_shape}\` *(requested \`${context.requestedExecutionShape}\`, downgraded — your tier doesn't have repair access)*\n`;
+    } else if (result.reasoning_effort && result.reasoning_effort !== 'standard') {
+      output += `**Reasoning Effort:** \`${result.reasoning_effort}\`\n`;
+    }
     if (!result.rules_based && !result.fallback_mode && result.metadata?.model_used) {
       output += `**Model:** ${result.metadata.model_used}\n`;
     }

package/lib/api-key-manager.js

@@ -220,30 +220,38 @@ class CloudApiKeyManager {
             this.log(`Backend validation failed: ${error.message}`, 'warn');
             await this.updateNetworkHealth(false, error.message);
             
-            // Enhanced fallback strategy
-            const cachedValidation = await this.getCachedValidation();
-            
-            if (cachedValidation && !this.isCacheExpired(cachedValidation)) {
-                this.log('Using cached API key validation', 'warn');
-                return cachedValidation.data;
-            }
-            
-            // SECURITY: Limited fallback for brief network issues only (2 hours max)
-            if (cachedValidation && !this.isFallbackCacheExpired(cachedValidation)) {
-                this.log('Using short-term fallback cache due to network issues', 'warn');
-                const fallbackData = cachedValidation.data;
-                fallbackData.fallback_mode = true;
-                fallbackData.network_issue = error.message;
-                fallbackData.expires_soon = true;
-                return fallbackData;
+            // A 4xx means the backend rejected THIS key itself — not a connectivity
+            // problem. Falling back to a cache entry here would mean any key,
+            // valid or not, silently inherits whatever key was last cached on this
+            // machine (cacheFile is a single global file, not namespaced per key).
+            // Cache fallback exists only for genuine network/5xx outages of the
+            // SAME key that was already validated and cached — never for a
+            // definitive rejection, and never for a cache entry from a different key.
+            const isClientError = error.statusCode >= 400 && error.statusCode < 500;
+
+            if (!isClientError) {
+                const cachedValidation = await this.getCachedValidation();
+                const cacheMatchesThisKey = cachedValidation
+                    && cachedValidation.apiKeyPrefix === this.apiKey.substring(0, 20) + '...';
+
+                if (cacheMatchesThisKey && !this.isCacheExpired(cachedValidation)) {
+                    this.log('Using cached API key validation', 'warn');
+                    return cachedValidation.data;
+                }
+
+                // SECURITY: Limited fallback for brief network issues only (2 hours max)
+                if (cacheMatchesThisKey && !this.isFallbackCacheExpired(cachedValidation)) {
+                    this.log('Using short-term fallback cache due to network issues', 'warn');
+                    const fallbackData = cachedValidation.data;
+                    fallbackData.fallback_mode = true;
+                    fallbackData.network_issue = error.message;
+                    fallbackData.expires_soon = true;
+                    return fallbackData;
+                }
             }
 
             // SECURITY: Offline mode removed - backend validation required
-            // No cache fallback beyond 2 hours
-
-            // A 4xx means the backend rejected the key itself, not a connectivity
-            // problem — telling the user to check their internet is wrong there.
-            const isClientError = error.statusCode >= 400 && error.statusCode < 500;
+            // No cache fallback beyond 2 hours, and none at all for a 4xx rejection.
             const hint = isClientError ? '' : ' Please check your internet connection.';
             throw new Error(`API key validation failed: ${error.message}.${hint}`);
         }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mcp-prompt-optimizer",
-  "version": "3.7.5",
+  "version": "3.8.2",
   "description": "Professional cloud-based MCP server for AI-powered prompt optimization with intelligent context detection, Bayesian optimization, AG-UI real-time optimization, template auto-save, optimization insights, personal model configuration via WebUI, team collaboration, enterprise-grade features, production resilience, and startup validation. Universal compatibility with Claude Desktop, Cursor, Windsurf, and 17+ MCP clients.",
   "main": "index.js",
   "bin": {
@@ -25,6 +25,8 @@
     "test:runner": "node tests/test-runner.js",
     "test:simple": "node tests/simple-test.js",
     "test:contract": "node tests/contract-check.js",
+    "test:e2e": "node tests/e2e-stdio-smoke.js",
+    "test:prod-canary": "node tests/prod-canary.js",
     "pretest": "npm run health-check",
     "prepublishOnly": "npm run test:quick",
     "version": "echo 'Updating version...' && npm run test:quick"