prism-mcp-server 7.2.0 → 7.3.3

package/README.md

@@ -28,7 +28,9 @@ Works with **Claude Desktop · Claude Code · Cursor · Windsurf · Cline · Gem
 - [What Makes Prism Different](#-what-makes-prism-different)
 - [Use Cases](#-use-cases)
 - [What's New](#-whats-new)
-  - [v7.2 Executive Function (Roadmap)](#v720--the-executive-function-update-)
+  - [v7.3.1 Dark Factory (Fail-Closed Execution)](#v731--dark-factory-fail-closed-execution-)
+  - [v7.2.0 Verification Harness (Planned)](#v720--verification-harness-front-loaded-testing-)
+  - [v7.4.0 Adversarial Dev Harness (Planned)](#v740--adversarial-dev-harness-anti-sycophancy-)
 - [How Prism Compares](#-how-prism-compares)
 - [Tool Reference](#-tool-reference)
 - [Environment Variables](#environment-variables)
@@ -411,7 +413,7 @@ Soft/hard delete (Art. 17), full export in JSON, Markdown, or Obsidian vault `.z
 
 **Consulting / multi-project** — Switch between client projects with progressive loading: `quick` (~50 tokens), `standard` (~200), or `deep` (~1000+).
 
-**Complex refactoring (v7.2 planned)** — Prism’s roadmap adds plan-first execution for multi-step changes with persistent plan-state tracking across sessions.
+**Complex refactoring (v7.2 planned)** — Prism’s roadmap adds verification-first execution for multi-step changes with contract-frozen assertions and gated finalization.
 
 **Team onboarding** — New team member's agent loads the full project history instantly.
 
@@ -440,24 +442,86 @@ Then continue a specific thread with a follow-up message to the selected agent,
 
 ## 🆕 What's New
 
-### v7.2.0 — The "Executive Function" Update 🔭
-> **Planned roadmap release.** Extends Prism from persistent memory toward autonomous plan execution.
+### v7.3.1 — Dark Factory (Fail-Closed Execution) 🏭
+> **Current stable release.** Hardened autonomous pipeline execution with a structured JSON action contract.
 
-- 🗺️ **Autonomous Plan Decomposition (planned)** — Proposed `session_plan_decompose` tool to transform ambiguous multi-step goals into a structured task DAG.
-- 🔄 **Self-Healing Execution Loop (planned)** — Proposed plan-state engine to capture failed steps, suggest corrective actions, and re-queue recoverable sub-tasks before escalation.
-- 📉 **DAG Plan Visualizer (planned)** — Proposed dashboard Plan/Goal Monitor to render step progress, dependency state, and execution pivots in real time.
-- 🧠 **Context-Aware Goal Tracking (planned)** — Proposed active-plan injection during context loading so agents track not only prior work but current plan position.
-- ⚙️ **Recursive Tool Chaining (planned)** — Proposed middleware path for lower-latency plan-step updates across complex workflows.
-- 🧪 **Plan Integrity Tests (planned)** — Proposed suite validating plan-state persistence across interruptions and session handoffs.
+When an AI agent executes code autonomously — no human watching, no approval step — a single hallucinated file path can write outside your project, corrupt sibling repos, or hit system files. This is the "dark factory" problem: **lights-out execution demands machine-enforced safety, not LLM good behavior.**
+
+> *"I started building testing harnesses with programmatic checks in the planning phase across 3 layers. I got this idea when I was doing a complex ETL process across 3 databases and I needed to stack 9's on data accuracy, but also across the agent layer. After a considerable amount of hair pulling, I started to front load. It's now part of my lifecycle harness that my dark factory uses by default."*
+> — [Stephen Driggs](https://linkedin.com/in/stephendriggs), VP Product AI at Shift4
+
+Prism v7.3.1 implements exactly this: a **3-gate fail-closed pipeline** where every `EXECUTE` step must pass parse, type, and scope validation before any filesystem side effect occurs.
+
+- 🔒 **Structured Action Contract** — `EXECUTE` steps must return machine-parseable JSON conforming to `{ actions: [{ type, targetPath, content? }] }`. Free-form text is rejected at the gate.
+- 🛡️ **3-Strategy Defensive Parser** — Raw JSON → fenced code block extraction → brace extraction. Handles adversarial LLM output (preamble text, markdown fences, trailing commentary) without ever executing malformed payloads.
+- ✅ **Type Validation** — Only `READ_FILE | WRITE_FILE | PATCH_FILE | RUN_TEST` are permitted. Novel action types invented by the LLM are rejected.
+- 📏 **Scope Validation** — Every `targetPath` is resolved against the pipeline's `workingDirectory` via `SafetyController.validateActionsInScope()`. Path traversal (`../`), sibling-prefix bypasses, and absolute paths outside the boundary are blocked.
+- 🚫 **Pipeline-Level Termination** — A scope violation doesn't just fail the step — it **terminates the entire pipeline** with `status: FAILED` and emits a `failure` experience event for the ML routing layer.
+
+<details>
+<summary><strong>🔬 The 3-Gate Architecture: How a Path Traversal Attack Fails</strong></summary>
+
+**Scenario:** An LLM running autonomously in a Dark Factory pipeline targeting `/home/user/my-app` produces this output for an EXECUTE step:
+
+```json
+{
+  "actions": [
+    { "type": "WRITE_FILE", "targetPath": "src/utils.ts", "content": "// valid" },
+    { "type": "WRITE_FILE", "targetPath": "../../.ssh/authorized_keys", "content": "ssh-rsa ATTACK..." }
+  ]
+}
+```
+
+**Gate 1 — Parse:** ✅ Valid JSON. The 3-strategy parser extracts it cleanly.
+
+**Gate 2 — Type:** ✅ Both actions use `WRITE_FILE`, a valid `ActionType`.
+
+**Gate 3 — Scope:** 🚫 **BLOCKED.** `SafetyController.validateActionsInScope()` resolves `../../.ssh/authorized_keys` to `/home/user/.ssh/authorized_keys` — which is **outside** `/home/user/my-app`. The violation is detected *before* any write occurs.
+
+**Result:** The entire pipeline is terminated immediately. No files are written — not even the valid `src/utils.ts`. A `failure` experience event is emitted so the ML router can learn to avoid this pattern.
+
+```
+ Pipeline FAILED: Scope violation — action[1] targetPath
+ "../../.ssh/authorized_keys" resolves outside workingDirectory
+```
+
+**Without v7.3.1:** The LLM's raw text output would be interpreted as instructions, and the agent runtime would attempt the write — potentially succeeding depending on filesystem permissions.
+
+**With v7.3.1:** The structured contract makes this class of attack impossible. The LLM never touches the filesystem directly; every action is validated through the 3-gate pipeline first.
+
+</details>
+
+<details>
+<summary><strong>🧪 Edge Cases Covered (67 tests)</strong></summary>
+
+| Category | Examples |
+|----------|----------|
+| **Parse adversarial output** | Prose preamble + JSON, nested fences, empty input, non-string input |
+| **Type coercion** | `"DELETE_FILE"`, `"EXEC_CMD"`, numeric types, null types |
+| **Path traversal** | `../`, `../../`, `/etc/passwd`, null bytes, unicode normalization, embedded newlines |
+| **Shape validation** | Missing `actions` array, non-object actions, empty `targetPath`, root-type coercion |
+| **Stress payloads** | 100-action arrays, 100KB content strings, 500-segment deep paths |
+
+</details>
+
+### v7.2.0 — Verification Harness (Front-Loaded Testing) 🔭
+> **Planned roadmap release.** Extends Prism from passive validation to contract-frozen, machine-verifiable execution gates.
+
+- 📋 **Spec-Freeze Contract (planned)** — v7.2 formalizes three artifacts with strict responsibilities: `implementation_plan.md` (**how**), `verification_harness.json` (**proof contract**), and `validation_result` (**immutable outcome record**).
+- 🔐 **Rubric Hash Lock (planned)** — `verification_harness.json` is generated before execution and hash-locked (`rubric_hash`) so criteria cannot drift mid-sprint.
+- 🔬 **Multi-Layer Verification (planned)** — Structured checks across **Data Accuracy**, **Agent Behavior**, and **Pipeline Integrity** using machine-parseable assertions.
+- 🤖 **Adversarial Validation Loop (planned)** — A second validation pass evaluates execution outputs against the frozen contract before progression.
+- 🚦 **Finalization Gates (planned)** — Gate policies (`warn` / `gate` / `abort`) evaluate `validation_result` against the frozen rubric before pipeline completion.
+- 🧠 **Routing Feedback Signals (planned)** — Router learning ingests raw verification signals (`pass_rate`, `critical_failures`, `coverage_score`, `rubric_hash`) for downstream confidence adjustment.
 
 <details>
 <summary><strong>🔬 Concept Example: Before vs. After v7.2</strong></summary>
 
 **Scenario:** "Refactor the Auth module and update the unit tests."
 
-**Before (linear prompting):** The agent executes in sequence but can lose place after errors unless the host prompt restates state.
+**Before:** Criteria emerge during or after coding; verification is inconsistent and hard to audit.
 
-**After (executive planning):** Agent decomposes to a DAG, executes per-step, recovers from failures via plan-state retries, and resumes from the correct dependency node.
+**After (verification-first):** Plan emits a frozen verification contract first, execution runs, validator emits immutable `validation_result`, and finalization gates enforce rubric compliance.
 
 </details>
 
@@ -739,13 +803,26 @@ Requires `PRISM_TASK_ROUTER_ENABLED=true` (or dashboard toggle).
 </details>
 
 <details>
-<summary><strong>Executive Planning (Planned for v7.2)</strong></summary>
+<summary><strong>Dark Factory Orchestration (3 tools)</strong></summary>
+
+Requires `PRISM_DARK_FACTORY_ENABLED=true`.
 
 | Tool | Purpose |
 |------|---------|
-| `session_plan_decompose` | Decompose natural language goals into a structured DAG of tasks |
-| `session_plan_step_update` | Atomically update the status/result of a specific sub-task |
-| `session_plan_get_active` | Retrieve the current execution DAG and task statuses |
+| `session_start_pipeline` | Create and enqueue a background autonomous pipeline |
+| `session_check_pipeline_status` | Poll the current step, iteration, and status of a pipeline |
+| `session_abort_pipeline` | Emergency kill switch to halt a running background pipeline |
+
+</details>
+
+<details>
+<summary><strong>Verification Harness (Planned for v7.2)</strong></summary>
+
+| Tool | Purpose |
+|------|---------|
+| `session_plan_decompose` | Decompose natural language goals into an execution plan that references verification requirements |
+| `session_plan_step_update` | Atomically update step status/result with verification context |
+| `session_plan_get_active` | Retrieve active plan state and current verification gating position |
 
 </details>
 
@@ -797,6 +874,7 @@ Requires `PRISM_TASK_ROUTER_ENABLED=true` (or dashboard toggle).
 | `PRISM_ACTR_WEIGHT_SIMILARITY` | No | Composite score similarity weight (default: `0.7`) |
 | `PRISM_ACTR_WEIGHT_ACTIVATION` | No | Composite score ACT-R activation weight (default: `0.3`) |
 | `PRISM_ACTR_ACCESS_LOG_RETENTION_DAYS` | No | Days before access logs are pruned by background scheduler (default: `90`) |
+| `PRISM_DARK_FACTORY_ENABLED` | No | `"true"` to enable Dark Factory autonomous pipeline tools (`session_start_pipeline`, `session_check_pipeline_status`, `session_abort_pipeline`) |
 
 </details>
 
@@ -827,6 +905,7 @@ Prism is a **stdio-based MCP server** that manages persistent agent memory. Here
 │         ↕                                                │
 │  ┌────────────────────────────────────────────────────┐  │
 │  │  Background Workers                                │  │
+│  │  • Dark Factory (3-gate fail-closed pipelines)     │  │
 │  │  • Scheduler (TTL, decay, compaction, purge)       │  │
 │  │  • Web Scholar (Brave → Firecrawl → LLM → Ledger)  │  │
 │  │  • Hivemind heartbeats & Telepathy broadcasts      │  │
@@ -891,7 +970,8 @@ Prism is evolving from smart session logging toward a **cognitive memory archite
 | **v7.0** | Candidate-Scoped Spreading Activation — `S_i = Σ(W × strength)` bounded to search result set; prevents God-node dominance | Spreading activation networks (Collins & Loftus, 1975) | ✅ Shipped |
 | **v7.0** | Composite Retrieval Scoring — `0.7 × similarity + 0.3 × σ(activation)`; configurable via `PRISM_ACTR_WEIGHT_*` | Hybrid cognitive-neural retrieval models | ✅ Shipped |
 | **v7.0** | AccessLogBuffer — in-memory batch-write buffer with 5s flush; prevents SQLite `SQLITE_BUSY` under parallel agents | Production reliability engineering | ✅ Shipped |
-| **v7.2** | Executive Planning & DAG tracking | Prefrontal cortex executive control + Directed Acyclic Graph planning | 🔭 Horizon |
+| **v7.3** | Dark Factory — 3-gate fail-closed EXECUTE pipeline (parse → type → scope) with structured JSON action contract | Industrial safety systems (defense-in-depth, fail-closed valves) | ✅ Shipped |
+| **v7.2** | Verification-first harness & contract-gated execution | Programmatic verification systems + adversarial validation loops | 🔭 Horizon |
 | **v7.x** | Affect-Tagged Memory — sentiment shapes what gets recalled | Affect-modulated retrieval (neuroscience) | 🔭 Horizon |
 | **v8+** | Zero-Search Retrieval — no index, no ANN, just ask the vector | Holographic Reduced Representations | 🔭 Horizon |
 
@@ -909,14 +989,17 @@ Shipped in v6.2.0. Edge synthesis, graph pruning with SLO observability, tempora
 ### v6.5: Cognitive Architecture ✅
 Shipped. Full Superposed Memory (SDM) + Hyperdimensional Computing (HDC/VSA) cognitive routing pipeline. Compositional memory states via XOR binding, Hamming resolution, and policy-gated routing (direct / clarify / fallback). 705 tests passing.
 
+### v7.3: Dark Factory — Fail-Closed Execution ✅
+Shipped. Structured JSON action contract for autonomous `EXECUTE` steps. 3-gate validation pipeline (parse → type → scope) terminates pipelines on any violation before filesystem side effects. 67 edge-case tests covering adversarial LLM output, path traversal, and type coercion.
+
 ### v7.1: Prism Task Router ✅
 Shipped. Deterministic task routing (`session_task_route`) with optional experience-based confidence adjustment for host vs. local Claw delegation.
 
 ### v7.0: ACT-R Activation Memory ✅
 Shipped. Scientifically-grounded retrieval re-ranking via ACT-R base-level activation (`B_i = ln(Σ t_j^(-d))`), candidate-scoped spreading activation, parameterized sigmoid normalization, composite scoring, and zero-cold-start access log infrastructure. 49 dedicated unit tests, 705 total passing.
 
-### v7.2: Executive Function 🔭
-Planned. Adds autonomous plan decomposition, DAG-backed step tracking, and self-healing execution loops for complex multi-step operations.
+### v7.2: Verification Harness 🔭
+Planned. Adds a spec-frozen verification contract (`implementation_plan.md` + `verification_harness.json` + immutable `validation_result`), multi-layer machine checks, and finalization gates before autonomous completion.
 
 ### Future Tracks
 - **v7.x: Affect-Tagged Memory** — Recall prioritization improves by weighting memories with affective/contextual valence, making surfaced context more behaviorally useful.

package/dist/cli.js

@@ -0,0 +1,50 @@
+#!/usr/bin/env node
+import { Command } from 'commander';
+import { SqliteStorage } from './storage/sqlite.js';
+import { handleVerifyStatus, handleGenerateHarness } from './verification/cliHandler.js';
+import * as path from 'path';
+const program = new Command();
+program
+    .name('prism')
+    .description('Prism Configuration & CLI')
+    .version('7.3.1');
+const verifyCmd = program
+    .command('verify')
+    .description('Manage the verification harness');
+verifyCmd
+    .command('status')
+    .description('Check the current verification state and view config drift')
+    .option('-p, --project <name>', 'Project name', path.basename(process.cwd()))
+    .option('-f, --force', 'Bypass verification failures and drift tracking constraints')
+    .option('-u, --user <id>', 'User ID for tenant isolation', 'default')
+    .option('--json', 'Emit machine-readable JSON output with stable keys')
+    .action(async (options) => {
+    const storage = new SqliteStorage();
+    await storage.initialize('./prism-local.db');
+    // H4 fix: Ensure storage is closed on exit to flush WAL and prevent data loss
+    try {
+        await handleVerifyStatus(storage, options.project, !!options.force, options.user, !!options.json);
+    }
+    finally {
+        await storage.close();
+    }
+});
+verifyCmd
+    .command('generate')
+    .description('Bless the current ./verification_harness.json as the canonical rubric')
+    .option('-p, --project <name>', 'Project name', path.basename(process.cwd()))
+    .option('-f, --force', 'Bypass verification failures and drift tracking constraints')
+    .option('-u, --user <id>', 'User ID for tenant isolation', 'default')
+    .option('--json', 'Emit machine-readable JSON output with stable keys')
+    .action(async (options) => {
+    const storage = new SqliteStorage();
+    await storage.initialize('./prism-local.db');
+    // H4 fix: Ensure storage is closed on exit to flush WAL and prevent data loss
+    try {
+        await handleGenerateHarness(storage, options.project, !!options.force, options.user, !!options.json);
+    }
+    finally {
+        await storage.close();
+    }
+});
+program.parse(process.argv);

package/dist/config.js

@@ -241,3 +241,19 @@ export const PRISM_TASK_ROUTER_ENABLED_ENV = process.env.PRISM_TASK_ROUTER_ENABL
 export const PRISM_TASK_ROUTER_CONFIDENCE_THRESHOLD = parseFloat(process.env.PRISM_TASK_ROUTER_CONFIDENCE_THRESHOLD || "0.6");
 /** Maximum complexity score (1-10) that Claw can handle. Tasks above this → host. (Default: 4) */
 export const PRISM_TASK_ROUTER_MAX_CLAW_COMPLEXITY = parseInt(process.env.PRISM_TASK_ROUTER_MAX_CLAW_COMPLEXITY || "4", 10);
+// ─── v7.2: Verification Harness ──────────────────────────────
+/** Master switch for the v7.2.0 enhanced verification harness. */
+export const PRISM_VERIFICATION_HARNESS_ENABLED = process.env.PRISM_VERIFICATION_HARNESS_ENABLED === "true";
+/** Comma-separated list of verification layers to run. */
+export const PRISM_VERIFICATION_LAYERS = (process.env.PRISM_VERIFICATION_LAYERS || "data,agent,pipeline").split(",").map(l => l.trim()).filter(Boolean);
+/** Default severity floor for all assertions. Overrides individual assertion severity when higher. */
+export const PRISM_VERIFICATION_DEFAULT_SEVERITY = (process.env.PRISM_VERIFICATION_DEFAULT_SEVERITY || "warn");
+// ─── v7.3: Dark Factory Orchestration ─────────────────────────
+// Autonomous pipeline runner: PLAN → EXECUTE → VERIFY → iterate.
+// Opt-in because it executes LLM calls in the background.
+/** Master switch for the Dark Factory background runner. */
+export const PRISM_DARK_FACTORY_ENABLED = process.env.PRISM_DARK_FACTORY_ENABLED === "true"; // Opt-in
+/** Poll interval for the runner loop (ms). Default: 30s. */
+export const PRISM_DARK_FACTORY_POLL_MS = parseInt(process.env.PRISM_DARK_FACTORY_POLL_MS || "30000", 10);
+/** Default max wall-clock time per pipeline (ms). Default: 15 minutes. */
+export const PRISM_DARK_FACTORY_MAX_RUNTIME_MS = parseInt(process.env.PRISM_DARK_FACTORY_MAX_RUNTIME_MS || "900000", 10);

package/dist/darkfactory/clawInvocation.js

@@ -0,0 +1,77 @@
+import { getLLMProvider } from '../utils/llm/factory.js';
+import { OpenAIAdapter } from '../utils/llm/adapters/openai.js';
+import { SafetyController } from './safetyController.js';
+import { debugLog } from '../utils/logger.js';
+/**
+ * JSON output schema instruction injected into EXECUTE step prompts.
+ * Forces the LLM to respond with machine-parseable structured output
+ * instead of free-form text. The runner validates this shape before
+ * any actions are applied.
+ */
+const EXECUTE_JSON_SCHEMA = `
+You MUST respond with ONLY a valid JSON object matching this schema:
+{
+  "actions": [
+    {
+      "type": "READ_FILE" | "WRITE_FILE" | "PATCH_FILE" | "RUN_TEST",
+      "targetPath": "<relative path within the workspace>",
+      "content": "<file content for WRITE_FILE>",
+      "patch": "<patch content for PATCH_FILE>",
+      "command": "<test command for RUN_TEST>"
+    }
+  ],
+  "notes": "<optional summary of what you did>"
+}
+
+RULES:
+- type MUST be one of: READ_FILE, WRITE_FILE, PATCH_FILE, RUN_TEST
+- targetPath MUST be a relative path within the workspace
+- Do NOT include any text outside the JSON object
+- Do NOT use markdown code fences
+- If you cannot complete the task, return: {"actions": [], "notes": "reason"}
+`.trim();
+/**
+ * Invocation wrapper that routes payload specs to the local Claw agent model (Qwen 2.5),
+ * or the active LLM provider as fallback.
+ *
+ * Uses SafetyController.generateBoundaryPrompt() for scope injection
+ * instead of inline prompt construction — single source of truth for safety rules.
+ *
+ * v7.3.1: EXECUTE steps request structured JSON output via EXECUTE_JSON_SCHEMA.
+ *         Non-EXECUTE steps continue to use free-form text output.
+ */
+export async function invokeClawAgent(spec, state, timeoutMs = 120000 // 2 min default timeout for internal executions
+) {
+    // BYOM Override: Provide path to use alternative open-source pipelines 
+    // (e.g. through the OpenAI structured adapter which also points to local endpoints like Ollama / vLLM if configured)
+    const llm = spec.modelOverride
+        ? new OpenAIAdapter() // Bypasses the factory to route locally
+        : getLLMProvider();
+    // Scope injection via SafetyController — single source of truth
+    const systemPrompt = SafetyController.generateBoundaryPrompt(spec, state);
+    // v7.3.1: EXECUTE steps get structured JSON output instructions
+    const isExecuteStep = state.current_step === 'EXECUTE';
+    const executePrompt = isExecuteStep
+        ? `Based on the system instructions, execute the necessary actions for the current step (${state.current_step}).\n\n${EXECUTE_JSON_SCHEMA}`
+        : `Based on the system instructions, execute the necessary task for the current step (${state.current_step}). Respond with your actions and observations.`;
+    debugLog(`[ClawInvocation] Launching agent on pipeline ${state.id} step=${state.current_step} iter=${state.iteration} with ${timeoutMs}ms limit.${isExecuteStep ? ' (JSON mode)' : ''}`);
+    try {
+        // Timeout Promise to ensure the runner thread does not block indefinitely
+        const timeboundExecution = Promise.race([
+            llm.generateText(executePrompt, systemPrompt),
+            new Promise((_, reject) => setTimeout(() => reject(new Error('LLM_EXECUTION_TIMEOUT')), timeoutMs))
+        ]);
+        const result = await timeboundExecution;
+        return {
+            success: true,
+            resultText: result
+        };
+    }
+    catch (error) {
+        debugLog(`[ClawInvocation] Exception during generation: ${error.message}`);
+        return {
+            success: false,
+            resultText: error.message
+        };
+    }
+}