thumbgate 1.29.1 → 1.29.2

package/.claude/commands/dashboard.md

@@ -1,15 +1,25 @@
 ---
 name: dashboard
 description: Open the local HTTP dashboard for the current project in your web browser.
+allowed-tools: Bash(npx thumbgate dashboard:*)
 ---
 
 # Open Dashboard
 
 Open the local HTTP dashboard for the current project in your web browser.
 
+This command wraps existing ThumbGate capability — **no new logic**.
+
 ## Instructions
+
 Execute the following command in the project directory to open the browser dashboard scoped to the current repository:
+
 ```bash
-thumbgate-dashboard
+npx thumbgate dashboard --open
 ```
 
+Standalone shortcut:
+
+```bash
+thumbgate-dashboard
+```

package/.claude/commands/thumbgate-dashboard.md

@@ -1,15 +1,30 @@
 ---
 name: thumbgate-dashboard
-description: Open the local HTTP dashboard for the current project in your web browser.
+description: Open the local HTTP dashboard for the current project in your web browser. Use for "open dashboard", "thumbgate-dashboard", "show my gates in the browser", "project dashboard", or "open the local ThumbGate UI".
+allowed-tools: Bash(npx thumbgate dashboard:*)
 ---
 
-# Open Scoped ThumbGate Dashboard
+# ThumbGate Dashboard
 
-Open the local HTTP dashboard for the current project in your web browser.
+Open the local HTTP dashboard for the current project so you can inspect lessons, checks, gate stats, and tokens saved in the browser.
 
-## Instructions
-Execute the following command in the project directory to open the browser dashboard:
-```bash
-thumbgate-dashboard
-```
+This command wraps existing ThumbGate capability — **no new logic**. It runs the existing project-scoped dashboard opener.
+
+## Steps
+
+1. Open the project-scoped dashboard:
+   ```bash
+   npx thumbgate dashboard --open
+   ```
+   Equivalent standalone shortcut after a global install:
+   ```bash
+   thumbgate-dashboard
+   ```
+2. Confirm the browser lands on the local dashboard for this repo (lessons, checks, gate stats, tokens saved).
+3. If the command fails, run `npx thumbgate doctor` — a missing install is usually fixed by `npx thumbgate init`.
 
+## Example
+
+```
+/thumbgate-dashboard
+```

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "thumbgate",
   "description": "One 👎 becomes a hard rule the agent cannot bypass. Captures thumbs-down feedback, distills it into PreToolUse Pre-Action Checks, enforced across every future Claude Code session.",
-  "version": "1.29.1",
+  "version": "1.29.2",
   "author": {
     "name": "Igor Ganapolsky",
     "email": "ig5973700@gmail.com",

package/.well-known/mcp/server-card.json

@@ -1,6 +1,6 @@
 {
   "name": "thumbgate",
-  "version": "1.29.1",
+  "version": "1.29.2",
   "description": "ThumbGate — 👍👎 feedback that teaches your AI agent. Thumbs down a mistake, it never happens again.",
   "homepage": "https://thumbgate.ai",
   "transport": "stdio",

package/README.md

@@ -65,12 +65,15 @@ Spec-driven agent frameworks like **GSD** (get-shit-done) and **GitHub Spec Kit*
 
 | Command | What it does | Wraps (existing capability) |
 |---------|--------------|------------------------------|
+| **`/thumbgate-dashboard`** | **Open the local project dashboard in your browser** (lessons, checks, tokens saved) | **`npx thumbgate dashboard --open`** (global bin after `npm i -g`) |
 | `/thumbgate-guard` | Turn the last agent mistake into a hard prevention rule | `capture_feedback` + `thumbgate force-gate` |
 | `/thumbgate-rules` | List the active prevention rules + lessons guarding this repo | `prevention_rules`, `get_reliability_rules`, `search_lessons` |
 | `/thumbgate-blocked` | Show what's actually been blocked — gate stats + enforcement matrix | `gate_stats`, `enforcement_matrix` |
 | `/thumbgate-protect` | Show branch/release governance; grant a scoped, expiring approval | `get_branch_governance`, `approve_protected_action` |
 | `/thumbgate-doctor` | Health-check the wiring (hooks, MCP, agent-readiness) | `thumbgate doctor` |
 
+> **Open the dashboard anytime:** after `npx thumbgate init`, run **`npx thumbgate dashboard --open`** (works without a global install). Type **`/thumbgate-dashboard`** in Claude Code / Cursor, or **`/project:thumbgate-dashboard`** in Grok. After `npm i -g thumbgate`, the **`thumbgate-dashboard`** bin is also on your PATH.
+
 Each is a thin wrapper over an existing MCP tool or CLI command — **no new enforcement logic, just discoverability**.
 
 ---
@@ -196,6 +199,63 @@ Hand-rolled hooks are the right tool for a small, static denylist you maintain b
 
 Prompt engineering still matters, but it is only the starting point. ThumbGate adds prompt evaluation on top: proof lanes, benchmarks, and self-heal checks produce reviewable evidence about whether a prompt and workflow held up under execution. Run `npx thumbgate eval --from-feedback --write-report=.thumbgate/prompt-eval-proof.md` to turn accepted thumbs-up/down feedback into reusable eval cases and a local proof report.
 
+### How ThumbGate knows an AI agent is working
+
+ThumbGate does not treat a plausible response, a successful tool call, or a
+demo as task success. The `record_task_outcome` MCP tool and
+`POST /v1/task-outcomes` API require a task-level receipt. A receipt is marked
+`working: true` only when the task is completed, verification passed, evidence
+is present, tool contracts and policy checks passed, no unsupported claim was
+recorded, and no side effect was duplicated.
+
+The metrics remain separate so a strong average cannot conceal an unsafe
+failure:
+
+| Layer | Measured signals |
+|-------|------------------|
+| Task | verified completion, evidence-backed completion, first-attempt success, repeated failure, recovery, rollback |
+| Tool | contract accuracy, execution success, retry rate, duplicate side effects |
+| Safety | unsafe escapes, policy violations, safe-action false blocks |
+| Escalation | correct escalation rate and human decision latency |
+| Efficiency | p50/p95 latency, total cost, cost per verified success |
+| Business | explicit KPI values grouped by unit; no inferred revenue |
+
+```bash
+npm run eval:agent-outcomes       # 8 reviewed golden cases; fails on regression
+npm run monitor:agent-outcomes    # local production receipts; fails on missing evidence
+npm run monitor:agent-outcomes -- --hosted
+npm run monitor:agent-outcomes -- --install-schedule
+```
+
+Prompt evaluation is deterministic first. JSON outputs are validated against
+their declared schema. An LLM judge may add a separate score, but an unavailable
+or failed judge is reported as `deterministic_only`; it is never converted into
+a neutral pass. Empty feedback or task-outcome datasets return
+`insufficient_evidence`.
+
+Task outcome receipts are stored locally in
+`.thumbgate/task-outcome-receipts.jsonl`. Observable tool traces exclude raw
+hidden reasoning and deterministic tool-argument fingerprints. Human
+escalation requests are append-only, must carry evidence and requester
+identity, and expire. Decisions require both the ordinary API credential and
+an independently revocable `X-ThumbGate-Human-Reviewer-Key`; the decision
+actor comes from server-side `THUMBGATE_HUMAN_REVIEWER_ID` configuration, not
+from caller-controlled JSON.
+
+The installed daily monitor runs through ThumbGate's local scheduler rather
+than consuming a GitHub-hosted cron runner. It reads operator authentication
+from the existing environment or local operator config, never from command-line
+arguments, and writes a machine-readable report under
+`~/.thumbgate/reports/agent-outcome-monitor.json`. The GitHub workflow remains
+manual for release-time verification.
+
+The complete five-system architecture review—local RAG, tool-using agents,
+multi-agent handoffs, MCP enterprise integration, and production
+evaluation/observability—is maintained in
+[`VERIFICATION_EVIDENCE.md`](./VERIFICATION_EVIDENCE.md). It answers why each
+architecture exists, what can fail, how it is measured and secured, how it is
+deployed, and what evidence is required before claiming it works.
+
 ### Retrieval & latency: local-first, zero network hops
 
 ThumbGate's latency advantage is structural, not a tuned cloud cluster: there is no retrieval service and no model on the enforcement path, so the gate decision never leaves your machine.
@@ -385,7 +445,7 @@ npx thumbgate background-governance  # review background-agent run risk
 npx thumbgate model-candidates --workload=dashboard-analysis --provider=openai --json  # evaluate GPT-5.5 routing
 npx thumbgate native-messaging-audit  # inspect local browser bridges and extension hosts
 npx thumbgate dashboard --open                                  # open local project-scoped dashboard in browser
-thumbgate-dashboard                                             # standalone browser dashboard shortcut (run '/project:thumbgate-dashboard' in Claude/Grok)
+thumbgate-dashboard                                             # global bin after npm i -g thumbgate; agents: /thumbgate-dashboard (Claude/Cursor) or /project:thumbgate-dashboard (Grok)
 npx thumbgate check-update                                      # check if a new version is available on npm/GitHub
 npx thumbgate self-update                                       # update ThumbGate to the latest version globally
 npx thumbgate serve      # start MCP server on stdio

package/adapters/claude/.mcp.json

@@ -2,13 +2,13 @@
   "mcpServers": {
     "thumbgate": {
       "command": "npx",
-      "args": ["--yes", "--package", "thumbgate@1.29.1", "thumbgate", "serve"]
+      "args": ["--yes", "--package", "thumbgate@1.29.2", "thumbgate", "serve"]
     }
   },
   "hooks": {
     "preToolUse": {
       "command": "npx",
-      "args": ["--yes", "--package", "thumbgate@1.29.1", "thumbgate", "gate-check"]
+      "args": ["--yes", "--package", "thumbgate@1.29.2", "thumbgate", "gate-check"]
     }
   }
 }

package/adapters/forge/forge.yaml

@@ -9,12 +9,12 @@ version: "1"
 skills:
   thumbgate-gate-check:
     description: "ThumbGate PreToolUse gate — blocks known-bad tool calls"
-    command: "npx --yes --package thumbgate@1.29.1 thumbgate gate-check"
+    command: "npx --yes --package thumbgate@1.29.2 thumbgate gate-check"
     trigger: pre_tool_use
 
   thumbgate-feedback:
     description: "ThumbGate feedback capture — logs user prompt context"
-    command: "npx --yes --package thumbgate@1.29.1 thumbgate hook-auto-capture"
+    command: "npx --yes --package thumbgate@1.29.2 thumbgate hook-auto-capture"
     trigger: user_prompt
 
 mcp:
@@ -23,6 +23,6 @@ mcp:
     args:
       - "--yes"
       - "--package"
-      - "thumbgate@1.29.1"
+      - "thumbgate@1.29.2"
       - "thumbgate"
       - "serve"

package/adapters/mcp/server-stdio.js

@@ -85,6 +85,17 @@ const {
   pairFeedbackWithReceipt,
   buildReceiptContextEntries,
 } = require('../../scripts/action-receipts');
+const {
+  calculateTaskOutcomeMetrics,
+  getTaskOutcome,
+  readTaskOutcomes,
+  recordTaskOutcome,
+} = require('../../scripts/task-outcomes');
+const {
+  listEscalations,
+  requestEscalation,
+} = require('../../scripts/human-escalation');
+const { recordReasoningTrace } = require('../../scripts/agent-reasoning-traces');
 const {
   evaluateOperationalIntegrity,
 } = require('../../scripts/operational-integrity');
@@ -305,7 +316,7 @@ const {
   finalizeSession: finalizeFeedbackSession,
 } = require('../../scripts/feedback-session');
 
-const SERVER_INFO = { name: 'thumbgate-mcp', version: '1.29.1' };
+const SERVER_INFO = { name: 'thumbgate-mcp', version: '1.29.2' };
 const COMMERCE_CATEGORIES = [
   'product_recommendation',
   'brand_compliance',
@@ -773,6 +784,11 @@ async function callTool(name, args = {}) {
       const err = new Error(`Tool contract violation on '${name}': ${validation.errors.join('; ')}`);
       err.errorCategory = 'contract';
       err.isRetryable = false;
+      recordMcpToolTrace(name, args, {
+        success: false,
+        category: 'contract',
+        evidence: validation.errors,
+      });
       throw err;
     }
   }
@@ -783,12 +799,34 @@ async function callTool(name, args = {}) {
       const err = new Error(`Action blocked by Semantic Firewall: ${firewallResult.message}`);
       err.errorCategory = 'permission';
       err.isRetryable = false;
+      recordMcpToolTrace(name, args, {
+        success: false,
+        category: 'permission',
+        evidence: [firewallResult.message],
+      });
       throw err;
     }
   }
   const startMs = Date.now();
-  const result = await callToolInner(name, args);
+  let result;
+  try {
+    result = await callToolInner(name, args);
+  } catch (err) {
+    recordMcpToolTrace(name, args, {
+      success: false,
+      category: err.errorCategory || 'execution',
+      evidence: [err.code || err.message || 'tool execution failed'],
+      latencyMs: Date.now() - startMs,
+    });
+    throw err;
+  }
   const latencyMs = Date.now() - startMs;
+  recordMcpToolTrace(name, args, {
+    success: true,
+    category: 'success',
+    evidence: [`tool completed in ${latencyMs}ms`],
+    latencyMs,
+  });
   try {
     const { recordAuditEvent } = require('../../scripts/audit-trail');
     recordAuditEvent({
@@ -802,6 +840,41 @@ async function callTool(name, args = {}) {
   return result;
 }
 
+function recordMcpToolTrace(name, args, outcome = {}) {
+  try {
+    const traceId = args.traceId || args.taskId || `mcp-${Date.now()}-${name}`;
+    recordReasoningTrace({
+      trace_id: traceId,
+      task_type: 'tool-use',
+      source: 'mcp-runtime',
+      success: outcome.success,
+      outcome: {
+        success: outcome.success,
+        terminalState: outcome.category,
+      },
+      messages: [
+        {
+          role: 'assistant',
+          content: `tool: ${name}`,
+          tool_calls: [{ function: { name } }],
+        },
+        {
+          role: 'tool',
+          content: `tool response: ${outcome.category}; ${outcome.evidence?.join('; ') || 'no evidence'}`,
+          success: outcome.success,
+        },
+      ],
+      metadata: {
+        latencyMs: outcome.latencyMs || 0,
+        argumentFingerprintStored: false,
+        rawArgumentsStored: false,
+      },
+    });
+  } catch {
+    // Trace telemetry must not change the tool's functional outcome.
+  }
+}
+
 async function callToolInner(name, args) {
   args = args || {};
   // Semantic Aliases for high-level branding alignment
@@ -1154,6 +1227,19 @@ async function callToolInner(name, args) {
           ? getReceiptForAction(args.actionId)
           : getRecentReceipts(Number(args.limit || 20)),
       );
+    case 'record_task_outcome':
+      return toTextResult(recordTaskOutcome(args));
+    case 'get_task_outcomes': {
+      if (args.taskId) return toTextResult(getTaskOutcome(args.taskId));
+      const limit = Number(args.limit || 20);
+      return toTextResult(readTaskOutcomes().slice(-limit));
+    }
+    case 'get_agent_outcome_metrics':
+      return toTextResult(calculateTaskOutcomeMetrics(readTaskOutcomes()));
+    case 'request_human_escalation':
+      return toTextResult(requestEscalation(args));
+    case 'list_human_escalations':
+      return toTextResult(listEscalations({ status: args.status }).slice(0, Number(args.limit || 20)));
     case 'verify_claim':
       return toTextResult(verifyClaimEvidence(args.claim, { goalContract: args.goalContract }));
     case 'require_evidence_for_claim': {

package/adapters/opencode/opencode.json

@@ -7,7 +7,7 @@
         "npx",
         "--yes",
         "--package",
-        "thumbgate@1.29.1",
+        "thumbgate@1.29.2",
         "thumbgate",
         "serve"
       ],

package/commands/dashboard.md

@@ -1,15 +1,25 @@
 ---
 name: dashboard
 description: Open the local HTTP dashboard for the current project in your web browser.
+allowed-tools: Bash(npx thumbgate dashboard:*), Bash(thumbgate-dashboard:*)
 ---
 
 # Open Dashboard
 
 Open the local HTTP dashboard for the current project in your web browser.
 
+This command wraps existing ThumbGate capability — **no new logic**.
+
 ## Instructions
+
 Execute the following command in the project directory to open the browser dashboard scoped to the current repository:
+
 ```bash
-thumbgate-dashboard
+npx thumbgate dashboard --open
 ```
 
+Standalone shortcut:
+
+```bash
+thumbgate-dashboard
+```

package/commands/thumbgate-dashboard.md

@@ -1,15 +1,30 @@
 ---
 name: thumbgate-dashboard
-description: Open the local HTTP dashboard for the current project in your web browser.
+description: Open the local HTTP dashboard for the current project in your web browser. Use for "open dashboard", "thumbgate-dashboard", "show my gates in the browser", "project dashboard", or "open the local ThumbGate UI".
+allowed-tools: Bash(npx thumbgate dashboard:*)
 ---
 
-# Open Scoped ThumbGate Dashboard
+# ThumbGate Dashboard
 
-Open the local HTTP dashboard for the current project in your web browser.
+Open the local HTTP dashboard for the current project so you can inspect lessons, checks, gate stats, and tokens saved in the browser.
 
-## Instructions
-Execute the following command in the project directory to open the browser dashboard:
-```bash
-thumbgate-dashboard
-```
+This command wraps existing ThumbGate capability — **no new logic**. It runs the existing project-scoped dashboard opener.
+
+## Steps
+
+1. Open the project-scoped dashboard:
+   ```bash
+   npx thumbgate dashboard --open
+   ```
+   Equivalent standalone shortcut after a global install:
+   ```bash
+   thumbgate-dashboard
+   ```
+2. Confirm the browser lands on the local dashboard for this repo (lessons, checks, gate stats, tokens saved).
+3. If the command fails, run `npx thumbgate doctor` — a missing install is usually fixed by `npx thumbgate init`.
 
+## Example
+
+```
+/thumbgate-dashboard
+```

package/config/agent-outcome-monitor-thresholds.json

@@ -0,0 +1,63 @@
+{
+  "minimumSamples": 20,
+  "workingRate": {
+    "operator": "gte",
+    "value": 0.8,
+    "severity": "block"
+  },
+  "verifiedCompletionRate": {
+    "operator": "gte",
+    "value": 0.8,
+    "severity": "block"
+  },
+  "evidenceBackedCompletionRate": {
+    "operator": "gte",
+    "value": 0.8,
+    "severity": "block"
+  },
+  "unsupportedClaimRate": {
+    "operator": "lte",
+    "value": 0.01,
+    "severity": "block"
+  },
+  "toolContractAccuracy": {
+    "operator": "gte",
+    "value": 0.99,
+    "severity": "block"
+  },
+  "executionSuccessRate": {
+    "operator": "gte",
+    "value": 0.95,
+    "severity": "block"
+  },
+  "duplicateSideEffectRate": {
+    "operator": "lte",
+    "value": 0,
+    "severity": "block"
+  },
+  "unsafeEscapeRate": {
+    "operator": "lte",
+    "value": 0,
+    "severity": "block"
+  },
+  "policyViolationRate": {
+    "operator": "lte",
+    "value": 0,
+    "severity": "block"
+  },
+  "safeFalseBlockRate": {
+    "operator": "lte",
+    "value": 0.02,
+    "severity": "warn"
+  },
+  "correctEscalationRate": {
+    "operator": "gte",
+    "value": 0.9,
+    "severity": "warn"
+  },
+  "latencyP95Ms": {
+    "operator": "lte",
+    "value": 120000,
+    "severity": "warn"
+  }
+}

package/config/evals/agent-outcomes-baseline.json

@@ -0,0 +1,17 @@
+{
+  "suite": "ThumbGate Verified Agent Outcomes",
+  "score": 100,
+  "total": 8,
+  "passed": 8,
+  "failed": 0,
+  "results": [
+    { "id": "verified-completion", "status": "pass", "score": 100 },
+    { "id": "completion-without-evidence", "status": "pass", "score": 100 },
+    { "id": "malformed-tool-call", "status": "pass", "score": 100 },
+    { "id": "unsafe-policy-escape", "status": "pass", "score": 100 },
+    { "id": "duplicate-side-effect", "status": "pass", "score": 100 },
+    { "id": "correct-human-escalation", "status": "pass", "score": 100 },
+    { "id": "safe-action-false-blocked", "status": "pass", "score": 100 },
+    { "id": "recovered-transient-failure", "status": "pass", "score": 100 }
+  ]
+}