thumbgate 1.29.2 → 1.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/.well-known/mcp/server-card.json +1 -1
- package/README.md +54 -16
- package/adapters/claude/.mcp.json +2 -2
- package/adapters/forge/forge.yaml +3 -3
- package/adapters/mcp/server-stdio.js +143 -14
- package/adapters/opencode/opencode.json +1 -1
- package/bench/observability-eval-suite.json +2 -2
- package/bin/cli.js +154 -36
- package/config/evals/generation-quality-golden.json +95 -0
- package/config/evals/rag-answer-quality-golden.json +91 -0
- package/config/evals/retrieval-hybrid-ablation.json +66 -0
- package/config/evals/retrieval-ranking-golden.json +522 -0
- package/config/gates/default.json +217 -50
- package/config/mcp-allowlists.json +215 -185
- package/config/model-tiers.json +7 -2
- package/config/post-deploy-marketing-pages.json +26 -1
- package/glama.json +6 -0
- package/package.json +94 -11
- package/public/architecture.html +130 -0
- package/public/assets/diagrams/agent-integration.png +0 -0
- package/public/assets/diagrams/before-after.svg +22 -0
- package/public/assets/diagrams/decision.svg +36 -0
- package/public/assets/diagrams/feedback-pipeline.png +0 -0
- package/public/assets/diagrams/hero-thumbs.svg +68 -0
- package/public/assets/diagrams/loop.svg +40 -0
- package/public/assets/diagrams/plugin-topology.png +0 -0
- package/public/assets/diagrams/pre-action-gate-loop.svg +59 -0
- package/public/assets/diagrams/self-improving-thumbs-loop.svg +105 -0
- package/public/assets/diagrams/stack.svg +18 -0
- package/public/assets/diagrams/thumbgate-architecture.png +0 -0
- package/public/case-studies.html +151 -0
- package/public/compare.html +1 -0
- package/public/dashboard.html +126 -28
- package/public/eval-scorecard.html +195 -0
- package/public/eval-scorecard.json +18 -0
- package/public/evaluations.html +168 -0
- package/public/index.html +143 -13
- package/public/numbers.html +3 -2
- package/public/pricing.html +143 -30
- package/public/whitepaper.html +189 -0
- package/scripts/a-plus-evidence-scorecard.js +303 -0
- package/scripts/activation-quickstart.js +1 -0
- package/scripts/agent-outcome-monitor.js +71 -1
- package/scripts/async-eval-observability.js +36 -11
- package/scripts/audit-trail.js +37 -1
- package/scripts/auto-promote-gates.js +149 -34
- package/scripts/billing.js +3 -1
- package/scripts/claude-feedback-sync.js +3 -2
- package/scripts/cli-feedback.js +13 -7
- package/scripts/colbert-style-maxsim.js +236 -0
- package/scripts/cross-encoder-reranker.js +359 -126
- package/scripts/dashboard-chat.js +350 -17
- package/scripts/document-intake.js +283 -7
- package/scripts/eval-quality-suite.js +204 -0
- package/scripts/feedback-aggregate.js +5 -2
- package/scripts/feedback-loop.js +359 -189
- package/scripts/feedback-paths.js +32 -13
- package/scripts/feedback-quality.js +53 -0
- package/scripts/filesystem-search.js +17 -7
- package/scripts/gates-engine.js +98 -4
- package/scripts/generate-case-study-outreach.js +253 -0
- package/scripts/generate-eval-scorecard.js +276 -0
- package/scripts/growth-campaigns.js +183 -0
- package/scripts/harness-tool-names.js +70 -0
- package/scripts/hook-runtime.js +10 -3
- package/scripts/jsonl-watcher.js +1 -0
- package/scripts/lesson-db.js +16 -5
- package/scripts/lesson-embedding-index.js +67 -20
- package/scripts/lesson-embedding-maintenance.js +177 -0
- package/scripts/lesson-inference.js +23 -4
- package/scripts/lesson-reranker.js +55 -9
- package/scripts/lesson-retrieval.js +375 -32
- package/scripts/lesson-search.js +48 -11
- package/scripts/llm-client.js +304 -15
- package/scripts/mcp-config.js +26 -5
- package/scripts/mcp-oauth.js +37 -2
- package/scripts/model-eval.js +308 -0
- package/scripts/model-tier-router.js +593 -0
- package/scripts/parallel-workflow-orchestrator.js +86 -22
- package/scripts/pragmatic-hybrid-search.js +379 -0
- package/scripts/published-cli.js +11 -1
- package/scripts/rag-document-pipeline.js +461 -0
- package/scripts/rag-structured-output.js +441 -0
- package/scripts/ragas-style-metrics.js +351 -0
- package/scripts/refresh-proof-pack.js +261 -0
- package/scripts/request-envelope.js +178 -0
- package/scripts/rerank-pipeline.js +370 -0
- package/scripts/rerank-quality-eval.js +155 -0
- package/scripts/retrieval-hybrid-ablation.js +120 -0
- package/scripts/retrieval-quality-tier.js +118 -0
- package/scripts/risk-scorer.js +144 -15
- package/scripts/secret-scanner.js +395 -4
- package/scripts/self-distill-agent.js +7 -1
- package/scripts/self-healing-check.js +25 -0
- package/scripts/skill-packs.js +183 -0
- package/scripts/slow-loop.js +72 -0
- package/scripts/statusline-links.js +1 -1
- package/scripts/statusline-local-stats.js +1 -1
- package/scripts/statusline.sh +8 -1
- package/scripts/telemetry-analytics.js +13 -1
- package/scripts/thumbgate-bench.js +13 -0
- package/scripts/thumbgate-search.js +98 -6
- package/scripts/tier-budget-guard.js +186 -0
- package/scripts/tool-kpi-tracker.js +124 -0
- package/scripts/tool-registry.js +95 -1
- package/scripts/vector-store.js +108 -4
- package/scripts/verify-marketing-pages-deployed.js +85 -3
- package/server.json +44 -0
- package/smithery.yaml +17 -0
- package/src/api/server.js +424 -99
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 300" role="img" aria-labelledby="t d">
|
|
2
|
+
<title id="t">ThumbGate feedback loop</title>
|
|
3
|
+
<desc id="d">Agent proposes a tool call, ThumbGate decides allow warn or deny, thumbs feedback becomes a rule that closes the loop.</desc>
|
|
4
|
+
<rect width="960" height="300" rx="24" fill="#0a0a0b"/>
|
|
5
|
+
<defs>
|
|
6
|
+
<marker id="arr" markerWidth="10" markerHeight="10" refX="8" refY="4" orient="auto"><path d="M0,0 L8,4 L0,8 Z" fill="#22d3ee"/></marker>
|
|
7
|
+
<marker id="arrG" markerWidth="10" markerHeight="10" refX="8" refY="4" orient="auto"><path d="M0,0 L8,4 L0,8 Z" fill="#4ade80"/></marker>
|
|
8
|
+
<linearGradient id="g" x1="0" y1="0" x2="1" y2="1"><stop offset="0" stop-color="#22d3ee"/><stop offset="1" stop-color="#4ade80"/></linearGradient>
|
|
9
|
+
</defs>
|
|
10
|
+
|
|
11
|
+
<!-- Step 1 -->
|
|
12
|
+
<rect x="40" y="70" width="180" height="120" rx="18" fill="#161618" stroke="#333"/>
|
|
13
|
+
<circle cx="70" cy="100" r="16" fill="#222" stroke="#22d3ee" stroke-width="2"/>
|
|
14
|
+
<text x="70" y="105" text-anchor="middle" fill="#22d3ee" font-family="system-ui,sans-serif" font-size="14" font-weight="800">1</text>
|
|
15
|
+
<text x="130" y="118" text-anchor="middle" fill="#f5f5f7" font-family="system-ui,sans-serif" font-size="18" font-weight="800">Agent</text>
|
|
16
|
+
<text x="130" y="146" text-anchor="middle" fill="#9ca3af" font-family="system-ui,sans-serif" font-size="13">proposes a tool</text>
|
|
17
|
+
<line x1="230" y1="130" x2="290" y2="130" stroke="#22d3ee" stroke-width="3" marker-end="url(#arr)"/>
|
|
18
|
+
|
|
19
|
+
<!-- Step 2 -->
|
|
20
|
+
<rect x="300" y="50" width="240" height="160" rx="18" fill="#07151a" stroke="url(#g)" stroke-width="2.5"/>
|
|
21
|
+
<circle cx="340" cy="90" r="16" fill="#0d2530" stroke="#22d3ee" stroke-width="2"/>
|
|
22
|
+
<text x="340" y="95" text-anchor="middle" fill="#22d3ee" font-family="system-ui,sans-serif" font-size="14" font-weight="800">2</text>
|
|
23
|
+
<text x="420" y="95" text-anchor="middle" fill="#22d3ee" font-family="system-ui,sans-serif" font-size="20" font-weight="900">ThumbGate</text>
|
|
24
|
+
<text x="420" y="130" text-anchor="middle" fill="#f5f5f7" font-family="system-ui,sans-serif" font-size="16" font-weight="700">ALLOW · WARN · DENY</text>
|
|
25
|
+
<text x="420" y="158" text-anchor="middle" fill="#9ca3af" font-family="system-ui,sans-serif" font-size="13">before the tool runs</text>
|
|
26
|
+
<text x="420" y="184" text-anchor="middle" fill="#4ade80" font-family="system-ui,sans-serif" font-size="18" font-weight="800">👍 👎</text>
|
|
27
|
+
<line x1="550" y1="130" x2="610" y2="130" stroke="#22d3ee" stroke-width="3" marker-end="url(#arr)"/>
|
|
28
|
+
|
|
29
|
+
<!-- Step 3 -->
|
|
30
|
+
<rect x="620" y="70" width="180" height="120" rx="18" fill="#161618" stroke="#333"/>
|
|
31
|
+
<circle cx="650" cy="100" r="16" fill="#222" stroke="#4ade80" stroke-width="2"/>
|
|
32
|
+
<text x="650" y="105" text-anchor="middle" fill="#4ade80" font-family="system-ui,sans-serif" font-size="14" font-weight="800">3</text>
|
|
33
|
+
<text x="710" y="118" text-anchor="middle" fill="#f5f5f7" font-family="system-ui,sans-serif" font-size="18" font-weight="800">Action</text>
|
|
34
|
+
<text x="710" y="146" text-anchor="middle" fill="#9ca3af" font-family="system-ui,sans-serif" font-size="13">shell · git · db</text>
|
|
35
|
+
|
|
36
|
+
<!-- Feedback loop -->
|
|
37
|
+
<path d="M710 200 C710 250 130 250 130 200" fill="none" stroke="#4ade80" stroke-width="3" stroke-dasharray="6 4" marker-end="url(#arrG)"/>
|
|
38
|
+
<rect x="260" y="232" width="320" height="40" rx="20" fill="#0d1b14" stroke="#4ade80"/>
|
|
39
|
+
<text x="420" y="258" text-anchor="middle" fill="#4ade80" font-family="system-ui,sans-serif" font-size="15" font-weight="800">👍 👎 feedback → lesson → rule → gate</text>
|
|
40
|
+
</svg>
|
|
Binary file
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="520" viewBox="0 0 1200 520" role="img" aria-labelledby="title desc">
|
|
2
|
+
<title id="title">ThumbGate pre-action gate loop</title>
|
|
3
|
+
<desc id="desc">Diagram showing an AI agent proposing a tool call, ThumbGate checking it before execution, then either blocking repeated failures or allowing safe actions with an audit receipt.</desc>
|
|
4
|
+
<defs>
|
|
5
|
+
<linearGradient id="bg" x1="0" x2="1" y1="0" y2="1">
|
|
6
|
+
<stop offset="0%" stop-color="#081014"/>
|
|
7
|
+
<stop offset="100%" stop-color="#111113"/>
|
|
8
|
+
</linearGradient>
|
|
9
|
+
<linearGradient id="cyan" x1="0" x2="1" y1="0" y2="0">
|
|
10
|
+
<stop offset="0%" stop-color="#22d3ee"/>
|
|
11
|
+
<stop offset="100%" stop-color="#4ade80"/>
|
|
12
|
+
</linearGradient>
|
|
13
|
+
<filter id="shadow" x="-10%" y="-10%" width="120%" height="120%">
|
|
14
|
+
<feDropShadow dx="0" dy="16" stdDeviation="18" flood-color="#000000" flood-opacity="0.35"/>
|
|
15
|
+
</filter>
|
|
16
|
+
<marker id="arrow" markerWidth="12" markerHeight="12" refX="10" refY="6" orient="auto">
|
|
17
|
+
<path d="M2,2 L10,6 L2,10 Z" fill="#22d3ee"/>
|
|
18
|
+
</marker>
|
|
19
|
+
</defs>
|
|
20
|
+
<rect width="1200" height="520" rx="28" fill="url(#bg)"/>
|
|
21
|
+
<text x="60" y="72" fill="#e8e8ec" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="34" font-weight="800">ThumbGate checks agent intent before execution</text>
|
|
22
|
+
<text x="60" y="106" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="17">Feedback becomes rules. Rules become PreToolUse gates. Gates produce receipts before files, APIs, or production systems change.</text>
|
|
23
|
+
|
|
24
|
+
<g filter="url(#shadow)">
|
|
25
|
+
<rect x="60" y="170" width="235" height="145" rx="18" fill="#161618" stroke="#222225"/>
|
|
26
|
+
<text x="86" y="205" fill="#22d3ee" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13" font-weight="800" letter-spacing="1.4">AGENT INTENT</text>
|
|
27
|
+
<text x="86" y="244" fill="#e8e8ec" font-family="SFMono-Regular,Consolas,monospace" font-size="18">git push --force</text>
|
|
28
|
+
<text x="86" y="272" fill="#e8e8ec" font-family="SFMono-Regular,Consolas,monospace" font-size="18">origin main</text>
|
|
29
|
+
<text x="86" y="296" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13">Still only proposed. Nothing has run.</text>
|
|
30
|
+
|
|
31
|
+
<rect x="390" y="160" width="270" height="165" rx="18" fill="#10181a" stroke="#22d3ee" stroke-opacity="0.52"/>
|
|
32
|
+
<text x="416" y="196" fill="#22d3ee" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13" font-weight="800" letter-spacing="1.4">PRETOOLUSE GATE</text>
|
|
33
|
+
<text x="416" y="232" fill="#e8e8ec" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="700">Known repeated failure?</text>
|
|
34
|
+
<text x="416" y="262" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="14">Checks active rules, protected paths, approvals,</text>
|
|
35
|
+
<text x="416" y="284" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="14">secrets, evidence, and workflow scope.</text>
|
|
36
|
+
|
|
37
|
+
<rect x="755" y="120" width="245" height="135" rx="18" fill="#201113" stroke="#f87171" stroke-opacity="0.55"/>
|
|
38
|
+
<text x="781" y="154" fill="#f87171" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13" font-weight="800" letter-spacing="1.4">BLOCK</text>
|
|
39
|
+
<text x="781" y="191" fill="#e8e8ec" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="700">Stop before damage</text>
|
|
40
|
+
<text x="781" y="220" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="14">Return the rule and safer next step.</text>
|
|
41
|
+
|
|
42
|
+
<rect x="755" y="305" width="245" height="135" rx="18" fill="#102017" stroke="#4ade80" stroke-opacity="0.55"/>
|
|
43
|
+
<text x="781" y="339" fill="#4ade80" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13" font-weight="800" letter-spacing="1.4">ALLOW</text>
|
|
44
|
+
<text x="781" y="376" fill="#e8e8ec" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="700">Run and log receipt</text>
|
|
45
|
+
<text x="781" y="405" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="14">Safe actions keep moving with proof.</text>
|
|
46
|
+
</g>
|
|
47
|
+
|
|
48
|
+
<path d="M310 242 H370" stroke="#22d3ee" stroke-width="3" marker-end="url(#arrow)"/>
|
|
49
|
+
<path d="M670 230 C710 190 725 175 745 170" stroke="#f87171" stroke-width="3" fill="none" marker-end="url(#arrow)"/>
|
|
50
|
+
<path d="M670 270 C710 335 725 365 745 375" stroke="#4ade80" stroke-width="3" fill="none" marker-end="url(#arrow)"/>
|
|
51
|
+
|
|
52
|
+
<g>
|
|
53
|
+
<rect x="60" y="380" width="600" height="70" rx="16" fill="#0a0a0b" stroke="#222225"/>
|
|
54
|
+
<text x="86" y="412" fill="#8b8b96" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13" font-weight="800" letter-spacing="1.4">FEEDBACK LOOP</text>
|
|
55
|
+
<text x="86" y="438" fill="#e8e8ec" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="16">Thumbs-down becomes a lesson, then a prevention rule shared across agents.</text>
|
|
56
|
+
</g>
|
|
57
|
+
<path d="M820 120 C760 70 240 72 196 165" stroke="url(#cyan)" stroke-width="3" fill="none" stroke-dasharray="8 8" marker-end="url(#arrow)"/>
|
|
58
|
+
<text x="900" y="486" fill="#22d3ee" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="16" font-weight="800">npx thumbgate init</text>
|
|
59
|
+
</svg>
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1100 420" role="img" aria-labelledby="title desc">
|
|
2
|
+
<title id="title">How ThumbGate self-improves from thumbs up and thumbs down</title>
|
|
3
|
+
<desc id="desc">Closed loop: you give thumbs up or thumbs down, ThumbGate stores a local lesson, promotes a prevention rule, and gates the next tool call before it runs.</desc>
|
|
4
|
+
<defs>
|
|
5
|
+
<linearGradient id="bg" x1="0" y1="0" x2="1" y2="1">
|
|
6
|
+
<stop offset="0%" stop-color="#081018"/>
|
|
7
|
+
<stop offset="100%" stop-color="#0d121c"/>
|
|
8
|
+
</linearGradient>
|
|
9
|
+
<linearGradient id="ring" x1="0" y1="0" x2="1" y2="1">
|
|
10
|
+
<stop offset="0%" stop-color="#22d3ee"/>
|
|
11
|
+
<stop offset="100%" stop-color="#4ade80"/>
|
|
12
|
+
</linearGradient>
|
|
13
|
+
<linearGradient id="upGlow" x1="0" y1="0" x2="0" y2="1">
|
|
14
|
+
<stop offset="0%" stop-color="#56e39f"/>
|
|
15
|
+
<stop offset="100%" stop-color="#1f9d62"/>
|
|
16
|
+
</linearGradient>
|
|
17
|
+
<linearGradient id="downGlow" x1="0" y1="0" x2="0" y2="1">
|
|
18
|
+
<stop offset="0%" stop-color="#ff8a9a"/>
|
|
19
|
+
<stop offset="100%" stop-color="#e11d48"/>
|
|
20
|
+
</linearGradient>
|
|
21
|
+
<filter id="soft" x="-20%" y="-20%" width="140%" height="140%">
|
|
22
|
+
<feDropShadow dx="0" dy="12" stdDeviation="16" flood-color="#000" flood-opacity="0.45"/>
|
|
23
|
+
</filter>
|
|
24
|
+
<marker id="arr" markerWidth="10" markerHeight="10" refX="8" refY="4" orient="auto">
|
|
25
|
+
<path d="M0,0 L8,4 L0,8 Z" fill="#22d3ee"/>
|
|
26
|
+
</marker>
|
|
27
|
+
<marker id="arrG" markerWidth="10" markerHeight="10" refX="8" refY="4" orient="auto">
|
|
28
|
+
<path d="M0,0 L8,4 L0,8 Z" fill="#4ade80"/>
|
|
29
|
+
</marker>
|
|
30
|
+
</defs>
|
|
31
|
+
|
|
32
|
+
<rect width="1100" height="420" rx="28" fill="url(#bg)"/>
|
|
33
|
+
|
|
34
|
+
<!-- Header -->
|
|
35
|
+
<text x="48" y="48" fill="#e8eef7" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="28" font-weight="800">Yes — self-improving. Not the model. The gate.</text>
|
|
36
|
+
<text x="48" y="78" fill="#9ca8ba" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="15">Your 👍 / 👎 teach local rules. Next matching tool call is checked before it runs.</text>
|
|
37
|
+
|
|
38
|
+
<!-- Giant thumbs pair -->
|
|
39
|
+
<g filter="url(#soft)" transform="translate(48, 110)">
|
|
40
|
+
<!-- Thumbs up card -->
|
|
41
|
+
<rect x="0" y="0" width="150" height="180" rx="22" fill="#0d1f18" stroke="#4ade80" stroke-width="2"/>
|
|
42
|
+
<circle cx="75" cy="72" r="42" fill="url(#upGlow)" opacity="0.18"/>
|
|
43
|
+
<!-- thumbs-up hand icon -->
|
|
44
|
+
<g transform="translate(42, 38) scale(1.45)" fill="none" stroke="#4ade80" stroke-width="3.2" stroke-linecap="round" stroke-linejoin="round">
|
|
45
|
+
<path d="M7 22V13a2 2 0 0 1 2-2h2.5l3.2-7.2A2.5 2.5 0 0 1 17 2.5 3.5 3.5 0 0 1 20.5 6v5H28a3 3 0 0 1 3 3.4l-1.6 9A3 3 0 0 1 26.5 26H9a2 2 0 0 1-2-2z"/>
|
|
46
|
+
<path d="M7 13H4a2 2 0 0 0-2 2v9a2 2 0 0 0 2 2h3"/>
|
|
47
|
+
</g>
|
|
48
|
+
<text x="75" y="148" text-anchor="middle" fill="#4ade80" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="800">👍 Keep</text>
|
|
49
|
+
<text x="75" y="170" text-anchor="middle" fill="#8fa09a" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="12">reinforce good pattern</text>
|
|
50
|
+
|
|
51
|
+
<!-- Thumbs down card -->
|
|
52
|
+
<rect x="170" y="0" width="150" height="180" rx="22" fill="#1f1014" stroke="#ff6480" stroke-width="2"/>
|
|
53
|
+
<circle cx="245" cy="72" r="42" fill="url(#downGlow)" opacity="0.18"/>
|
|
54
|
+
<!-- thumbs-down hand icon -->
|
|
55
|
+
<g transform="translate(212, 38) scale(1.45)" fill="none" stroke="#ff6480" stroke-width="3.2" stroke-linecap="round" stroke-linejoin="round">
|
|
56
|
+
<path d="M7 2v9a2 2 0 0 0 2 2h2.5l3.2 7.2A2.5 2.5 0 0 0 17 21.5 3.5 3.5 0 0 0 20.5 18v-5H28a3 3 0 0 0 3-3.4l-1.6-9A3 3 0 0 0 26.5-2H9a2 2 0 0 0-2 2z" transform="translate(0 8)"/>
|
|
57
|
+
<path d="M7 11H4a2 2 0 0 1-2-2V0a2 2 0 0 1 2-2h3" transform="translate(0 8)"/>
|
|
58
|
+
</g>
|
|
59
|
+
<text x="245" y="148" text-anchor="middle" fill="#ff8a9a" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="800">👎 Fix</text>
|
|
60
|
+
<text x="245" y="170" text-anchor="middle" fill="#b08a93" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="12">never repeat this</text>
|
|
61
|
+
</g>
|
|
62
|
+
|
|
63
|
+
<!-- Arrow to lesson -->
|
|
64
|
+
<path d="M340 200 H390" stroke="#22d3ee" stroke-width="3" marker-end="url(#arr)"/>
|
|
65
|
+
|
|
66
|
+
<!-- Step 2: Local lesson -->
|
|
67
|
+
<g filter="url(#soft)">
|
|
68
|
+
<rect x="400" y="130" width="180" height="140" rx="20" fill="#111722" stroke="#273246" stroke-width="2"/>
|
|
69
|
+
<text x="490" y="168" text-anchor="middle" fill="#22d3ee" font-family="ui-monospace,SFMono-Regular,Menlo,monospace" font-size="12" font-weight="800" letter-spacing="1.2">LOCAL LESSON</text>
|
|
70
|
+
<text x="490" y="202" text-anchor="middle" fill="#f5f7fb" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="800">SQLite memory</text>
|
|
71
|
+
<text x="490" y="228" text-anchor="middle" fill="#9ca8ba" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13">survives sessions</text>
|
|
72
|
+
<text x="490" y="248" text-anchor="middle" fill="#9ca8ba" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13">not model weights</text>
|
|
73
|
+
</g>
|
|
74
|
+
|
|
75
|
+
<path d="M590 200 H640" stroke="#22d3ee" stroke-width="3" marker-end="url(#arr)"/>
|
|
76
|
+
|
|
77
|
+
<!-- Step 3: Rule / gate -->
|
|
78
|
+
<g filter="url(#soft)">
|
|
79
|
+
<rect x="650" y="130" width="180" height="140" rx="20" fill="#0c1820" stroke="url(#ring)" stroke-width="2.5"/>
|
|
80
|
+
<text x="740" y="168" text-anchor="middle" fill="#22d3ee" font-family="ui-monospace,SFMono-Regular,Menlo,monospace" font-size="12" font-weight="800" letter-spacing="1.2">PREVENTION RULE</text>
|
|
81
|
+
<text x="740" y="202" text-anchor="middle" fill="#f5f7fb" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="18" font-weight="800">Auto-promoted</text>
|
|
82
|
+
<text x="740" y="228" text-anchor="middle" fill="#9ca8ba" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13">from repeats</text>
|
|
83
|
+
<text x="740" y="248" text-anchor="middle" fill="#9ca8ba" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="13">stale rules expire</text>
|
|
84
|
+
</g>
|
|
85
|
+
|
|
86
|
+
<path d="M840 200 H890" stroke="#22d3ee" stroke-width="3" marker-end="url(#arr)"/>
|
|
87
|
+
|
|
88
|
+
<!-- Step 4: Gate next action -->
|
|
89
|
+
<g filter="url(#soft)">
|
|
90
|
+
<rect x="900" y="118" width="168" height="164" rx="20" fill="#111722" stroke="#273246" stroke-width="2"/>
|
|
91
|
+
<text x="984" y="152" text-anchor="middle" fill="#22d3ee" font-family="ui-monospace,SFMono-Regular,Menlo,monospace" font-size="12" font-weight="800" letter-spacing="1.2">NEXT TOOL CALL</text>
|
|
92
|
+
<rect x="922" y="168" width="52" height="26" rx="13" fill="rgba(86,227,159,0.12)" stroke="#56e39f"/>
|
|
93
|
+
<text x="948" y="186" text-anchor="middle" fill="#56e39f" font-family="ui-monospace,SFMono-Regular,Menlo,monospace" font-size="11" font-weight="800">ALLOW</text>
|
|
94
|
+
<rect x="982" y="168" width="48" height="26" rx="13" fill="rgba(255,209,102,0.12)" stroke="#ffd166"/>
|
|
95
|
+
<text x="1006" y="186" text-anchor="middle" fill="#ffd166" font-family="ui-monospace,SFMono-Regular,Menlo,monospace" font-size="11" font-weight="800">WARN</text>
|
|
96
|
+
<rect x="948" y="206" width="52" height="26" rx="13" fill="rgba(255,100,124,0.12)" stroke="#ff647c"/>
|
|
97
|
+
<text x="974" y="224" text-anchor="middle" fill="#ff647c" font-family="ui-monospace,SFMono-Regular,Menlo,monospace" font-size="11" font-weight="800">DENY</text>
|
|
98
|
+
<text x="984" y="258" text-anchor="middle" fill="#f5f7fb" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="15" font-weight="700">before it runs</text>
|
|
99
|
+
</g>
|
|
100
|
+
|
|
101
|
+
<!-- Closing the loop dashed arrow back -->
|
|
102
|
+
<path d="M984 300 C984 360 120 360 120 300" fill="none" stroke="#4ade80" stroke-width="3" stroke-dasharray="8 6" marker-end="url(#arrG)"/>
|
|
103
|
+
<rect x="360" y="332" width="380" height="40" rx="20" fill="#0d1b14" stroke="#4ade80"/>
|
|
104
|
+
<text x="550" y="358" text-anchor="middle" fill="#4ade80" font-family="-apple-system,BlinkMacSystemFont,Segoe UI,sans-serif" font-size="15" font-weight="800">Loop closes · under your control · no model retrain</text>
|
|
105
|
+
</svg>
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 960 240" role="img" aria-labelledby="t d">
|
|
2
|
+
<title id="t">Where ThumbGate sits in the stack</title>
|
|
3
|
+
<desc id="d">Model to agent runtime to ThumbGate pre-action gate to tools.</desc>
|
|
4
|
+
<rect width="960" height="240" fill="#0a0a0b"/>
|
|
5
|
+
<g font-family="system-ui,sans-serif">
|
|
6
|
+
<rect x="60" y="36" width="840" height="40" rx="10" fill="#161618" stroke="#333"/>
|
|
7
|
+
<text x="480" y="62" text-anchor="middle" fill="#9ca3af" font-size="15" font-weight="600">Model (Claude · GPT · Gemini · local)</text>
|
|
8
|
+
<text x="480" y="100" text-anchor="middle" fill="#22d3ee" font-size="22">↓</text>
|
|
9
|
+
<rect x="60" y="112" width="840" height="40" rx="10" fill="#161618" stroke="#333"/>
|
|
10
|
+
<text x="480" y="138" text-anchor="middle" fill="#e5e7eb" font-size="15" font-weight="600">Agent runtime (Claude Code · Cursor · Codex · …)</text>
|
|
11
|
+
<text x="480" y="176" text-anchor="middle" fill="#22d3ee" font-size="22">↓</text>
|
|
12
|
+
<rect x="60" y="188" width="400" height="40" rx="10" fill="#07151a" stroke="#22d3ee" stroke-width="2"/>
|
|
13
|
+
<text x="260" y="214" text-anchor="middle" fill="#22d3ee" font-size="15" font-weight="800">ThumbGate pre-action gate</text>
|
|
14
|
+
<text x="490" y="214" text-anchor="middle" fill="#4ade80" font-size="20">→</text>
|
|
15
|
+
<rect x="520" y="188" width="380" height="40" rx="10" fill="#161618" stroke="#333"/>
|
|
16
|
+
<text x="710" y="214" text-anchor="middle" fill="#e5e7eb" font-size="15" font-weight="600">shell · git · db · MCP tools</text>
|
|
17
|
+
</g>
|
|
18
|
+
</svg>
|
|
Binary file
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html lang="en">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="UTF-8">
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
+
<meta name="generator" content="ThumbGate">
|
|
7
|
+
<meta name="author" content="Igor Ganapolsky">
|
|
8
|
+
<title>Case Studies — ThumbGate Dogfood Proof</title>
|
|
9
|
+
<meta name="description" content="Real ThumbGate case studies from dogfood on this product: sudo evasion grid, fail-open hook recovery, catastrophic free-tier floor, and adversarial measurement. No fabricated logos.">
|
|
10
|
+
<meta property="og:title" content="ThumbGate Case Studies">
|
|
11
|
+
<meta property="og:description" content="Five reproducible dogfood narratives: 62 evasion holes closed, fail-open recovery, catastrophic safety floor, and adversarial grids.">
|
|
12
|
+
<meta property="og:type" content="article">
|
|
13
|
+
<meta property="og:url" content="https://thumbgate.ai/case-studies">
|
|
14
|
+
<link rel="canonical" href="https://thumbgate.ai/case-studies">
|
|
15
|
+
<link rel="icon" type="image/png" href="/thumbgate-icon.png">
|
|
16
|
+
<script defer data-domain="thumbgate.ai" src="https://plausible.io/js/script.js"></script>
|
|
17
|
+
<script type="application/ld+json">
|
|
18
|
+
{
|
|
19
|
+
"@context": "https://schema.org",
|
|
20
|
+
"@type": "CollectionPage",
|
|
21
|
+
"name": "ThumbGate Case Studies",
|
|
22
|
+
"description": "Reproducible dogfood case studies for ThumbGate pre-action checks. No fabricated customer logos.",
|
|
23
|
+
"url": "https://thumbgate.ai/case-studies",
|
|
24
|
+
"dateModified": "2026-07-29"
|
|
25
|
+
}
|
|
26
|
+
</script>
|
|
27
|
+
<style>
|
|
28
|
+
:root {
|
|
29
|
+
--bg:#0b0f14; --panel:#111823; --border:#1e2a3a; --text:#e6edf3;
|
|
30
|
+
--muted:#8b98a5; --cyan:#39c5cf; --green:#3fb950; --amber:#d29922;
|
|
31
|
+
}
|
|
32
|
+
* { margin:0; padding:0; box-sizing:border-box; }
|
|
33
|
+
body { background:var(--bg); color:var(--text); font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,sans-serif; line-height:1.65; }
|
|
34
|
+
nav { padding:1rem 2rem; border-bottom:1px solid var(--border); display:flex; gap:1.25rem; flex-wrap:wrap; align-items:center; }
|
|
35
|
+
nav a { color:var(--muted); text-decoration:none; font-size:0.9rem; }
|
|
36
|
+
nav a:hover { color:var(--cyan); }
|
|
37
|
+
nav .brand { color:var(--text); font-weight:700; }
|
|
38
|
+
.container { max-width:900px; margin:0 auto; padding:2.5rem 1.5rem 4rem; }
|
|
39
|
+
h1 { font-size:2rem; margin-bottom:0.4rem; }
|
|
40
|
+
h2 { font-size:1.25rem; margin:0 0 0.35rem; }
|
|
41
|
+
.subtitle { color:var(--muted); font-size:1.05rem; margin-bottom:1.5rem; }
|
|
42
|
+
article { background:var(--panel); border:1px solid var(--border); border-radius:12px; padding:1.35rem 1.4rem; margin:1.1rem 0; }
|
|
43
|
+
.meta { color:var(--muted); font-size:0.85rem; margin-bottom:0.75rem; }
|
|
44
|
+
.metric { display:inline-block; background:#0d1420; border:1px solid var(--border); color:var(--cyan); padding:0.2rem 0.55rem; border-radius:6px; font-size:0.82rem; font-weight:600; margin:0 0.35rem 0.35rem 0; }
|
|
45
|
+
p { margin:0.55rem 0; }
|
|
46
|
+
pre { background:#0d1420; border:1px solid var(--border); border-radius:8px; padding:0.85rem 1rem; overflow-x:auto; font-size:0.84rem; margin:0.75rem 0; }
|
|
47
|
+
code { color:var(--cyan); }
|
|
48
|
+
a.inline { color:var(--cyan); }
|
|
49
|
+
a.cta { display:inline-block; background:var(--cyan); color:#001018; font-weight:700; padding:0.7rem 1rem; border-radius:8px; text-decoration:none; margin:0.5rem 0.6rem 0.5rem 0; }
|
|
50
|
+
.note { border-left:3px solid var(--amber); background:var(--panel); padding:0.7rem 1rem; margin:1.25rem 0; color:var(--muted); }
|
|
51
|
+
.footer-note { color:var(--muted); font-size:0.85rem; margin-top:2.5rem; text-align:center; }
|
|
52
|
+
.footer-note a { color:var(--cyan); text-decoration:none; }
|
|
53
|
+
</style>
|
|
54
|
+
</head>
|
|
55
|
+
<body>
|
|
56
|
+
<nav>
|
|
57
|
+
<a class="brand" href="/">ThumbGate</a>
|
|
58
|
+
<a href="/whitepaper">White paper</a>
|
|
59
|
+
<a href="/architecture">Architecture</a>
|
|
60
|
+
<a href="/eval-scorecard">Scorecard</a>
|
|
61
|
+
<a href="/evaluations">Evaluations</a>
|
|
62
|
+
<a href="/case-studies">Case studies</a>
|
|
63
|
+
<a href="/numbers">Numbers</a>
|
|
64
|
+
<a href="/pricing">Pricing</a>
|
|
65
|
+
</nav>
|
|
66
|
+
<main class="container">
|
|
67
|
+
<h1>Case studies</h1>
|
|
68
|
+
<p class="subtitle">Real work on this codebase. No fabricated logos, no aspirational customer counts — every technical claim is verifiable against the open repo or the commands recorded in the full write-up.</p>
|
|
69
|
+
|
|
70
|
+
<div class="note">
|
|
71
|
+
<strong>Honesty boundary:</strong> these are first-party dogfood / incident narratives, not third-party customer ROI claims.
|
|
72
|
+
Full longform: <a class="inline" href="https://github.com/IgorGanapolsky/ThumbGate/blob/main/docs/THUMBGATE-CASE-STUDIES.md">docs/THUMBGATE-CASE-STUDIES.md</a>.
|
|
73
|
+
</div>
|
|
74
|
+
|
|
75
|
+
<article id="sudo-evasion">
|
|
76
|
+
<h2>1. A guardrail you could walk past with <code>sudo</code></h2>
|
|
77
|
+
<p class="meta">Hard technical problem · shipped in v1.29.2</p>
|
|
78
|
+
<span class="metric">62 evasion holes → 0</span>
|
|
79
|
+
<span class="metric">14 commands × 9 transforms</span>
|
|
80
|
+
<span class="metric">published tarball verified</span>
|
|
81
|
+
<p><strong>Problem.</strong> Catastrophic gates (<code>rm -rf ~</code>, force-push, hard reset) matched the happy path but missed wrappers: <code>sudo rm -rf ~</code> produced no match on 1.29.1.</p>
|
|
82
|
+
<p><strong>Action.</strong> Built an adversarial grid instead of hand-picking examples; canonicalized commands centrally; re-evaluated only when the original form missed.</p>
|
|
83
|
+
<pre>rm -rf ~ → deny
|
|
84
|
+
sudo rm -rf ~ → deny (after fix)
|
|
85
|
+
thumbgate@1.29.1 → 34+ holes
|
|
86
|
+
thumbgate@1.29.2 → 0 holes</pre>
|
|
87
|
+
<p><strong>Artifact.</strong> <code>tests/gate-evasion-matrix.test.js</code> in CI + 6-hourly job against the published npm artifact.</p>
|
|
88
|
+
</article>
|
|
89
|
+
|
|
90
|
+
<article id="hypothesis-report">
|
|
91
|
+
<h2>2. A seven-defect report where most defects weren’t real</h2>
|
|
92
|
+
<p class="meta">Ambiguous problem · investigation discipline</p>
|
|
93
|
+
<span class="metric">2 of 7 defects real</span>
|
|
94
|
+
<span class="metric">disconfirming evidence first</span>
|
|
95
|
+
<p><strong>Problem.</strong> A confident multi-defect report with statistics would have forced a core rewrite if accepted wholesale.</p>
|
|
96
|
+
<p><strong>Action.</strong> Treated each claim as a hypothesis: read the consumer’s config, recompute totals from source data, reproduce before believing.</p>
|
|
97
|
+
<p><strong>Result.</strong> Strict-mode env explained “warn-by-default not working.” Filtered tables understated warns. The investigation also surfaced the sudo evasion class above — worse than anything in the original report.</p>
|
|
98
|
+
</article>
|
|
99
|
+
|
|
100
|
+
<article id="fail-open">
|
|
101
|
+
<h2>3. Production failure: a firewall enforcing nothing</h2>
|
|
102
|
+
<p class="meta">Incident · fail-open PreToolUse hook</p>
|
|
103
|
+
<span class="metric">hook binary missing</span>
|
|
104
|
+
<span class="metric">erroring hook = fail open</span>
|
|
105
|
+
<span class="metric">silent-gate canary added</span>
|
|
106
|
+
<p><strong>Problem.</strong> Local state vanished and <code>thumbgate-hook</code> was missing. An erroring PreToolUse hook fails open — the product looked “fine” while blocking nothing.</p>
|
|
107
|
+
<p><strong>Action.</strong> Restored via <code>thumbgate init</code>, then proved enforcement with the same dangerous commands that must deny.</p>
|
|
108
|
+
<p><strong>Lesson.</strong> This product’s failure mode is silence. Uptime green is not enough; known-dangerous commands must still be denied on a schedule against the published artifact.</p>
|
|
109
|
+
</article>
|
|
110
|
+
|
|
111
|
+
<article id="catastrophic-floor">
|
|
112
|
+
<h2>4. Monetization that almost sold off the safety floor</h2>
|
|
113
|
+
<p class="meta">Stakeholder conflict · product policy</p>
|
|
114
|
+
<span class="metric">catastrophic gates exempt from daily cap</span>
|
|
115
|
+
<span class="metric">hotfix bypass still hard-floors secrets</span>
|
|
116
|
+
<p><strong>Problem.</strong> Free-tier daily block caps + warn-by-default posture could downgrade irreversible actions to warnings after the cap.</p>
|
|
117
|
+
<p><strong>Action.</strong> Split <em>how many</em> blocks free users get from <em>which</em> actions can never be discounted. Named catastrophic IDs stay deny regardless of tier.</p>
|
|
118
|
+
<p><strong>Result.</strong> Upgrade pressure remains on negotiable gates; irreversible classes are not for sale.</p>
|
|
119
|
+
</article>
|
|
120
|
+
|
|
121
|
+
<article id="evasion-matrix">
|
|
122
|
+
<h2>5. Built the measurement before claiming “fixed”</h2>
|
|
123
|
+
<p class="meta">Instrument first · property tests</p>
|
|
124
|
+
<span class="metric">relative invariant</span>
|
|
125
|
+
<span class="metric">vacuity guards</span>
|
|
126
|
+
<span class="metric">artifact, not only source</span>
|
|
127
|
+
<p><strong>Problem.</strong> “We fixed the bypass class” had been claimed twice and been wrong twice.</p>
|
|
128
|
+
<p><strong>Action.</strong> Built a property suite over commands × spellings, with sandboxing and a minimum number of plain-form denials so empty measurements cannot pass.</p>
|
|
129
|
+
<p><strong>Result.</strong> “Is it fixed?” became a command with a number, including against the npm tarball users actually install.</p>
|
|
130
|
+
</article>
|
|
131
|
+
|
|
132
|
+
<p>
|
|
133
|
+
<a class="cta" href="/whitepaper">Evaluation white paper</a>
|
|
134
|
+
<a class="cta" href="/eval-scorecard">Bench scorecard</a>
|
|
135
|
+
<a class="cta" href="/diagnostic">$499 diagnostic gate</a>
|
|
136
|
+
</p>
|
|
137
|
+
|
|
138
|
+
<p class="footer-note">
|
|
139
|
+
Want a customer case study here? Ship with ThumbGate and write <a href="mailto:igor.ganapolsky@gmail.com">igor.ganapolsky@gmail.com</a>.
|
|
140
|
+
We only publish reproducible claims.
|
|
141
|
+
</p>
|
|
142
|
+
<p class="footer-note">
|
|
143
|
+
<a href="/">Home</a> ·
|
|
144
|
+
<a href="/pricing">Pricing</a> ·
|
|
145
|
+
<a href="/privacy">Privacy</a> ·
|
|
146
|
+
<a href="/terms">Terms</a> ·
|
|
147
|
+
<a href="/support">Support</a>
|
|
148
|
+
</p>
|
|
149
|
+
</main>
|
|
150
|
+
</body>
|
|
151
|
+
</html>
|
package/public/compare.html
CHANGED
|
@@ -296,6 +296,7 @@
|
|
|
296
296
|
<li><a href="/compare/arcade">ThumbGate vs Arcade</a> — coding-agent guardrails vs enterprise API auth delegation</li>
|
|
297
297
|
<li><a href="/compare/databricks-unity-ai-gateway">ThumbGate vs Databricks Unity AI Gateway</a> — enterprise gateway vs local pre-action gates</li>
|
|
298
298
|
<li><a href="/compare/hermes-everos-memory">ThumbGate vs Hermes and EverOS</a> — local-first behavior gates vs passive agent memory stores</li>
|
|
299
|
+
<li><a href="/compare/far-ai">ThumbGate vs FAR.AI</a> — runtime agent enforcement vs frontier safety research and evaluation</li>
|
|
299
300
|
</ul>
|
|
300
301
|
|
|
301
302
|
<h2>How It Works</h2>
|