skills-ws 1.5.3 → 1.5.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# skills.ws
|
|
2
2
|
|
|
3
|
-
Agent skills for AI coding assistants.
|
|
3
|
+
Agent skills for AI coding assistants. 83 skills across 8 categories — built for OpenClaw, Claude Code, Cursor, Codex, and any agent that supports the SKILL.md format.
|
|
4
4
|
|
|
5
5
|
**Website:** [skills.ws](https://skills.ws) | **npm:** [skills-ws](https://www.npmjs.com/package/skills-ws) | **Docs:** [llms-full.txt](https://skills.ws/llms-full.txt)
|
|
6
6
|
|
|
@@ -158,7 +158,7 @@ skills-ws/
|
|
|
158
158
|
│ └── skills.ts # Skill data access + TypeScript interfaces
|
|
159
159
|
├── skills/ # Raw SKILL.md files (81 directories)
|
|
160
160
|
├── public/
|
|
161
|
-
│ ├── skills.json # Skills database (
|
|
161
|
+
│ ├── skills.json # Skills database (83 skills, all metadata + content)
|
|
162
162
|
│ ├── llms.txt # LLM-readable skill index
|
|
163
163
|
│ ├── llms-full.txt # Full content dump for LLMs
|
|
164
164
|
│ ├── robots.txt # Crawl directives
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "skills-ws",
|
|
3
|
-
"version": "1.5.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "1.5.5",
|
|
4
|
+
"description": "83 agent skills for AI coding assistants \u2014 marketing, growth, web3, dev, design & operations. Built for OpenClaw, Claude Code, Cursor, and Codex.",
|
|
5
5
|
"scripts": {
|
|
6
6
|
"test": "node test/cli.test.mjs"
|
|
7
7
|
},
|
|
@@ -0,0 +1,411 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: polymarket-trading
|
|
3
|
+
description: Polymarket prediction market trading — market analysis, edge calculation, bookmaker cross-referencing, order placement via CLOB API, position management, and redemption. Covers sports betting strategy, risk management, and the full Polymarket SDK workflow.
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Polymarket Trading
|
|
8
|
+
|
|
9
|
+
Complete framework for analyzing, trading, and managing positions on Polymarket — the world's largest prediction market.
|
|
10
|
+
|
|
11
|
+
## Table of Contents
|
|
12
|
+
|
|
13
|
+
1. [Market Analysis Framework](#market-analysis-framework)
|
|
14
|
+
2. [Edge Calculation](#edge-calculation)
|
|
15
|
+
3. [Bookmaker Cross-Referencing](#bookmaker-cross-referencing)
|
|
16
|
+
4. [Risk Management Rules](#risk-management-rules)
|
|
17
|
+
5. [APIs & Data Sources](#apis--data-sources)
|
|
18
|
+
6. [Trading via CLOB](#trading-via-clob)
|
|
19
|
+
7. [Position Management & Redemption](#position-management--redemption)
|
|
20
|
+
8. [Understanding Polymarket Mechanics](#understanding-polymarket-mechanics)
|
|
21
|
+
9. [Common Pitfalls](#common-pitfalls)
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Market Analysis Framework
|
|
26
|
+
|
|
27
|
+
### The Scan Pipeline
|
|
28
|
+
|
|
29
|
+
For every potential bet, follow this pipeline in order:
|
|
30
|
+
|
|
31
|
+
```
|
|
32
|
+
1. Polymarket prices → identify markets with volume > $10K
|
|
33
|
+
2. Filter → bookmaker favorites > 65% implied probability
|
|
34
|
+
3. Injury/news check → any material changes not priced in?
|
|
35
|
+
4. Form & H2H analysis → recent performance, matchup history
|
|
36
|
+
5. Cross-reference 3+ bookmaker sources → calculate true probability
|
|
37
|
+
6. Calculate edge → only bet if edge > 10% vs Polymarket price
|
|
38
|
+
7. Size the bet → based on edge magnitude and bankroll
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
### Market Selection Criteria
|
|
42
|
+
|
|
43
|
+
**Good markets:**
|
|
44
|
+
- High volume (> $10K) — ensures liquidity for entry and exit
|
|
45
|
+
- Near-term resolution (days, not months) — capital efficiency
|
|
46
|
+
- Binary outcomes with clear resolution criteria
|
|
47
|
+
- Markets where bookmaker odds exist for cross-referencing
|
|
48
|
+
|
|
49
|
+
**Bad markets:**
|
|
50
|
+
- Low liquidity (< $5K volume) — wide spreads eat your edge
|
|
51
|
+
- Subjective resolution criteria — dispute risk
|
|
52
|
+
- Markets with insider information advantage (crypto governance, company decisions)
|
|
53
|
+
- Long-dated futures that tie up capital for months
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Edge Calculation
|
|
58
|
+
|
|
59
|
+
### The Math
|
|
60
|
+
|
|
61
|
+
```
|
|
62
|
+
Bookmaker Implied Probability = 1 / Decimal Odds
|
|
63
|
+
Edge = (True Probability - Polymarket Price) / Polymarket Price × 100
|
|
64
|
+
|
|
65
|
+
Example:
|
|
66
|
+
Bookmaker odds: -225 (decimal 1.44) → Implied probability: 69.4%
|
|
67
|
+
Polymarket price: $0.645 (64.5%)
|
|
68
|
+
Edge = (69.4 - 64.5) / 64.5 × 100 = 7.6%
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
### American Odds to Probability
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
Negative odds (favorites): Probability = |odds| / (|odds| + 100)
|
|
75
|
+
-225 → 225 / 325 = 69.2%
|
|
76
|
+
|
|
77
|
+
Positive odds (underdogs): Probability = 100 / (odds + 100)
|
|
78
|
+
+150 → 100 / 250 = 40.0%
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Removing the Vig
|
|
82
|
+
|
|
83
|
+
Bookmaker odds include a margin (vig). To get true probabilities:
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
1. Convert both sides to implied probabilities
|
|
87
|
+
2. Sum them (will be > 100%, e.g., 105%)
|
|
88
|
+
3. Divide each by the sum to normalize to 100%
|
|
89
|
+
|
|
90
|
+
Example:
|
|
91
|
+
Team A: -200 → 66.7% Team B: +170 → 37.0%
|
|
92
|
+
Sum: 103.7%
|
|
93
|
+
True A: 66.7 / 103.7 = 64.3%
|
|
94
|
+
True B: 37.0 / 103.7 = 35.7%
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Edge Thresholds
|
|
98
|
+
|
|
99
|
+
| Edge | Action |
|
|
100
|
+
|------|--------|
|
|
101
|
+
| < 5% | Skip — too thin, transaction costs eat it |
|
|
102
|
+
| 5-10% | Marginal — only if very high conviction + multiple sources agree |
|
|
103
|
+
| **> 10%** | **Target zone — place the bet** |
|
|
104
|
+
| > 20% | Strong edge — size up, but verify it's not a trap (news you missed?) |
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## Bookmaker Cross-Referencing
|
|
109
|
+
|
|
110
|
+
### Why Cross-Reference?
|
|
111
|
+
|
|
112
|
+
Polymarket prices are set by traders, not oddsmakers. They systematically:
|
|
113
|
+
- **Overvalue favorites** by 3-7% in major sports markets
|
|
114
|
+
- **Undervalue underdogs/draws** in 3-way football markets
|
|
115
|
+
- **Lag behind** sharp bookmaker lines by hours
|
|
116
|
+
|
|
117
|
+
### Sources to Cross-Reference
|
|
118
|
+
|
|
119
|
+
| Source | Use | Notes |
|
|
120
|
+
|--------|-----|-------|
|
|
121
|
+
| **Pinnacle** | Sharpest lines globally | Gold standard, lowest vig |
|
|
122
|
+
| **Bet365** | Popular, liquid markets | Good for mainstream sports |
|
|
123
|
+
| **DraftKings/FanDuel** | US sports | NFL, NBA, MLB, NHL |
|
|
124
|
+
| **Betfair Exchange** | True market prices | No vig, just commission |
|
|
125
|
+
| **OddsPortal/OddsChecker** | Aggregators | Compare across 20+ books |
|
|
126
|
+
| **Action Network** | Analysis + odds | Good injury/form context |
|
|
127
|
+
|
|
128
|
+
### The 3-Source Rule
|
|
129
|
+
|
|
130
|
+
Never bet based on a single bookmaker. Always confirm with **3+ independent sources**:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
✅ Good: Pinnacle -220, Bet365 -225, DraftKings -215 → consensus ~69%
|
|
134
|
+
Polymarket at 60¢ → 15% edge → BET
|
|
135
|
+
|
|
136
|
+
❌ Bad: Only one bookmaker has odds, others don't list the market
|
|
137
|
+
→ Information asymmetry, you might be wrong
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Risk Management Rules
|
|
143
|
+
|
|
144
|
+
### Bankroll Management
|
|
145
|
+
|
|
146
|
+
```
|
|
147
|
+
Max single bet: 10% of bankroll
|
|
148
|
+
Typical bet size: 2-5% of bankroll
|
|
149
|
+
Max daily exposure: 25% of bankroll
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
### Hard Rules
|
|
153
|
+
|
|
154
|
+
1. **Sport only** — No crypto, politics, or geopolitics bets. Crypto markets are manipulated by insiders.
|
|
155
|
+
2. **Only heavy favorites** — Bookmaker implied probability > 65%
|
|
156
|
+
3. **Edge > 10%** — No exceptions for "gut feelings"
|
|
157
|
+
4. **3+ sources minimum** — Cross-reference before every bet
|
|
158
|
+
5. **No long shots** — Underdogs and parlays are money pits
|
|
159
|
+
6. **The best trade is sometimes no trade** — Don't force action
|
|
160
|
+
|
|
161
|
+
### When NOT to Bet
|
|
162
|
+
|
|
163
|
+
- Market is illiquid (< $5K volume, wide spreads)
|
|
164
|
+
- News is breaking and odds haven't settled
|
|
165
|
+
- You can't find 3 bookmakers listing the event
|
|
166
|
+
- The edge comes from a single outlier source
|
|
167
|
+
- You're chasing losses from a previous bet
|
|
168
|
+
- Resolution criteria are ambiguous
|
|
169
|
+
|
|
170
|
+
### Track Record Requirements
|
|
171
|
+
|
|
172
|
+
- Target **80%+ win rate** on individual bets
|
|
173
|
+
- If below 60% over 10+ bets, stop and re-evaluate strategy
|
|
174
|
+
- Log every bet: market, entry price, bookmaker consensus, edge, result
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## APIs & Data Sources
|
|
179
|
+
|
|
180
|
+
### Gamma API (Public, No Auth)
|
|
181
|
+
|
|
182
|
+
Market discovery and search. Base: `https://gamma-api.polymarket.com`
|
|
183
|
+
|
|
184
|
+
```
|
|
185
|
+
GET /public-search?q=<query> — Search markets/events
|
|
186
|
+
GET /events?active=true&closed=false — List active events
|
|
187
|
+
GET /events?tag_slug=<slug> — Events by category (sports, politics, crypto)
|
|
188
|
+
GET /markets?slug=<slug> — Market details by slug
|
|
189
|
+
GET /tags — All available categories
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
**Key response fields:**
|
|
193
|
+
- `outcomePrices` — Current Yes/No prices (JSON string, parse it)
|
|
194
|
+
- `clobTokenIds` — Token IDs needed for CLOB trading (JSON string)
|
|
195
|
+
- `volume` — Total dollar volume traded
|
|
196
|
+
- `negRisk` — If true, uses negRisk contract (multi-outcome markets)
|
|
197
|
+
- `groupItemTitle` — The outcome name in grouped markets
|
|
198
|
+
|
|
199
|
+
### CLOB API (Public reads, Auth for trading)
|
|
200
|
+
|
|
201
|
+
Order book and trading. Base: `https://clob.polymarket.com`
|
|
202
|
+
|
|
203
|
+
```
|
|
204
|
+
# Public (no auth)
|
|
205
|
+
GET /midpoint?token_id=<id> — Midpoint price
|
|
206
|
+
GET /book?token_id=<id> — Full order book
|
|
207
|
+
GET /spread?token_id=<id> — Bid-ask spread
|
|
208
|
+
GET /price?token_id=<id>&side=buy|sell — Best available price
|
|
209
|
+
GET /tick-size?token_id=<id> — Min price increment
|
|
210
|
+
|
|
211
|
+
# Authenticated (requires L2 API key)
|
|
212
|
+
POST /order — Place order
|
|
213
|
+
DELETE /order/<id> — Cancel order
|
|
214
|
+
GET /orders — Open orders
|
|
215
|
+
GET /balances — CLOB balances
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
### Data API (Public, No Auth)
|
|
219
|
+
|
|
220
|
+
Positions and history. Base: `https://data-api.polymarket.com`
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
GET /positions?user=<wallet_address> — All positions for a wallet
|
|
224
|
+
GET /trades?user=<wallet_address> — Trade history
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
### Authentication Model
|
|
228
|
+
|
|
229
|
+
Two-layer auth system:
|
|
230
|
+
- **L1 (Wallet Signature)**: EIP-712 signature from your Polygon wallet — used to derive API credentials
|
|
231
|
+
- **L2 (API Key)**: HMAC-SHA256 headers for all trading operations
|
|
232
|
+
|
|
233
|
+
**Headers for authenticated requests:**
|
|
234
|
+
```
|
|
235
|
+
POLY_ADDRESS — Wallet address
|
|
236
|
+
POLY_SIGNATURE — HMAC signature of request
|
|
237
|
+
POLY_TIMESTAMP — Unix timestamp
|
|
238
|
+
POLY_NONCE — Request nonce
|
|
239
|
+
POLY_API_KEY — Your API key
|
|
240
|
+
POLY_PASSPHRASE — Your passphrase
|
|
241
|
+
POLY_SECRET — Your API secret (used for HMAC)
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
---
|
|
245
|
+
|
|
246
|
+
## Trading via CLOB
|
|
247
|
+
|
|
248
|
+
### Using the TypeScript SDK
|
|
249
|
+
|
|
250
|
+
```javascript
|
|
251
|
+
const { ClobClient, Side } = require('@polymarket/clob-client');
|
|
252
|
+
const { Wallet } = require('ethers');
|
|
253
|
+
|
|
254
|
+
// Initialize client
|
|
255
|
+
const wallet = new Wallet(PRIVATE_KEY);
|
|
256
|
+
const client = new ClobClient(
|
|
257
|
+
'https://clob.polymarket.com',
|
|
258
|
+
137, // Polygon chainId
|
|
259
|
+
wallet,
|
|
260
|
+
creds // { apiKey, secret, passphrase }
|
|
261
|
+
);
|
|
262
|
+
|
|
263
|
+
// Derive API credentials (first time)
|
|
264
|
+
const creds = await client.createOrDeriveApiKey();
|
|
265
|
+
|
|
266
|
+
// Place a limit buy order
|
|
267
|
+
const order = await client.createAndPostOrder({
|
|
268
|
+
tokenID: '<token_id>', // From Gamma API clobTokenIds
|
|
269
|
+
price: 0.65, // Max price willing to pay
|
|
270
|
+
size: 10, // Dollar amount
|
|
271
|
+
side: Side.BUY,
|
|
272
|
+
}, { tickSize: '0.01' }); // Check tick-size endpoint first
|
|
273
|
+
|
|
274
|
+
// Cancel an order
|
|
275
|
+
await client.cancelOrder(orderId);
|
|
276
|
+
|
|
277
|
+
// Get open orders
|
|
278
|
+
const orders = await client.getOpenOrders();
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
### Order Flow
|
|
282
|
+
|
|
283
|
+
```
|
|
284
|
+
1. Search market on Gamma → get slug
|
|
285
|
+
2. Get market details → extract clobTokenIds and outcomePrices
|
|
286
|
+
3. Identify the outcome you want (Yes token ID vs No token ID)
|
|
287
|
+
4. Check tick-size for that token
|
|
288
|
+
5. Check current best price: GET /price?token_id=X&side=buy
|
|
289
|
+
6. Place limit order at your target price
|
|
290
|
+
7. Monitor: GET /orders to check if filled
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
### Token ID Selection
|
|
294
|
+
|
|
295
|
+
Grouped markets (like "Who wins UFC 326?") have multiple outcomes. Each outcome has a Yes and No token:
|
|
296
|
+
|
|
297
|
+
```
|
|
298
|
+
Market: "UFC 326 Main Event"
|
|
299
|
+
Outcome: "Max Holloway"
|
|
300
|
+
→ Yes Token ID: 7068099725... (buy this if you think Holloway wins)
|
|
301
|
+
→ No Token ID: 1293847561... (buy this if you think Holloway loses)
|
|
302
|
+
Outcome: "Charles Oliveira"
|
|
303
|
+
→ Yes Token ID: 8843920183...
|
|
304
|
+
→ No Token ID: 5567382910...
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
The Gamma API returns `clobTokenIds` as a JSON string with `[NoTokenId, YesTokenId]` — **index 1 is Yes**.
|
|
308
|
+
|
|
309
|
+
---
|
|
310
|
+
|
|
311
|
+
## Position Management & Redemption
|
|
312
|
+
|
|
313
|
+
### Checking Positions
|
|
314
|
+
|
|
315
|
+
```
|
|
316
|
+
GET https://data-api.polymarket.com/positions?user=<wallet_address>
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
Returns all current positions with:
|
|
320
|
+
- `asset` — Token ID
|
|
321
|
+
- `size` — Number of shares
|
|
322
|
+
- `avgPrice` — Average entry price
|
|
323
|
+
- `currentPrice` — Current market price
|
|
324
|
+
- `pnl` — Unrealized P&L
|
|
325
|
+
|
|
326
|
+
### Redeeming Resolved Positions
|
|
327
|
+
|
|
328
|
+
When a market resolves, winning shares are worth $1.00. You need to call the contract to redeem:
|
|
329
|
+
|
|
330
|
+
**Standard markets** (2-outcome, `negRisk: false`):
|
|
331
|
+
```javascript
|
|
332
|
+
// Call ConditionalTokens contract: redeemPositions()
|
|
333
|
+
const CTF_ADDRESS = '0x4D97DCd97eC945f40cF65F87097ACe5EA0476045';
|
|
334
|
+
```
|
|
335
|
+
|
|
336
|
+
**NegRisk markets** (multi-outcome, `negRisk: true`):
|
|
337
|
+
```javascript
|
|
338
|
+
// Call NegRiskAdapter: redeemPositions()
|
|
339
|
+
const NEG_RISK_ADAPTER = '0xd91E80cF2E7be2e162c6513ceD06f1dD0dA35296';
|
|
340
|
+
// Also call NegRiskCTFExchange for conversion
|
|
341
|
+
const NEG_RISK_EXCHANGE = '0xC5d563A36AE78145C45a50134d48A1215220f80a';
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
### Exit Strategies
|
|
345
|
+
|
|
346
|
+
- **Winner**: Hold until resolution → redeem at $1.00
|
|
347
|
+
- **Cut losses**: Sell on the CLOB if the market moves against you
|
|
348
|
+
- **Take profit**: If price moved significantly in your favor before resolution, consider selling early to lock in gains and free capital
|
|
349
|
+
|
|
350
|
+
---
|
|
351
|
+
|
|
352
|
+
## Understanding Polymarket Mechanics
|
|
353
|
+
|
|
354
|
+
### How Prices Work
|
|
355
|
+
|
|
356
|
+
- Prices = probabilities ($0.65 = market says 65% chance of Yes)
|
|
357
|
+
- Markets resolve to $1.00 (correct outcome) or $0.00 (incorrect)
|
|
358
|
+
- Your profit = $1.00 - entry price (per share, if you win)
|
|
359
|
+
- Your loss = entry price (per share, if you lose)
|
|
360
|
+
|
|
361
|
+
### Where Polymarket Misprices
|
|
362
|
+
|
|
363
|
+
| Pattern | Why | How to Exploit |
|
|
364
|
+
|---------|-----|----------------|
|
|
365
|
+
| Favorites overvalued by 3-7% | Retail bias toward "safe" bets | Compare vs sharp bookmaker lines |
|
|
366
|
+
| Underdogs/draws undervalued | People avoid complexity | 3-way football markets (win/draw/lose) |
|
|
367
|
+
| Slow to react to news | Traders aren't 24/7 | Fast reaction to injury reports, lineups |
|
|
368
|
+
| Low-volume markets inefficient | Not enough informed traders | Small edges in niche markets |
|
|
369
|
+
|
|
370
|
+
### Polymarket vs Bookmakers
|
|
371
|
+
|
|
372
|
+
| Feature | Polymarket | Traditional Bookmaker |
|
|
373
|
+
|---------|-----------|----------------------|
|
|
374
|
+
| Vig/margin | 0% (peer-to-peer) | 3-10% |
|
|
375
|
+
| Liquidity | Variable | Guaranteed |
|
|
376
|
+
| Resolution | Smart contract | Bookmaker decides |
|
|
377
|
+
| Settlement | USDC on Polygon | Fiat |
|
|
378
|
+
| Edge | Retail-driven inefficiencies | Sharp lines, hard to beat |
|
|
379
|
+
|
|
380
|
+
### Chain Details
|
|
381
|
+
|
|
382
|
+
- **Chain**: Polygon (MATIC for gas, USDC.e for trading)
|
|
383
|
+
- **USDC.e contract**: `0x2791Bca1f2de4661ED88A30C99A7a9449Aa84174`
|
|
384
|
+
- **Note**: Polymarket uses USDC.e (bridged), NOT native USDC
|
|
385
|
+
- **Geo**: Restricted in some countries (US blocked, most of EU is fine)
|
|
386
|
+
|
|
387
|
+
---
|
|
388
|
+
|
|
389
|
+
## Common Pitfalls
|
|
390
|
+
|
|
391
|
+
### ❌ Mistakes to Avoid
|
|
392
|
+
|
|
393
|
+
1. **Betting on crypto/politics markets** — Insider manipulation is rampant (project teams, whale wallets, political operatives)
|
|
394
|
+
2. **Chasing long shots** — A 5¢ token that could pay $1 sounds amazing; it almost never hits
|
|
395
|
+
3. **Ignoring liquidity** — A "great price" means nothing if you can't exit
|
|
396
|
+
4. **Single-source analysis** — One bookmaker can be wrong; always cross-reference
|
|
397
|
+
5. **Overexposure** — Never have > 25% of bankroll in active bets
|
|
398
|
+
6. **Ignoring the vig** — Bookmaker odds include margin; remove it before comparing
|
|
399
|
+
7. **Trading illiquid markets** — Wide spreads (> 5¢) silently destroy your edge
|
|
400
|
+
8. **Holding long-dated positions** — Capital is locked; shorter resolution = better capital efficiency
|
|
401
|
+
9. **Not tracking results** — Without a log, you can't evaluate if your strategy works
|
|
402
|
+
10. **Emotional trading** — If you just lost, don't immediately place another bet
|
|
403
|
+
|
|
404
|
+
### ✅ Habits of Profitable Traders
|
|
405
|
+
|
|
406
|
+
1. Systematic scan pipeline for every bet (not ad-hoc)
|
|
407
|
+
2. Spreadsheet tracking all bets with entry, target, result, edge
|
|
408
|
+
3. Walk away when there's no edge — most days have no good bets
|
|
409
|
+
4. Focus on 1-2 sports you know deeply rather than spreading thin
|
|
410
|
+
5. Check injury reports, team news, and lineup confirmations before betting
|
|
411
|
+
6. Review win/loss ratio monthly and adjust thresholds if needed
|
|
@@ -0,0 +1,458 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: security-pentester
|
|
3
|
+
description: "Autonomous web application penetration testing — OWASP Top 10 exploitation, white-box source-aware scanning, CI/CD security gates, vulnerability report interpretation, and remediation workflows. Powered by Shannon pentest framework."
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
category: dev
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Security Pentester
|
|
9
|
+
|
|
10
|
+
Autonomous web application penetration testing. Source-aware scanning that only reports vulnerabilities it can prove with a working exploit.
|
|
11
|
+
|
|
12
|
+
## Core Principle
|
|
13
|
+
|
|
14
|
+
**No Exploit, No Report.** Every finding includes a reproducible proof-of-concept. PoC validation significantly reduces false positives, but Critical/High findings should always be manually verified (see section 4 — False Positive Identification).
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## 1. Vulnerability Coverage
|
|
19
|
+
|
|
20
|
+
### OWASP Top 10 Testing Matrix
|
|
21
|
+
|
|
22
|
+
| Category | What Shannon Tests | Techniques |
|
|
23
|
+
|----------|-------------------|------------|
|
|
24
|
+
| **SQL Injection** | Union-based, blind (boolean/time), error-based, second-order | Payload fuzzing, source-guided parameter discovery |
|
|
25
|
+
| **Command Injection** | OS command injection via user input | Backtick, pipe, semicolon, `$()` injection patterns |
|
|
26
|
+
| **XSS** | Reflected, stored, DOM-based | Context-aware payload generation, filter bypass |
|
|
27
|
+
| **SSRF** | Internal network access, cloud metadata | `http://169.254.169.254`, internal service probing |
|
|
28
|
+
| **Broken Authentication** | Credential stuffing, session fixation, JWT attacks | Brute force, token manipulation, 2FA bypass |
|
|
29
|
+
| **Broken Authorization** | IDOR, privilege escalation, role bypass | Horizontal/vertical access control testing |
|
|
30
|
+
|
|
31
|
+
### OWASP Web Security Testing Guide (WSTG) Coverage
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
WSTG-INFO — Information Gathering ✓ Automated
|
|
35
|
+
WSTG-CONF — Configuration Management ✓ Automated
|
|
36
|
+
WSTG-IDNT — Identity Management ✓ Automated
|
|
37
|
+
WSTG-ATHN — Authentication Testing ✓ Automated
|
|
38
|
+
WSTG-ATHZ — Authorization Testing ✓ Automated
|
|
39
|
+
WSTG-SESS — Session Management ✓ Automated
|
|
40
|
+
WSTG-INPV — Input Validation ✓ Automated
|
|
41
|
+
WSTG-ERRH — Error Handling ✓ Automated
|
|
42
|
+
WSTG-CRYP — Cryptography ◐ Partial (TLS config, weak hashing)
|
|
43
|
+
WSTG-BUSN — Business Logic ✗ Pro only
|
|
44
|
+
WSTG-CLNT — Client-Side Testing ✓ Automated (DOM XSS, open redirects)
|
|
45
|
+
WSTG-APIS — API Testing ✓ Automated (REST, limited GraphQL)
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## 2. Running a Pentest
|
|
51
|
+
|
|
52
|
+
### Quick Start
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
# Clone Shannon
|
|
56
|
+
git clone https://github.com/KeygraphHQ/shannon.git
|
|
57
|
+
cd shannon
|
|
58
|
+
|
|
59
|
+
# Set API key (use >> to append if .env already exists)
|
|
60
|
+
echo "ANTHROPIC_API_KEY=your-key-here" >> .env
|
|
61
|
+
|
|
62
|
+
# Run against a target (black-box)
|
|
63
|
+
./shannon start URL=https://target-app.example.com REPO=my-app
|
|
64
|
+
|
|
65
|
+
# Run with source code (white-box — recommended, finds more vulns)
|
|
66
|
+
./shannon start URL=https://target-app.example.com REPO=my-app
|
|
67
|
+
# Place source code in workspaces/my-app/repo/ before running
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Configuration (shannon.yaml)
|
|
71
|
+
|
|
72
|
+
```yaml
|
|
73
|
+
# Authentication config — tell Shannon how to log in
|
|
74
|
+
auth:
|
|
75
|
+
login_url: /login
|
|
76
|
+
credentials:
|
|
77
|
+
- username: testuser@example.com
|
|
78
|
+
password: TestPass123!
|
|
79
|
+
role: user
|
|
80
|
+
- username: admin@example.com
|
|
81
|
+
password: AdminPass456!
|
|
82
|
+
role: admin
|
|
83
|
+
|
|
84
|
+
# Scope rules
|
|
85
|
+
rules:
|
|
86
|
+
avoid:
|
|
87
|
+
- /api/admin/delete-all # Don't hit destructive endpoints
|
|
88
|
+
- /api/billing/* # Skip billing endpoints
|
|
89
|
+
- /logout # Don't log yourself out
|
|
90
|
+
focus:
|
|
91
|
+
- /api/* # Prioritize API endpoints
|
|
92
|
+
- /dashboard/* # Focus on authenticated surfaces
|
|
93
|
+
|
|
94
|
+
# 2FA support (if app uses TOTP)
|
|
95
|
+
totp:
|
|
96
|
+
secret: JBSWY3DPEHPK3PXP # PLACEHOLDER — replace with your test account's actual TOTP secret
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### CLI Commands
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
./shannon start URL=<url> REPO=<name> # Start full pentest
|
|
103
|
+
./shannon start URL=<url> REPO=<name> CONFIG=shannon.yaml # With config
|
|
104
|
+
./shannon workspaces # List all workspaces
|
|
105
|
+
./shannon logs ID=<workflow-id> # Tail live logs
|
|
106
|
+
./shannon query ID=<workflow-id> # Check progress
|
|
107
|
+
./shannon stop # Stop containers (preserves data)
|
|
108
|
+
./shannon stop CLEAN=true # Full cleanup — DELETES all workspace data
|
|
109
|
+
# WARNING: Export reports before CLEAN=true — it removes reports, PoCs, and logs
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## 3. Understanding the Pipeline
|
|
115
|
+
|
|
116
|
+
### 4-Phase Architecture
|
|
117
|
+
|
|
118
|
+
```
|
|
119
|
+
Phase 1: RECONNAISSANCE
|
|
120
|
+
├── Pre-Recon (source code analysis with configured LLM)
|
|
121
|
+
│ └── Outputs: code_analysis_deliverable.md
|
|
122
|
+
└── Recon (attack surface mapping with Playwright + Nmap)
|
|
123
|
+
└── Outputs: recon_deliverable.md
|
|
124
|
+
|
|
125
|
+
Phase 2: VULNERABILITY ANALYSIS (5 parallel agents)
|
|
126
|
+
├── Injection Analysis → injection_analysis.md + exploitation_queue.json
|
|
127
|
+
├── XSS Analysis → xss_analysis.md + exploitation_queue.json
|
|
128
|
+
├── Auth Analysis → auth_analysis.md + exploitation_queue.json
|
|
129
|
+
├── SSRF Analysis → ssrf_analysis.md + exploitation_queue.json
|
|
130
|
+
└── AuthZ Analysis → authz_analysis.md + exploitation_queue.json
|
|
131
|
+
|
|
132
|
+
Phase 3: EXPLOITATION (5 parallel agents, conditional)
|
|
133
|
+
├── Injection Exploit → injection_exploitation_evidence.md
|
|
134
|
+
├── XSS Exploit → xss_exploitation_evidence.md
|
|
135
|
+
├── Auth Exploit → auth_exploitation_evidence.md
|
|
136
|
+
├── SSRF Exploit → ssrf_exploitation_evidence.md
|
|
137
|
+
└── AuthZ Exploit → authz_exploitation_evidence.md
|
|
138
|
+
|
|
139
|
+
Phase 4: REPORTING
|
|
140
|
+
└── comprehensive_security_assessment_report.md
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
### What Each Phase Does
|
|
144
|
+
|
|
145
|
+
**Pre-Recon** reads source code to understand the application architecture, identify entry points, map data flows, and find potential vulnerability patterns before any network interaction.
|
|
146
|
+
|
|
147
|
+
**Recon** maps the live attack surface: crawls the app with a headless browser, enumerates API endpoints, identifies technologies, scans for open ports.
|
|
148
|
+
|
|
149
|
+
**Vulnerability Analysis** agents work in parallel, each specializing in one category. They combine source code knowledge with recon data to hypothesize specific vulnerabilities and create exploitation queues.
|
|
150
|
+
|
|
151
|
+
**Exploitation** agents receive the queues and attempt real attacks using browser automation (Playwright) and HTTP requests. Only proven exploits are included in the final report.
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## 4. Interpreting Reports
|
|
156
|
+
|
|
157
|
+
### Severity Levels
|
|
158
|
+
|
|
159
|
+
| Severity | Definition | Action |
|
|
160
|
+
|----------|-----------|--------|
|
|
161
|
+
| **Critical** | Direct data breach, RCE, full authentication bypass | Fix immediately, consider taking app offline |
|
|
162
|
+
| **High** | Significant data exposure, privilege escalation, stored XSS | Fix within 24-48 hours |
|
|
163
|
+
| **Medium** | Limited data exposure, CSRF, reflected XSS, information disclosure | Fix within 1-2 weeks |
|
|
164
|
+
| **Low** | Minor information leaks, missing headers, verbose errors | Fix in next sprint |
|
|
165
|
+
|
|
166
|
+
### Reading a Finding
|
|
167
|
+
|
|
168
|
+
Each finding in the report includes:
|
|
169
|
+
|
|
170
|
+
```markdown
|
|
171
|
+
## [CRITICAL] SQL Injection in /api/users/search
|
|
172
|
+
|
|
173
|
+
**Endpoint:** GET /api/users/search?q=
|
|
174
|
+
**Parameter:** q
|
|
175
|
+
**Type:** Union-based SQL injection
|
|
176
|
+
|
|
177
|
+
### Proof of Concept
|
|
178
|
+
GET /api/users/search?q=' UNION SELECT username,password,NULL FROM users--
|
|
179
|
+
|
|
180
|
+
### Response Evidence
|
|
181
|
+
HTTP/1.1 200 OK
|
|
182
|
+
[{"username":"admin","password":"$2b$12$...","3":null}]
|
|
183
|
+
|
|
184
|
+
### Source Code Reference
|
|
185
|
+
File: src/routes/users.ts:42
|
|
186
|
+
const results = await db.query(`SELECT * FROM users WHERE name LIKE '%${req.query.q}%'`);
|
|
187
|
+
|
|
188
|
+
### Remediation
|
|
189
|
+
Use parameterized queries:
|
|
190
|
+
const results = await db.query('SELECT * FROM users WHERE name LIKE $1', [`%${req.query.q}%`]);
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
### False Positive Identification
|
|
194
|
+
|
|
195
|
+
Shannon's "no exploit, no report" policy minimizes false positives, but review for:
|
|
196
|
+
|
|
197
|
+
- **Environment-specific**: Exploit only works in test environment (different DB, debug mode)
|
|
198
|
+
- **Already mitigated**: WAF or middleware blocks the attack in production but not staging
|
|
199
|
+
- **Intended behavior**: Feature that looks like a vulnerability (e.g., admin search returns all users by design)
|
|
200
|
+
- **LLM hallucination**: Report claims a vulnerability but the PoC doesn't actually demonstrate impact
|
|
201
|
+
|
|
202
|
+
Always verify the PoC manually for Critical/High findings before filing tickets.
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## 5. CI/CD Integration
|
|
207
|
+
|
|
208
|
+
### Pre-Deploy Security Gate
|
|
209
|
+
|
|
210
|
+
```yaml
|
|
211
|
+
# .github/workflows/security.yml
|
|
212
|
+
name: Security Pentest
|
|
213
|
+
on:
|
|
214
|
+
pull_request:
|
|
215
|
+
branches: [main]
|
|
216
|
+
schedule:
|
|
217
|
+
- cron: '0 2 * * 1' # Weekly Monday 2am
|
|
218
|
+
|
|
219
|
+
jobs:
|
|
220
|
+
pentest:
|
|
221
|
+
runs-on: ubuntu-latest
|
|
222
|
+
steps:
|
|
223
|
+
- uses: actions/checkout@v4
|
|
224
|
+
|
|
225
|
+
- name: Start test application
|
|
226
|
+
run: docker compose -f docker-compose.test.yml up -d
|
|
227
|
+
|
|
228
|
+
- name: Wait for app
|
|
229
|
+
run: |
|
|
230
|
+
for i in $(seq 1 30); do
|
|
231
|
+
curl -s http://localhost:3000/health && break
|
|
232
|
+
sleep 2
|
|
233
|
+
done
|
|
234
|
+
|
|
235
|
+
- name: Run Shannon pentest
|
|
236
|
+
env:
|
|
237
|
+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
|
238
|
+
run: |
|
|
239
|
+
git clone https://github.com/KeygraphHQ/shannon.git /tmp/shannon
|
|
240
|
+
cd /tmp/shannon
|
|
241
|
+
./shannon start URL=http://host.docker.internal:3000 REPO=pr-${{ github.event.pull_request.number }}
|
|
242
|
+
|
|
243
|
+
- name: Check for critical findings
|
|
244
|
+
run: |
|
|
245
|
+
REPORT="/tmp/shannon/workspaces/pr-${{ github.event.pull_request.number }}/comprehensive_security_assessment_report.md"
|
|
246
|
+
if [ ! -f "$REPORT" ]; then
|
|
247
|
+
echo "::error::Security report not found at $REPORT — pentest may have failed. Blocking deploy."
|
|
248
|
+
exit 1
|
|
249
|
+
fi
|
|
250
|
+
# Count severity headings (format: ## [CRITICAL] or ## [HIGH])
|
|
251
|
+
CRITICAL_COUNT=$(grep -c '^\#\#.*\[CRITICAL\]' "$REPORT" || true)
|
|
252
|
+
HIGH_COUNT=$(grep -c '^\#\#.*\[HIGH\]' "$REPORT" || true)
|
|
253
|
+
if [ "$CRITICAL_COUNT" -gt 0 ]; then
|
|
254
|
+
echo "::error::$CRITICAL_COUNT critical vulnerabilities found! Review the security report."
|
|
255
|
+
cat "$REPORT"
|
|
256
|
+
exit 1
|
|
257
|
+
fi
|
|
258
|
+
if [ "$HIGH_COUNT" -gt 0 ]; then
|
|
259
|
+
echo "::warning::$HIGH_COUNT high-severity vulnerabilities found. Manual review required."
|
|
260
|
+
fi
|
|
261
|
+
|
|
262
|
+
- name: Upload report
|
|
263
|
+
if: always()
|
|
264
|
+
uses: actions/upload-artifact@v4
|
|
265
|
+
with:
|
|
266
|
+
name: security-report
|
|
267
|
+
path: /tmp/shannon/workspaces/pr-*/comprehensive_security_assessment_report.md
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
### Integration Patterns
|
|
271
|
+
|
|
272
|
+
| Pattern | When | Cost | Coverage |
|
|
273
|
+
|---------|------|------|----------|
|
|
274
|
+
| **Full pentest on PR** | Every pull request to main | ~$50/run | Complete |
|
|
275
|
+
| **Weekly scheduled** | Cron job on staging | ~$200/month | Complete |
|
|
276
|
+
| **Quick single-category** | Pre-merge for risky changes | ~$10/run | One vuln type |
|
|
277
|
+
| **Pre-release gate** | Before production deploy | ~$50/run | Complete |
|
|
278
|
+
|
|
279
|
+
### Cost Management
|
|
280
|
+
|
|
281
|
+
```
|
|
282
|
+
Estimated costs per run (varies with model selection and pricing):
|
|
283
|
+
- Simple app (5-10 endpoints): ~$15-25
|
|
284
|
+
- Medium app (20-50 endpoints): ~$30-50
|
|
285
|
+
- Complex app (100+ endpoints): ~$50-100
|
|
286
|
+
|
|
287
|
+
Note: Costs are approximate and depend on the configured model. Check your
|
|
288
|
+
API provider's current pricing for accurate estimates.
|
|
289
|
+
|
|
290
|
+
Cost reduction strategies:
|
|
291
|
+
1. Use CONFIG to narrow scope (focus/avoid rules)
|
|
292
|
+
2. Run single-category scans for targeted checks
|
|
293
|
+
3. Use named workspaces to resume interrupted scans
|
|
294
|
+
4. Schedule full scans weekly, quick scans on PRs
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
## 6. Post-Pentest Workflow
|
|
300
|
+
|
|
301
|
+
### Triage → Fix → Verify
|
|
302
|
+
|
|
303
|
+
```
|
|
304
|
+
1. TRIAGE (Day 0)
|
|
305
|
+
├── Read the full report
|
|
306
|
+
├── Verify all Critical/High PoCs manually
|
|
307
|
+
├── Create tickets with severity labels
|
|
308
|
+
├── Assign owners and deadlines
|
|
309
|
+
└── Notify stakeholders for Critical findings
|
|
310
|
+
|
|
311
|
+
2. FIX (Day 1-14, based on severity)
|
|
312
|
+
├── Critical: same day
|
|
313
|
+
├── High: within 48 hours
|
|
314
|
+
├── Medium: within 2 weeks
|
|
315
|
+
└── Low: next sprint
|
|
316
|
+
|
|
317
|
+
3. VERIFY (After fix)
|
|
318
|
+
├── Re-run Shannon against the same workspace
|
|
319
|
+
│ └── ./shannon start URL=<url> REPO=<same-name> WORKSPACE=verify
|
|
320
|
+
├── Completed agents are skipped (resumable)
|
|
321
|
+
├── Confirm the PoC no longer works
|
|
322
|
+
└── Update ticket status
|
|
323
|
+
|
|
324
|
+
4. DOCUMENT
|
|
325
|
+
├── Archive the report
|
|
326
|
+
├── Update security runbook with new patterns
|
|
327
|
+
├── Add regression tests for each finding
|
|
328
|
+
└── Schedule next pentest
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
### Regression Testing
|
|
332
|
+
|
|
333
|
+
For each finding, create a permanent test:
|
|
334
|
+
|
|
335
|
+
```javascript
|
|
336
|
+
// tests/security/sql-injection.test.ts
|
|
337
|
+
describe('SQL Injection regression', () => {
|
|
338
|
+
it('should not be vulnerable to union-based injection in /api/users/search', async () => {
|
|
339
|
+
const res = await request(app)
|
|
340
|
+
.get("/api/users/search")
|
|
341
|
+
.query({ q: "' UNION SELECT username,password,NULL FROM users--" });
|
|
342
|
+
|
|
343
|
+
// Should NOT return other users' data
|
|
344
|
+
expect(res.body).not.toEqual(
|
|
345
|
+
expect.arrayContaining([
|
|
346
|
+
expect.objectContaining({ username: 'admin' })
|
|
347
|
+
])
|
|
348
|
+
);
|
|
349
|
+
});
|
|
350
|
+
|
|
351
|
+
it('should use parameterized queries', async () => {
|
|
352
|
+
const res = await request(app)
|
|
353
|
+
.get("/api/users/search")
|
|
354
|
+
.query({ q: "test" });
|
|
355
|
+
|
|
356
|
+
expect(res.status).toBe(200);
|
|
357
|
+
// Normal search should still work
|
|
358
|
+
});
|
|
359
|
+
});
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
---
|
|
363
|
+
|
|
364
|
+
## 7. What Shannon Doesn't Cover
|
|
365
|
+
|
|
366
|
+
Supplement with manual testing or other tools:
|
|
367
|
+
|
|
368
|
+
| Gap | Alternative |
|
|
369
|
+
|-----|------------|
|
|
370
|
+
| Business logic flaws | Manual review, threat modeling |
|
|
371
|
+
| Mobile app testing | OWASP MAS, Frida, Objection |
|
|
372
|
+
| Infrastructure/cloud | ScoutSuite, Prowler, CloudSploit |
|
|
373
|
+
| Container security | Trivy, Grype, Docker Bench |
|
|
374
|
+
| API rate limiting | Custom load testing (k6, Artillery) |
|
|
375
|
+
| GraphQL deep testing | InQL, graphql-cop |
|
|
376
|
+
| WebSocket testing | OWASP ZAP WebSocket plugin |
|
|
377
|
+
| Dependency vulnerabilities | npm audit, Snyk, Socket.dev |
|
|
378
|
+
| Secrets in source code | TruffleHog, GitLeaks, detect-secrets |
|
|
379
|
+
| CORS misconfiguration | CORScanner, manual review |
|
|
380
|
+
| HTTP request smuggling | smuggler, h2csmuggler |
|
|
381
|
+
| Race conditions / TOCTOU | Turbo Intruder, manual testing |
|
|
382
|
+
| Cache poisoning | Web Cache Deception Scanner |
|
|
383
|
+
| Host header injection | Manual review of password reset flows |
|
|
384
|
+
|
|
385
|
+
### Complementary Tool Stack
|
|
386
|
+
|
|
387
|
+
```bash
|
|
388
|
+
# Run alongside Shannon for full coverage:
|
|
389
|
+
|
|
390
|
+
# Dependency scanning
|
|
391
|
+
npm audit --production
|
|
392
|
+
npx snyk test
|
|
393
|
+
|
|
394
|
+
# Secret detection
|
|
395
|
+
trufflehog git file://. --only-verified
|
|
396
|
+
|
|
397
|
+
# Container scanning
|
|
398
|
+
trivy image myapp:latest
|
|
399
|
+
|
|
400
|
+
# Infrastructure
|
|
401
|
+
prowler aws --severity critical high
|
|
402
|
+
|
|
403
|
+
# API fuzzing
|
|
404
|
+
schemathesis run http://localhost:3000/openapi.json
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
---
|
|
408
|
+
|
|
409
|
+
## 8. Safe Testing Practices
|
|
410
|
+
|
|
411
|
+
### Rules of Engagement
|
|
412
|
+
|
|
413
|
+
```
|
|
414
|
+
DO:
|
|
415
|
+
✓ Only test applications you own or have written authorization to test
|
|
416
|
+
✓ Use staging/test environments, never production
|
|
417
|
+
✓ Create dedicated test accounts with known credentials
|
|
418
|
+
✓ Set scope rules to avoid destructive endpoints
|
|
419
|
+
✓ Review reports before sharing (may contain sensitive data)
|
|
420
|
+
✓ Keep API keys secure (Shannon uses significant API credits)
|
|
421
|
+
|
|
422
|
+
DON'T:
|
|
423
|
+
✗ Point Shannon at production systems
|
|
424
|
+
✗ Test third-party services without explicit written permission
|
|
425
|
+
✗ Share reports containing valid credentials or PII
|
|
426
|
+
✗ Run without scope rules on apps with destructive endpoints
|
|
427
|
+
✗ Ignore the cost — monitor API spend during runs
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
### Test Environment Setup
|
|
431
|
+
|
|
432
|
+
```yaml
|
|
433
|
+
# docker-compose.test.yml — isolated test environment
|
|
434
|
+
services:
|
|
435
|
+
app:
|
|
436
|
+
build: .
|
|
437
|
+
environment:
|
|
438
|
+
- NODE_ENV=test
|
|
439
|
+
- DATABASE_URL=postgres://test:test@db:5432/testdb
|
|
440
|
+
ports:
|
|
441
|
+
- "3000:3000"
|
|
442
|
+
networks:
|
|
443
|
+
- pentest-net
|
|
444
|
+
|
|
445
|
+
db:
|
|
446
|
+
image: postgres:16
|
|
447
|
+
environment:
|
|
448
|
+
- POSTGRES_DB=testdb
|
|
449
|
+
- POSTGRES_USER=test
|
|
450
|
+
- POSTGRES_PASSWORD=test
|
|
451
|
+
networks:
|
|
452
|
+
- pentest-net
|
|
453
|
+
|
|
454
|
+
networks:
|
|
455
|
+
pentest-net:
|
|
456
|
+
driver: bridge
|
|
457
|
+
# Isolated network — no access to host or internet
|
|
458
|
+
```
|
|
@@ -0,0 +1,446 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: security-sentinel
|
|
3
|
+
description: "Autonomous security vigilance — URL/phishing scanning, wallet scam detection, domain threat analysis, email header validation, smart contract risk assessment, and threat intelligence lookups. Teaches agents to proactively protect users from scams, malware, and fraud."
|
|
4
|
+
version: 1.0.0
|
|
5
|
+
category: dev
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Security Sentinel
|
|
9
|
+
|
|
10
|
+
Autonomous threat detection and response. Scan URLs, wallets, domains, emails, and contracts before trusting them.
|
|
11
|
+
|
|
12
|
+
## Decision Framework
|
|
13
|
+
|
|
14
|
+
When an agent encounters untrusted input, classify it and run ALL matching checks in parallel:
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
Classification (applied independently — input may match multiple):
|
|
18
|
+
─────────────────────────────────────────────────────────────────
|
|
19
|
+
Contains URL pattern → URL scan + domain threat check
|
|
20
|
+
Contains wallet address → Wallet reputation + contract scan (if contract)
|
|
21
|
+
Contains email headers → Header analysis + sender domain check
|
|
22
|
+
Contains domain name → WHOIS age + DNS + SSL + typosquatting check
|
|
23
|
+
Contains contract address → Bytecode analysis + honeypot detection
|
|
24
|
+
Contains IP/hash/IOC → Threat intelligence lookup
|
|
25
|
+
|
|
26
|
+
Example: A URL with a wallet address as a query parameter triggers
|
|
27
|
+
BOTH a URL scan AND a wallet reputation check.
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Final severity = highest severity across all matched checks.
|
|
31
|
+
|
|
32
|
+
**Severity responses:**
|
|
33
|
+
- **Clean** → proceed normally
|
|
34
|
+
- **Suspicious** → warn the user, explain why, let them decide
|
|
35
|
+
- **Malicious** → block the action, explain the threat, suggest alternatives
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## 1. URL & Phishing Detection
|
|
40
|
+
|
|
41
|
+
### Scan Before Clicking
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
# VirusTotal URL scan
|
|
45
|
+
vt url "https://example.com" --include=last_analysis_stats,reputation
|
|
46
|
+
|
|
47
|
+
# Google Safe Browsing (via API)
|
|
48
|
+
curl -s "https://safebrowsing.googleapis.com/v4/threatMatches:find?key=$GSB_API_KEY" \
|
|
49
|
+
-d '{
|
|
50
|
+
"threatInfo": {
|
|
51
|
+
"threatTypes": ["MALWARE", "SOCIAL_ENGINEERING", "UNWANTED_SOFTWARE"],
|
|
52
|
+
"platformTypes": ["ANY_PLATFORM"],
|
|
53
|
+
"threatEntryTypes": ["URL"],
|
|
54
|
+
"threatEntries": [{"url": "https://example.com"}]
|
|
55
|
+
}
|
|
56
|
+
}'
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### Phishing Indicators (Heuristic)
|
|
60
|
+
|
|
61
|
+
Check URLs against these red flags:
|
|
62
|
+
|
|
63
|
+
| Indicator | Risk | Example |
|
|
64
|
+
|-----------|------|---------|
|
|
65
|
+
| Homoglyph characters | High | `goog1e.com` (1 instead of l) |
|
|
66
|
+
| Excessive subdomains | Medium | `login.secure.account.example.xyz` |
|
|
67
|
+
| Recently registered (<30 days) | High | WHOIS creation_date check |
|
|
68
|
+
| Free hosting/URL shortener | Medium | `bit.ly`, `000webhostapp.com` |
|
|
69
|
+
| IP address as URL | High | `http://192.168.1.1/login` |
|
|
70
|
+
| Misspelled brand names | High | `paypa1.com`, `arnazon.com` |
|
|
71
|
+
| HTTP (no TLS) for login page | Critical | `http://bank.example.com/login` |
|
|
72
|
+
| Suspicious TLD | Medium | `.xyz`, `.top`, `.buzz`, `.tk` |
|
|
73
|
+
|
|
74
|
+
### Typosquatting Detection
|
|
75
|
+
|
|
76
|
+
```python
|
|
77
|
+
# Levenshtein distance check against known brands
|
|
78
|
+
from difflib import SequenceMatcher
|
|
79
|
+
|
|
80
|
+
KNOWN_BRANDS = [
|
|
81
|
+
"google.com", "facebook.com", "paypal.com", "amazon.com",
|
|
82
|
+
"microsoft.com", "apple.com", "netflix.com", "coinbase.com",
|
|
83
|
+
"binance.com", "metamask.io", "uniswap.org", "opensea.io"
|
|
84
|
+
]
|
|
85
|
+
|
|
86
|
+
def check_typosquat(domain: str, threshold: float = 0.8) -> list:
|
|
87
|
+
alerts = []
|
|
88
|
+
domain_base = domain.split('.')[0].lower()
|
|
89
|
+
for brand in KNOWN_BRANDS:
|
|
90
|
+
brand_base = brand.split('.')[0].lower()
|
|
91
|
+
ratio = SequenceMatcher(None, domain_base, brand_base).ratio()
|
|
92
|
+
if ratio >= threshold and domain != brand:
|
|
93
|
+
alerts.append(f"'{domain}' resembles '{brand}' (similarity: {ratio:.0%})")
|
|
94
|
+
return alerts
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## 2. Wallet & Address Reputation
|
|
100
|
+
|
|
101
|
+
### Before Transacting
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
# Check address against known scam databases
|
|
105
|
+
# ChainAbuse API
|
|
106
|
+
curl -s "https://api.chainabuse.com/v0/addresses/$ADDRESS" \
|
|
107
|
+
-H "Authorization: Bearer $CHAINABUSE_API_KEY"
|
|
108
|
+
|
|
109
|
+
# Etherscan labels (free)
|
|
110
|
+
curl -s "https://api.etherscan.io/api?module=account&action=txlist&address=$ADDRESS&startblock=0&endblock=99999999&page=1&offset=1&apikey=$ETHERSCAN_API_KEY"
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Scam Wallet Red Flags
|
|
114
|
+
|
|
115
|
+
| Signal | Risk Level | What to Check |
|
|
116
|
+
|--------|-----------|---------------|
|
|
117
|
+
| Address reported on ChainAbuse | Critical | Direct scam reports from victims |
|
|
118
|
+
| OFAC/SDN sanctioned address | Critical | US Treasury sanctions list |
|
|
119
|
+
| Tornado Cash interaction | Context-dependent | See mixer assessment below |
|
|
120
|
+
| High-frequency small txs | Medium | Dust attack / address poisoning pattern |
|
|
121
|
+
| Contract with no verified source | Medium | Etherscan/Basescan verification status |
|
|
122
|
+
| Recently created + high value received | High | Potential rug pull collection wallet |
|
|
123
|
+
|
|
124
|
+
### Address Poisoning Detection
|
|
125
|
+
|
|
126
|
+
```
|
|
127
|
+
Attacker creates addresses that look like your recent contacts:
|
|
128
|
+
|
|
129
|
+
Real: 0xAbC1234567890DEF1234567890abcdef12345678
|
|
130
|
+
Fake: 0xAbC12...............different............45678
|
|
131
|
+
^^^^^ same prefix/suffix
|
|
132
|
+
|
|
133
|
+
Defense: Always verify the FULL address, not just first/last characters.
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
### Mixer / Privacy Protocol Assessment
|
|
137
|
+
|
|
138
|
+
Do NOT automatically flag all mixer interactions as suspicious. Apply contextual analysis:
|
|
139
|
+
|
|
140
|
+
```
|
|
141
|
+
HIGH RISK (flag as Suspicious):
|
|
142
|
+
- Direct deposits/withdrawals > $10,000 equivalent
|
|
143
|
+
- Multiple mixer interactions within 24 hours
|
|
144
|
+
- Mixer usage immediately followed by transfers to exchanges
|
|
145
|
+
- Address appears on OFAC SDN list regardless of mixer use
|
|
146
|
+
|
|
147
|
+
LOWER RISK (note but do not flag):
|
|
148
|
+
- Single small-value mixer interaction
|
|
149
|
+
- Interaction via intermediary contract (indirect)
|
|
150
|
+
- Known privacy-preserving DeFi protocols (not mixers)
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
When mixer interaction is detected, include this context:
|
|
154
|
+
"This address has interacted with [protocol]. Privacy tool usage alone
|
|
155
|
+
is not inherently malicious. Risk assessment considers transaction
|
|
156
|
+
patterns, volume, and regulatory context."
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## 3. Smart Contract Risk Assessment
|
|
161
|
+
|
|
162
|
+
### Honeypot Detection
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
# Quick honeypot check (token contracts)
|
|
166
|
+
# A honeypot lets you buy but blocks selling
|
|
167
|
+
|
|
168
|
+
# Check with honeypot.is API
|
|
169
|
+
curl -s "https://api.honeypot.is/v2/IsHoneypot?address=$TOKEN_ADDRESS&chainID=1"
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
### Rug Pull Indicators
|
|
173
|
+
|
|
174
|
+
| Check | How | Red Flag |
|
|
175
|
+
|-------|-----|----------|
|
|
176
|
+
| Ownership | Read `owner()` or `Ownable` | Owner can mint unlimited tokens |
|
|
177
|
+
| Renounced | Check if owner is `0x0` | Not renounced = owner can rug |
|
|
178
|
+
| Liquidity lock | Check LP token holder | LP tokens not locked or short lock |
|
|
179
|
+
| Proxy contract | Check for `delegatecall` patterns | Owner can change logic at will |
|
|
180
|
+
| Hidden mint | Search for `_mint` outside constructor | Can inflate supply post-launch |
|
|
181
|
+
| Transfer restrictions | Check `_transfer` overrides | May block selling |
|
|
182
|
+
| Fee manipulation | Check `setFee`/`setTax` functions | Owner can set 100% sell tax |
|
|
183
|
+
| Blacklist function | Search for `blacklist`/`isBlacklisted` | Owner can freeze your tokens |
|
|
184
|
+
|
|
185
|
+
### Automated Contract Scan Checklist
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
1. Is source code verified on block explorer? → No = HIGH RISK
|
|
189
|
+
2. Is ownership renounced (owner == 0x0)? → No = CHECK FURTHER
|
|
190
|
+
3. Are there mint functions callable by owner? → Yes = HIGH RISK
|
|
191
|
+
4. Are there blacklist/whitelist functions? → Yes = MEDIUM RISK
|
|
192
|
+
5. Is there a max transaction/wallet limit? → Check if owner-adjustable
|
|
193
|
+
6. Are LP tokens locked? For how long? → <30 days = HIGH RISK
|
|
194
|
+
7. Are there pausable functions? → Yes = MEDIUM RISK (could be legitimate)
|
|
195
|
+
8. Does the contract use upgradeable proxy? → Yes = CHECK proxy admin
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## 4. Email Header Analysis
|
|
201
|
+
|
|
202
|
+
### Validate Sender Authenticity
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
# Check SPF record
|
|
206
|
+
dig TXT example.com | grep "v=spf1"
|
|
207
|
+
|
|
208
|
+
# Check DKIM selector
|
|
209
|
+
dig TXT selector._domainkey.example.com
|
|
210
|
+
|
|
211
|
+
# Check DMARC policy
|
|
212
|
+
dig TXT _dmarc.example.com
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Header Red Flags
|
|
216
|
+
|
|
217
|
+
| Header Field | Check | Red Flag |
|
|
218
|
+
|-------------|-------|----------|
|
|
219
|
+
| `Return-Path` | Match with `From` | Different domain = spoofing attempt |
|
|
220
|
+
| `Received` chain | Trace hops | Unexpected mail servers |
|
|
221
|
+
| `Authentication-Results` | SPF/DKIM/DMARC | `fail` or `none` on any |
|
|
222
|
+
| `X-Mailer` | Software used | Bulk mailer or suspicious client |
|
|
223
|
+
| `Reply-To` | Match with `From` | Different address = phishing likely |
|
|
224
|
+
| `Message-ID` domain | Match with sender | Mismatch = forged email |
|
|
225
|
+
|
|
226
|
+
### Interpreting Authentication Results
|
|
227
|
+
|
|
228
|
+
```
|
|
229
|
+
Authentication-Results: mx.google.com;
|
|
230
|
+
dkim=pass header.d=example.com; ← GOOD: signed by claimed domain
|
|
231
|
+
spf=pass (google.com: domain of noreply@example.com designates 1.2.3.4 as permitted sender);
|
|
232
|
+
dmarc=pass (p=REJECT) ← GOOD: strict DMARC policy
|
|
233
|
+
|
|
234
|
+
If ANY of dkim/spf/dmarc = fail → SUSPICIOUS
|
|
235
|
+
If sender domain has no DMARC record → MEDIUM RISK (no spoofing protection)
|
|
236
|
+
If DMARC policy = none → LOW protection (monitoring only, not enforcing)
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
---
|
|
240
|
+
|
|
241
|
+
## 5. Domain Intelligence
|
|
242
|
+
|
|
243
|
+
### WHOIS Age Check
|
|
244
|
+
|
|
245
|
+
```bash
|
|
246
|
+
# Check domain registration age
|
|
247
|
+
whois example.com | grep -i "creation date"
|
|
248
|
+
|
|
249
|
+
# Risk thresholds:
|
|
250
|
+
# < 7 days → CRITICAL (almost certainly malicious for financial/brand domains)
|
|
251
|
+
# < 30 days → HIGH
|
|
252
|
+
# < 90 days → MEDIUM (could be legitimate startup)
|
|
253
|
+
# > 1 year → LOW (domain age alone is not sufficient)
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
### SSL/TLS Assessment
|
|
257
|
+
|
|
258
|
+
```bash
|
|
259
|
+
# Check certificate details
|
|
260
|
+
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -text -noout
|
|
261
|
+
|
|
262
|
+
# Key checks:
|
|
263
|
+
# - Issuer: Let's Encrypt = free (not inherently bad, but scammers use it)
|
|
264
|
+
# - Subject Alternative Names: does it cover expected domains?
|
|
265
|
+
# - Expiry: very short cert rotation could indicate automation abuse
|
|
266
|
+
# - Self-signed: CRITICAL for any production site
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
### DNS Anomalies
|
|
270
|
+
|
|
271
|
+
```bash
|
|
272
|
+
# Check for suspicious DNS patterns
|
|
273
|
+
dig A example.com +short # IP resolution
|
|
274
|
+
dig MX example.com +short # Mail servers
|
|
275
|
+
dig NS example.com +short # Name servers
|
|
276
|
+
dig TXT example.com +short # SPF, verification records
|
|
277
|
+
|
|
278
|
+
# Red flags:
|
|
279
|
+
# - Cloudflare/hosting IP resolving to a brand-impersonating domain
|
|
280
|
+
# - No MX records for a domain claiming to send email
|
|
281
|
+
# - Recently changed NS records (domain hijack indicator)
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
---
|
|
285
|
+
|
|
286
|
+
## 6. Threat Intelligence Lookups
|
|
287
|
+
|
|
288
|
+
### IOC Enrichment
|
|
289
|
+
|
|
290
|
+
```bash
|
|
291
|
+
# AbuseIPDB — check IP reputation
|
|
292
|
+
curl -s "https://api.abuseipdb.com/api/v2/check?ipAddress=1.2.3.4&maxAgeInDays=90" \
|
|
293
|
+
-H "Key: $ABUSEIPDB_API_KEY" \
|
|
294
|
+
-H "Accept: application/json"
|
|
295
|
+
|
|
296
|
+
# PhishTank — check known phishing URLs
|
|
297
|
+
curl -s "https://checkurl.phishtank.com/checkurl/" \
|
|
298
|
+
-d "url=https://suspicious.example.com&format=json&app_key=$PHISHTANK_API_KEY"
|
|
299
|
+
|
|
300
|
+
# OTX AlienVault — threat indicators
|
|
301
|
+
curl -s "https://otx.alienvault.com/api/v1/indicators/domain/example.com/general" \
|
|
302
|
+
-H "X-OTX-API-KEY: $OTX_API_KEY"
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
### Threat Intelligence Decision Matrix
|
|
306
|
+
|
|
307
|
+
```
|
|
308
|
+
Each source has a confidence weight:
|
|
309
|
+
- VirusTotal (multi-engine): weight = engines_flagging / total_engines (0.0–1.0)
|
|
310
|
+
- Google Safe Browsing: weight = 0.9 (high-confidence source)
|
|
311
|
+
- AbuseIPDB: weight = reported_confidence / 100
|
|
312
|
+
- PhishTank (community): weight = 0.6 if verified, 0.3 if unverified
|
|
313
|
+
- OTX AlienVault: weight = 0.5
|
|
314
|
+
|
|
315
|
+
Scoring (sum of weights from all sources):
|
|
316
|
+
- Combined weight = 0 → CLEAN
|
|
317
|
+
- Combined weight < 0.5 → LOW CONFIDENCE (note in output, proceed with caution)
|
|
318
|
+
- Combined weight 0.5–1.49 → SUSPICIOUS (warn user, provide source details)
|
|
319
|
+
- Combined weight >= 1.5 → MALICIOUS (block and explain)
|
|
320
|
+
|
|
321
|
+
IMPORTANT:
|
|
322
|
+
- NEVER dismiss a single source automatically — a VirusTotal result with 30+
|
|
323
|
+
engine flags (weight >= 0.4) is a strong signal on its own
|
|
324
|
+
- New threats often start with only one vendor detecting them
|
|
325
|
+
- Check the specific threat type (phishing vs malware vs adware)
|
|
326
|
+
- Recent reports carry more weight than old ones
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
---
|
|
330
|
+
|
|
331
|
+
## 7. Continuous Monitoring Playbook
|
|
332
|
+
|
|
333
|
+
### Agent-Initiated Security Checks
|
|
334
|
+
|
|
335
|
+
An autonomous security agent should proactively scan at these trigger points:
|
|
336
|
+
|
|
337
|
+
```
|
|
338
|
+
TRIGGER ACTION FREQUENCY
|
|
339
|
+
──────────────────────────────── ────────────────────────────── ──────────
|
|
340
|
+
User shares a URL → url_scan + domain_threat Every time
|
|
341
|
+
User provides wallet address → wallet_check Every time
|
|
342
|
+
New dependency added → npm audit + snyk check On change
|
|
343
|
+
Pre-deployment → header_scan + ssl_audit Per deploy
|
|
344
|
+
Weekly maintenance → full domain posture check Weekly
|
|
345
|
+
Email campaign setup → SPF/DKIM/DMARC validation On setup
|
|
346
|
+
Smart contract interaction → contract_scan + honeypot Every time
|
|
347
|
+
File download from external → VirusTotal file hash check Every time
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
### Incident Response Quick Actions
|
|
351
|
+
|
|
352
|
+
```
|
|
353
|
+
1. PHISHING DETECTED
|
|
354
|
+
→ Block URL in security headers (CSP)
|
|
355
|
+
→ Notify affected users
|
|
356
|
+
→ Report to PhishTank/Google Safe Browsing
|
|
357
|
+
→ Check if credentials were entered → force password reset
|
|
358
|
+
|
|
359
|
+
2. SCAM WALLET DETECTED
|
|
360
|
+
→ Block transaction
|
|
361
|
+
→ Warn user with specific evidence
|
|
362
|
+
→ Report to ChainAbuse
|
|
363
|
+
→ Check transaction history for prior interactions
|
|
364
|
+
|
|
365
|
+
3. COMPROMISED DOMAIN DETECTED
|
|
366
|
+
→ Revoke any API keys associated with domain
|
|
367
|
+
→ Update DNS if you control it
|
|
368
|
+
→ Notify users who may have visited
|
|
369
|
+
→ Check for data exfiltration in logs
|
|
370
|
+
|
|
371
|
+
4. MALICIOUS CONTRACT DETECTED
|
|
372
|
+
→ Revoke token approvals (approve(0))
|
|
373
|
+
→ Warn user with contract analysis
|
|
374
|
+
→ Check for pending transactions to cancel
|
|
375
|
+
→ Report to block explorer
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
---
|
|
379
|
+
|
|
380
|
+
## 8. Result Caching
|
|
381
|
+
|
|
382
|
+
Cache scan results to preserve API quota and avoid redundant checks:
|
|
383
|
+
|
|
384
|
+
| Check Type | Cache TTL | Cache Key |
|
|
385
|
+
|-----------|----------|-----------|
|
|
386
|
+
| URL scan | 1 hour | Normalized URL (strip tracking params) |
|
|
387
|
+
| Domain WHOIS | 24 hours | Domain name |
|
|
388
|
+
| Wallet reputation | 15 minutes | Address (lowercased) |
|
|
389
|
+
| Contract scan | 1 hour | Contract address + chain ID |
|
|
390
|
+
| Threat intel IOC | 30 minutes | IOC value |
|
|
391
|
+
|
|
392
|
+
- Cache is in-memory only (no persistence across sessions)
|
|
393
|
+
- Force refresh available via user request: "rescan [target]"
|
|
394
|
+
- Cache hit returns cached result with age note (e.g., "cached 12 min ago")
|
|
395
|
+
|
|
396
|
+
---
|
|
397
|
+
|
|
398
|
+
## 9. API Quick Reference
|
|
399
|
+
|
|
400
|
+
### Free Tier APIs
|
|
401
|
+
|
|
402
|
+
| Service | Free Limit | Best For | Notes |
|
|
403
|
+
|---------|-----------|----------|-------|
|
|
404
|
+
| VirusTotal | 4/min, 500/day | URL, file, domain, IP scans | |
|
|
405
|
+
| AbuseIPDB | 1,000/day | IP reputation | |
|
|
406
|
+
| PhishTank | Deprecated | Known phishing URL check | API access restricted; use as supplementary source only if legacy key available |
|
|
407
|
+
| OpenPhish | Community feed, updated every 12h | Phishing URL feed | Free, no API key needed — recommended PhishTank replacement |
|
|
408
|
+
| OTX AlienVault | Unlimited | Threat indicators, IOCs | |
|
|
409
|
+
| Google Safe Browsing | 10,000/day | URL safety check | |
|
|
410
|
+
| Etherscan | 5/sec | Contract verification, tx history | |
|
|
411
|
+
| Honeypot.is | Unlimited | Token honeypot detection | |
|
|
412
|
+
| WHOIS (CLI) | ~30-50/min per registrar | Domain age and registrar | Rate varies by TLD server; implement backoff on failures |
|
|
413
|
+
|
|
414
|
+
### Environment Variables
|
|
415
|
+
|
|
416
|
+
```bash
|
|
417
|
+
VT_API_KEY= # VirusTotal
|
|
418
|
+
GSB_API_KEY= # Google Safe Browsing
|
|
419
|
+
ABUSEIPDB_API_KEY= # AbuseIPDB
|
|
420
|
+
PHISHTANK_API_KEY= # PhishTank (deprecated — optional, legacy keys only)
|
|
421
|
+
OTX_API_KEY= # AlienVault OTX
|
|
422
|
+
ETHERSCAN_API_KEY= # Etherscan (or Basescan, etc.)
|
|
423
|
+
CHAINABUSE_API_KEY= # ChainAbuse
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
### Graceful Degradation
|
|
427
|
+
|
|
428
|
+
Not all API keys are required. The agent should adapt based on what's available:
|
|
429
|
+
|
|
430
|
+
```
|
|
431
|
+
Keys configured Capability level Behavior
|
|
432
|
+
─────────────── ─────────────────── ────────────────────────────────────────────
|
|
433
|
+
All keys Full All checks enabled
|
|
434
|
+
4-6 keys Partial Run available checks, warn about gaps
|
|
435
|
+
1-3 keys Degraded Heuristic-heavy mode, warn prominently
|
|
436
|
+
0 keys Heuristic-only Pattern matching only, no external lookups
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
On startup, log which checks are unavailable:
|
|
440
|
+
- Example: "VT_API_KEY not set — URL reputation checks will use heuristics only"
|
|
441
|
+
|
|
442
|
+
On API errors during operation:
|
|
443
|
+
- Timeout (>5s): skip source, note in output, continue with other sources
|
|
444
|
+
- Rate limited (429): queue and retry with exponential backoff, warn user of delay
|
|
445
|
+
- Server error (5xx): skip source, note in output, continue
|
|
446
|
+
- All external sources fail: switch to heuristic mode and warn explicitly
|