@chdlabs/my-shield 0.1.3 → 0.3.0

package/README.md

@@ -1,6 +1,6 @@
 # 🛡️ @chdlabs/my-shield
 
-> High-performance, zero-latency security shield middleware for Express. Built with pure JavaScript probabilistic data structures (Bloom filter + LRU cache), honeypot traps, and a self-improving 404 intelligence engine.
+> High-performance, zero-latency security shield middleware for Express. Built with pure JavaScript probabilistic data structures (Bloom filter + LRU cache), honeypot traps, MiniRocket behavioral biometrics, pluggable threat persistence, and container-native enterprise IP resolution.
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 [![Node: >=20](https://img.shields.io/badge/node-%3E%3D20.0.0-brightgreen.svg)](https://nodejs.org/)
@@ -12,12 +12,16 @@
 
 -  **Sub-0.1ms Clean Pass**: Legitimate requests pass through the L1 Bloom filter in `< 0.1ms`.
 -  **Zero False-Positive Blocks**: Two-tiered L1 caching (Bloom filter + LRU confirmed threat cache) guarantees statistical Bloom false positives are never blocked.
--  **Silent Honeypot Decoys**: Static trap routes (`/.env`, `/wp-login.php`, `/.git/config`, etc.) return a silent `200 OK` decoy response after a brief delay and instantly blacklist scanners.
+-  **Enterprise Client IP Resolution (`IPResolver`)**: Production-tested parsing for Cloudflare (`CF-Connecting-IP`), Akamai (`True-Client-IP`), Nginx (`X-Real-IP`), and multi-hop HAProxy chains (`X-Forwarded-For`) with IPv6-mapped IPv4 normalization and port stripping.
+-  **Docker & Gateway Protection**: Docker bridge gateways (`172.17.0.1`) and loopback addresses (`127.0.0.1`, `::1`) are permanently immune from bans, preventing accidental container isolation.
+-  **Mission-Critical Route Protection**: First-class `shield.strictBotGuard()` to protect `/login`, `/checkout`, and `/register` against credential stuffing, carding (card testing), and automated account spam.
+-  **Dual-Mode Behavioral Biometrics**: Calibrated Ridge decision boundary (`0.50`) supporting non-blocking `'audit'` mode (for searches and queries) alongside strict `'block'` mode (for auth and checkout).
+-  **First-Party Client SDK (`@chdlabs/my-shield/client`)**: Full-featured `ShieldClient` with pointer freshness guards ($t_{\text{idle}} \le 2.5\text{s}$), buffer flushes on consumption, and automatic `fetch`/`Axios` interceptors.
+-  **Pluggable Threat Persistence**: Zero-overhead in-memory adapter, local JSON file persistence (`FileStorageAdapter`), or universal database hooks (`PostgreSQL`, `MySQL`, `Redis`, `Prisma`).
+-  **Built-in Admin OTP / MFA Challenge Gate**: Standalone, dark-mode OTP verification portal (`shield.otpGate()`) with timing-safe HMAC validation and brute-force protection.
+-  **Silent Honeypot Decoys**: Static trap routes (`/.env`, `/wp-login.php`, `/.git/config`, etc.) return a silent `200 OK` decoy response and instantly blacklist scanners.
 -  **404 Anomaly & Intelligence**: Captures every 404 URL probe, bans scanners probing $\ge 5$ unique paths in 60s, and auto-promotes frequent attack targets to active honeypots.
--  **Single-File Deduplicated Persistence**: Preserves learned probe patterns and promoted honeypots across process restarts in an atomic JSON file (`404_intelligence.json`).
--  **Self-Contained Security Dashboard**: Built-in dark-themed telemetry console with real-time Server-Sent Events (SSE), charts, and active block management.
--  **Safe By Default**: Admin dashboard is disabled by default (`admin.enabled: false`) with dedicated RBAC integration to prevent unauthorized telemetry access.
--  **Zero Production Dependencies**: Pure JavaScript built exclusively with Node.js built-ins (`crypto`, `fs`, `path`, `events`).
+-  **Zero Production Dependencies**: Pure JavaScript built exclusively with Node.js built-ins (`crypto`, `fs`, `path`, `events`, `net`).
 
 ---
 
@@ -28,9 +32,14 @@ Incoming Request
        │
        ▼
 ┌───────────────────────────────────────┐
+│ Enterprise IPResolver                 │
+│ Cloudflare / Nginx / HAProxy / Direct │
+│ Gateway Protection (172.17.0.1 Immune)│
+└──────────────────┬────────────────────┘
+                   ▼
+┌───────────────────────────────────────┐
 │ L1 Shield Guard                       │
-│                                       │
-│ 1. IP Whitelist?   ──▶ Pass           │
+│ 1. IP Whitelisted? ──▶ Pass           │
 │ 2. Bloom Filter?   ──▶ False = Pass   │
 │ 3. LRU Cache?      ──▶ Found = Block  │
 └──────────────────┬────────────────────┘
@@ -42,13 +51,20 @@ Incoming Request
 └──────────────────┬────────────────────┘
                    ▼
 ┌───────────────────────────────────────┐
+│ Route-Level Bot Defenses (Optional)   │
+│ • strictBotGuard() on /login & /pay   │
+│ • behavioralGuard({ mode: 'audit' })  │
+│ • costThrottle() on high-cost routes  │
+└──────────────────┬────────────────────┘
+                   ▼
+┌───────────────────────────────────────┐
 │ Application Routes (Your API)         │
 └──────────────────┬────────────────────┘
                    ▼
 ┌───────────────────────────────────────┐
 │ 404 Intelligence Collector            │
 │ ≥ 5 unique 404s/60s? ──▶ Ban IP       │
-│ Dedup snapshot saved to disk          │
+│ Auto-promotes frequent targets        │
 └───────────────────────────────────────┘
 ```
 
@@ -72,63 +88,40 @@ const app = express();
 
 const shield = createShield({
   whitelist: ['127.0.0.1', '::1'],
+  trustProxy: true, // Auto-resolve Cloudflare, Nginx, or HAProxy client IPs
+  protectGateways: true, // Prevents 172.17.0.1 & loopbacks from ever being banned
+  storage: 'file', // Persist bans to ./data/shield/blocked_ips.json across restarts
   admin: {
-    enabled: true // Set to true to activate dashboard & telemetry (Default: false)
-  },
-  bloom: {
-    expectedItems: 100_000,
-    falsePositiveRate: 0.01
-  },
-  lru: {
-    maxSize: 10_000,
-    ttlMs: 5 * 60 * 1000 // 5 minutes
-  },
-  honeypot: {
-    enabled: true,
-    delayMs: 500
-  },
-  intelligence: {
-    anomalyThreshold: 5,     // 5 unique 404s in 60s -> BAN
-    anomalyWindowMs: 60_000,
-    captureAll: true,
-    filePersistence: {
-      enabled: true,
-      directory: './data/shield',
-      filename: '404_intelligence.json',
-      intervalMs: 60 * 60 * 1000, // 1 hour save interval
-      loadOnStartup: true
-    },
-    autoPromote: {
-      enabled: true,
-      minHits: 50,
-      minUniqueIPs: 10
-    }
+    enabled: true // Set to true to activate dashboard (Default: false)
   }
 });
 
+// Hydrate persisted threats into Bloom and LRU on application startup
+await shield.init();
+
 // 1. Mount main defense guard (FIRST middleware after parsing)
 app.use(shield.middleware());
 
 // 2. Mount honeypot trap router (BEFORE application routes)
 app.use(shield.honeypot());
 
-
-// -----------------> ADD IN HERE ALL YOUR APPLICATION ROUTES <----------------//
-app.get('/api/data', (req, res) => res.json({ status: 'ok' }));
-// -----------------> ADD IN HERE ALL YOUR APPLICATION ROUTES <----------------//
-
-
-
+// 3. Application Routes
+app.get('/api/catalog', (req, res) => res.json({ items: [] }));
 
 // 4. Mount 404 intelligence collector (AFTER application routes)
 app.use(shield.notFoundCollector());
 
-// 5. Mount admin dashboard (protect API endpoints with your app's auth & RBAC middleware)
-app.use('/admin/shield', shield.adminRouter({
-  authMiddleware: [authenticateUser, requireAdminRole]
+// 5. Standalone OTP Protected Admin Console
+app.use('/admin/shield', shield.otpGate({
+  adminEmail: 'security@company.com',
+  async sendOtp(email, code) {
+    // Send OTP via your email service (Nodemailer, SendGrid, etc.)
+    console.log(`[OTP] Sent code ${code} to ${email}`);
+  }
 }));
+app.use('/admin/shield', shield.adminRouter());
 
-// Graceful shutdown: flushes 404 intelligence to disk
+// Graceful shutdown: flushes intelligence and threat persistence to disk
 process.on('SIGTERM', async () => {
   await shield.shutdown();
   process.exit(0);
@@ -137,74 +130,217 @@ process.on('SIGTERM', async () => {
 
 ---
 
-## 🔐 Admin Dashboard & Role-Based Access Control (RBAC)
+## 🛡️ Protecting Mission-Critical Routes (`/login`, `/checkout`, `/register`) From Bots
 
-The built-in admin dashboard provides live telemetry, attack velocity, probe intelligence, active threat management, and real-time Server-Sent Events (SSE).
+Automated bots attack high-value mutation endpoints in three distinct ways:
+1. **Credential Stuffing & Brute Force (`/login`)**: Trying thousands of leaked email/password combinations.
+2. **Card Testing / Carding (`/checkout`)**: Testing stolen credit card numbers, which triggers costly payment gateway decline fees and chargebacks.
+3. **Account Farming (`/register`)**: Creating fake accounts for spamming or exploiting promotional credits.
+
+### 1. Backend Route Protection (`shield.strictBotGuard()`)
+
+Mount `shield.strictBotGuard()` directly on your sensitive POST endpoints. It verifies behavioral mouse dynamics, rejects synthetic or automated bot vectors with `403 Forbidden`, and **instantly auto-bans the attacker's IP**:
 
-### 1. Enabling the Dashboard
-By default, the admin dashboard is **disabled** (`admin.enabled: false`). If mounted while disabled, `/admin/shield` safely returns `404 Not Found`. To enable it, explicitly configure:
 ```javascript
-const shield = createShield({
-  admin: {
-    enabled: true // Required to activate dashboard routes
-  }
+import express from 'express';
+import { createShield } from '@chdlabs/my-shield';
+
+const app = express();
+app.use(express.json());
+
+const shield = createShield({ trustProxy: true });
+await shield.init();
+
+// Global ingress guard
+app.use(shield.middleware());
+
+// ── Protect High-Risk Mutations ──
+
+// Protect Login against credential stuffing
+app.post('/api/auth/login', shield.strictBotGuard(), (req, res) => {
+  // Only authentic human users reach this handler!
+  res.json({ success: true, token: 'session_token' });
 });
-```
 
-### 2. Restricting Access to Administrators Only
-Do **not** wrap `shield.adminRouter()` at the top-level route (e.g., `app.use('/admin/shield', requireAdmin, ...)`). Doing so causes browser page visits to fail with an immediate `401 Unauthorized` JSON response because web browsers do not send `Authorization` headers on initial page navigations.
+// Protect User Registration against fake accounts
+app.post('/api/auth/register', shield.strictBotGuard(), (req, res) => {
+  res.json({ success: true, user: req.body.email });
+});
 
-Instead, pass your authentication and authorization middleware into `shield.adminRouter({ authMiddleware })`:
-```javascript
-app.use('/admin/shield', shield.adminRouter({
-  authMiddleware: [authenticateUser, requireAdminRole]
-}));
+// Protect Checkout: Combine Strict Bot Defense with Cost Throttling
+app.post('/api/checkout',
+  shield.strictBotGuard(),
+  shield.costThrottle({ weight: 5, maxCost: 50, windowMs: 60000 }),
+  (req, res) => {
+    res.json({ success: true, orderId: 'ord_123' });
+  }
+);
+
+// ── Protect High-Volume Reads with Audit Mode ──
+// Does not reject searches, but logs kinematics and attaches X-Shield-Verdict header
+app.get('/api/search', shield.behavioralGuard({ mode: 'audit' }), (req, res) => {
+  res.json({ results: [] });
+});
 ```
 
-#### Why this pattern is secure:
-1. **Public Static Delivery**: The browser can freely download the dashboard SPA shell (`index.html`, `dashboard.css`, `dashboard.js`).
-2. **Protected API & Telemetry**: All data-bearing endpoints (`/admin/shield/api/stats`, `/admin/shield/api/blocked`, `/admin/shield/api/stream`, etc.) are strictly guarded by your `authMiddleware`.
-3. **Graceful UI Feedback**: If a non-admin or unauthenticated visitor accesses the page, the dashboard renders an informative warning banner rather than a raw, broken JSON error.
+### 2. Frontend Client Integration (`ShieldClient`)
+
+Install `@chdlabs/my-shield` in your frontend application (or import `./client`).
 
-### 3. Complete Express RBAC Example
+The client captures human pointer micro-movements as the user fills out the form and clicks "Submit" or "Pay Now". When `instrumentFetch()` is enabled, it automatically attaches the `x-shield-telemetry` header to your API requests:
 
 ```javascript
-// Step A: Authentication middleware (verifies JWT / Bearer token)
-function authenticateUser(req, res, next) {
-  const authHeader = req.headers.authorization || (req.query?.token ? `Bearer ${req.query.token}` : null);
-  if (!authHeader || !authHeader.startsWith('Bearer ')) {
-    return res.status(401).json({ error: 'Authentication required. Missing Authorization header.' });
-  }
+// client/index.js (React, Vue, or Vanilla JS)
+import { ShieldClient } from '@chdlabs/my-shield/client';
+
+// 1. Initialize client SDK
+const shieldClient = new ShieldClient({
+  sampleIntervalMs: 10,
+  windowLength: 128,
+  minPoints: 15,          // Requires at least 15 raw events to prevent flatlines
+  maxIdleMs: 2500,        // Freshness Guard: Discards trajectory if pointer idle > 2.5s
+  flushOnConsumption: true, // Prevents replaying identical trajectories
+  autoInjectTripwire: true // Injects invisible honeypot decoy trap into DOM
+});
 
-  const token = authHeader.split(' ')[1];
+// 2. Start recording kinematics
+shieldClient.start();
+
+// 3. Transparently instrument fetch (or Axios)
+shieldClient.instrumentFetch({
+  headerName: 'x-shield-telemetry',
+  endpoints: ['/api/auth/*', '/api/checkout*']
+});
+
+// Now standard fetch calls automatically include kinematics telemetry!
+async function submitLogin(email, password) {
   try {
-    const claims = verifyJwt(token); // Verify your JWT / PQC token
-    req.user = claims;
-    next();
+    const res = await fetch('/api/auth/login', {
+      method: 'POST',
+      headers: { 'Content-Type': 'application/json' },
+      body: JSON.stringify({ email, password })
+    });
+
+    if (res.status === 403) {
+      const err = await res.json();
+      if (err.shieldCode === 'AUTOMATED_KINEMATICS_DETECTED') {
+        alert('Bot behavior detected. Please interact with the page normally.');
+        return;
+      }
+    }
+
+    const data = await res.json();
+    console.log('Login successful:', data);
   } catch (err) {
-    return res.status(401).json({ error: 'Invalid or expired session token.' });
+    console.error('Request failed:', err);
   }
 }
+```
+
+---
+
+## 🌐 Enterprise Reverse Proxies & Gateway Protection
+
+When deploying in Docker, Kubernetes, AWS, or behind Cloudflare, standard `req.ip` lookups fail or ban internal container gateways. `IPResolver` resolves this natively:
+
+```javascript
+const shield = createShield({
+  // Trust proxy headers: true | false | 'loopback,linklocal,uniquelocal'
+  trustProxy: true,
+
+  // Permanently protects Docker bridge (172.17.0.1) & loopbacks from bans
+  protectGateways: true,
+
+  // Optional custom subnets to shield from accidental bans
+  protectedSubnets: ['10.0.0.1', '192.168.1.1']
+});
+```
+
+### Supported Proxy Headers (in priority order):
+1. `cf-connecting-ip` (Cloudflare)
+2. `true-client-ip` (Akamai & Cloudflare Enterprise)
+3. `x-real-ip` (Nginx `proxy_set_header X-Real-IP $remote_addr`)
+4. `x-client-ip`
+5. `x-forwarded-for` (Multi-hop chains are parsed to find the first non-internal public IP)
+
+If `trustProxy: false` is set (direct internet exposure without reverse proxy), all proxy headers are ignored to prevent header spoofing injection attacks.
+
+---
+
+## 💾 Pluggable Threat Persistence
 
-// Step B: Role-Based Access Control (RBAC) middleware
-function requireAdminRole(req, res, next) {
-  // Check user role from token claims or database lookup
-  if (!req.user || (req.user.role !== 'admin' && req.user.role !== 'super_admin')) {
-    return res.status(403).json({ error: 'Access denied: Administrator privileges required.' });
+In containerized or serverless deployments, in-memory caches reset on every restart. Shield supports pluggable persistence adapters to maintain active threat bans:
+
+### 1. Single-Container File Persistence (Zero External Dependencies)
+```javascript
+const shield = createShield({
+  storage: {
+    type: 'file',
+    directory: './data/shield',
+    filename: 'blocked_ips.json',
+    debounceMs: 500 // Debounced atomic writes
   }
-  next();
-}
+});
+await shield.init();
+```
 
-// Step C: Mount protected admin router
-app.use('/admin/shield', shield.adminRouter({
-  authMiddleware: [authenticateUser, requireAdminRole]
-}));
+### 2. Database & Cache Persistence (PostgreSQL, Redis, MongoDB, Prisma)
+Plug any database into Shield with three async callbacks:
+
+```javascript
+const shield = createShield({
+  storage: {
+    async onLoad() {
+      // Return active unexpired blocked IPs on boot
+      const { rows } = await db.query(
+        'SELECT ip, reason, metadata, expires_at AS "expiresAt" FROM blocked_threats WHERE expires_at IS NULL OR expires_at > NOW()'
+      );
+      return rows;
+    },
+    async onBlock(ip, reason, metadata, expiresAt) {
+      // Save or update blocked threat
+      await db.query(
+        `INSERT INTO blocked_threats (ip, reason, metadata, expires_at)
+         VALUES ($1, $2, $3, $4)
+         ON CONFLICT (ip) DO UPDATE SET reason = $2, metadata = $3, expires_at = $4`,
+        [ip, reason, JSON.stringify(metadata), expiresAt ? new Date(expiresAt) : null]
+      );
+    },
+    async onUnblock(ip) {
+      await db.query('DELETE FROM blocked_threats WHERE ip = $1', [ip]);
+    }
+  }
+});
+
+// Restores unexpired database threats into Bloom filter and LRU cache
+await shield.init();
 ```
 
-### 4. How the Dashboard Authenticates
-The dashboard client automatically looks for the session token in the following order:
-1. **URL Query Parameter**: Navigating to `http://your-server.com/admin/shield?token=<jwt>` extracts the token into `sessionStorage` and automatically scrubs the parameter from the browser's address bar to prevent token leakage.
-2. **Storage Fallback**: Checks `sessionStorage` and `localStorage` under standard keys: `shield_session_token`, `pqc_session_token`, `token`, `access_token`, `auth_token`, or `jwt`.
+---
+
+## 🔑 Built-in Admin OTP / MFA Challenge Gate
+
+Shield includes a standalone, zero-dependency dark-mode OTP challenge gate (`shield.otpGate()`) that protects `/admin/shield` without requiring external login libraries:
+
+```javascript
+app.use('/admin/shield', shield.otpGate({
+  adminEmail: 'admin@company.com',
+  jwtSecret: process.env.SHIELD_SECRET, // Signs session cookies
+  codeTtlMs: 5 * 60 * 1000,             // Code valid for 5 minutes
+  maxAttempts: 5,                        // Locks out after 5 invalid tries
+  async sendOtp(email, code) {
+    // Deliver via email or SMS
+    await mailer.sendMail({
+      to: email,
+      subject: 'Shield Admin Passcode',
+      text: `Your one-time passcode is: ${code}`
+    });
+  }
+}));
+
+// Mount dashboard router
+app.use('/admin/shield', shield.adminRouter());
+```
 
 ---
 
@@ -213,25 +349,18 @@ The dashboard client automatically looks for the session token in the following
 | Option | Type | Default | Description |
 |---|---|---|---|
 | `whitelist` | `string[]` | `[]` | List of IP addresses that are never blocked |
-| `admin.enabled` | `boolean` | `false` | Explicit switch to enable the admin dashboard & telemetry |
-| `bloom.expectedItems` | `number` | `100000` | Estimated distinct IPs to hold in filter |
+| `trustProxy` | `boolean\|string` | `'loopback,linklocal,uniquelocal'` | Reverse proxy trust configuration |
+| `protectGateways` | `boolean` | `true` | Protects Docker bridge (`172.17.0.1`) and loopback from bans |
+| `storage` | `string\|Object` | `'memory'` | Storage adapter (`'memory'`, `'file'`, or callback hooks) |
+| `admin.enabled` | `boolean` | `false` | Explicit switch to enable admin dashboard & telemetry |
+| `bloom.expectedItems` | `number` | `100000` | Estimated distinct IPs to hold in Bloom filter |
 | `bloom.falsePositiveRate` | `number` | `0.01` | Target false positive probability (1%) |
-| `bloom.sizeBytes` | `number` | `null` | Explicit bit array size in bytes |
 | `lru.maxSize` | `number` | `10000` | Max entries in confirmed threat cache |
 | `lru.ttlMs` | `number` | `300000` | Threat cache TTL in ms (5 minutes) |
-| `honeypot.enabled` | `boolean` | `true` | Enable/disable honeypot traps |
+| `honeypot.enabled` | `boolean` | `true` | Enable/disable static honeypot traps |
 | `honeypot.delayMs` | `number` | `500` | Delay before sending silent 200 OK decoy |
-| `honeypot.additionalRoutes` | `string[]` | `[]` | Custom trap path patterns |
 | `intelligence.anomalyThreshold` | `number` | `5` | Unique 404s before IP is banned |
 | `intelligence.anomalyWindowMs` | `number` | `60000` | Sliding window in ms (60 seconds) |
-| `intelligence.filePersistence.enabled` | `boolean` | `true` | Enable periodic JSON snapshot saves |
-| `intelligence.filePersistence.directory` | `string` | `'./data/shield'` | Destination directory for JSON snapshot |
-| `intelligence.filePersistence.filename` | `string` | `'404_intelligence.json'` | Single deduplicated filename |
-| `intelligence.filePersistence.intervalMs`| `number` | `3600000` | Snapshot save interval (1 hour) |
-| `intelligence.filePersistence.loadOnStartup` | `boolean` | `true` | Restore intelligence on process start |
-| `intelligence.autoPromote.enabled` | `boolean` | `true` | Auto-promote frequent 404s to traps |
-| `intelligence.autoPromote.minHits` | `number` | `50` | Min 404 hits before promotion |
-| `intelligence.autoPromote.minUniqueIPs` | `number` | `10` | Min distinct IPs probing path |
 
 ---
 
@@ -242,7 +371,6 @@ The dashboard client automatically looks for the session token in the following
 shield.blockIP('198.51.100.42', 'manual_ban', { admin: 'dev1' });
 shield.unblockIP('198.51.100.42');
 shield.allowIP('192.168.1.50');       // Add to whitelist
-shield.removeAllowedIP('192.168.1.50');
 shield.isWhitelisted('192.168.1.50'); // Returns boolean
 
 // Telemetry & Metrics
@@ -250,10 +378,14 @@ shield.getStats();          // Returns aggregate stats, LRU sizes, traps list
 shield.getIntelligence();   // Full 404 intelligence snapshot
 shield.getBloomHealth();    // Saturation percentage and estimated FPR
 
-// Route-Level Denial-of-Wallet Cost Throttling
+// Route-Level Bot Defenses
+app.post('/api/auth/login', shield.strictBotGuard(), handler);
+app.get('/api/search', shield.behavioralGuard({ mode: 'audit' }), handler);
+
+// Denial-of-Wallet Cost Throttling
 app.post('/api/ai-generate', shield.costThrottle({ weight: 10, maxCost: 100 }), handler);
 
-// Route-Level Token / Credential Leak Detection
+// API Key / Secret Leak Detector
 app.use(shield.keyLeakDetector());
 ```