@sandrobuilds/tracerney 0.9.6

package/README.md

@@ -0,0 +1,702 @@
+# Tracerney
+
+**Transparent Proxy Runtime Sentinel** — Enterprise-Grade Prompt Injection Defense for LLM Applications
+
+[![npm version](https://badge.fury.io/js/tracerney.svg)](https://www.npmjs.com/package/tracerney)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+Tracerney is a production-ready security middleware that sits between your application and LLM providers. It detects and blocks prompt injection attacks with **three hardened layers**:
+
+1. **Layer 1 (Vanguard)**: Regex patterns with Unicode normalization — <2ms, blocks known attacks
+2. **Layer 2 (Sentinel)**: Backend LLM verification for novel attacks, with rate limiting to prevent cost spikes
+3. **Layer 3 (Jitter)**: Random response delays to mask which layer blocked an attack
+
+**Zero data leaves your infrastructure** — you control the backend endpoints.
+
+## 🚀 Quick Start
+
+**Install:**
+```bash
+npm install @sandrobuilds/tracerney
+```
+
+**Setup (30 seconds) — RECOMMENDED approach:**
+```typescript
+import { Tracerney, ShieldBlockError } from '@sandrobuilds/tracerney';
+
+// Just provide your domain URL — paths are auto-constructed!
+const shield = new Tracerney({
+  baseUrl: 'http://localhost:3000',        // Automatically creates all endpoints
+  allowedTools: ['search', 'calculator'],
+  apiKey: process.env.TRACERNY_API_KEY,
+});
+
+// Wrap your LLM call (detects at Layer 1 + Layer 2)
+try {
+  const response = await shield.wrap(() =>
+    openai.chat.completions.create({
+      model: 'gpt-4',
+      messages: [{ role: 'user', content: userInput }],
+    })
+  );
+  console.log(response);
+} catch (error) {
+  if (error instanceof ShieldBlockError) {
+    console.error('🛡️ Attack blocked:', error.event.blockReason);
+    // Layer 1 hit: error.event.metadata.patternName
+    // Layer 2 hit: "LLM Sentinel" detected novel attack
+  }
+}
+```
+
+That's it! Your LLM is now protected.
+
+## Philosophy
+
+We are not a "platform" where you send data to our servers first. We are a **Runtime Sentinel** that lives inside your process:
+
+- **🚀 No latency friction**: Detection happens locally in <2ms
+- **🔒 No privacy concerns**: Your prompts never leave your infrastructure
+- **📦 3 lines of code**: Wrap your existing LLM call
+- **👨‍💻 Developer-first**: Fast integration, zero configuration needed
+
+---
+
+## Common Setups
+
+### Next.js (API Route)
+
+```typescript
+// app/api/chat/route.ts
+import { Tracerney, ShieldBlockError } from '@sandrobuilds/tracerney';
+import OpenAI from 'openai';
+
+const openai = new OpenAI();
+
+export async function POST(req: Request) {
+  const { userMessage } = await req.json();
+
+  try {
+    const response = await shield.wrap(
+      () => openai.chat.completions.create({
+        model: 'gpt-4',
+        messages: [{ role: 'user', content: userMessage }],
+      }),
+      { prompt: userMessage } // Pre-scan before LLM
+    );
+
+    return Response.json({ success: true, data: response });
+  } catch (error) {
+    if (error instanceof ShieldBlockError) {
+      return Response.json(
+        { error: 'Request blocked for security' },
+        { status: 403 }
+      );
+    }
+    throw error;
+  }
+}
+```
+
+### Node.js / Express
+
+```typescript
+import { Tracerney, ShieldBlockError } from '@sandrobuilds/tracerney';
+import OpenAI from 'openai';
+
+const shield = new Tracerney({ allowedTools: ['search'] });
+const openai = new OpenAI();
+
+app.post('/chat', async (req, res) => {
+  try {
+    const response = await shield.wrap(() =>
+      openai.chat.completions.create({
+        model: 'gpt-4',
+        messages: [{ role: 'user', content: req.body.message }],
+      })
+    );
+    res.json(response);
+  } catch (error) {
+    if (error instanceof ShieldBlockError) {
+      return res.status(403).json({ error: 'Blocked' });
+    }
+    throw error;
+  }
+});
+```
+
+### Minimal Setup (No Telemetry)
+
+```typescript
+// Just blocking, no monitoring
+const shield = new Tracerney({
+  allowedTools: ['search'],
+  // No apiEndpoint = no telemetry
+});
+
+await shield.wrap(() => llmCall());
+```
+
+### With Monitoring Dashboard
+
+```typescript
+// Using baseUrl — all endpoints auto-configured
+const shield = new Tracerney({
+  baseUrl: 'http://localhost:3000',
+  allowedTools: ['search', 'calculator'],
+  apiKey: process.env.TRACERNY_API_KEY,
+  enableTelemetry: true, // Events + patterns auto-configured
+});
+```
+
+---
+
+## Architecture
+
+### The Request-Response Lifecycle
+
+```
+Request → Vanguard (Regex Check) → LLM Provider
+                                        ↓
+         ← Tool-Guard (Schema Check) ← Response
+         ↓
+    App Logic (if clean) / Block (if dirty)
+         ↓
+    Async Signal (Non-blocking Telemetry)
+```
+
+## Three-Layer Defense Architecture
+
+### Layer 1: Vanguard (Pattern Matching)
+**Speed**: <2ms | **Coverage**: Known attacks
+
+Fast regex-based detection with Unicode normalization to prevent homoglyph evasion.
+
+**Improved patterns detect:**
+- `ignore [your|my] instructions` (not just "previous")
+- `forget|disregard [all] [rules|guidelines]`
+- `reveal|show system prompt`
+- `act as unrestricted AI` (jailbreak)
+- SQL/code injection
+- Token smuggling
+- **20+ OWASP-mapped patterns**
+
+**Example: Homoglyph evasion blocked**
+```
+Input:    "ｉｇｎｏｒｅ your instructions"  (fullwidth)
+Normalized: "ignore your instructions"
+Result:   ✅ BLOCKED by Pattern_001
+```
+
+### Layer 2: Sentinel (LLM Verification)
+**Speed**: 200-1500ms | **Coverage**: Novel attacks
+
+Backend-side LLM classifier runs only if Layer 1 misses. Uses OpenRouter Gemini with:
+- **Rate limiting**: 5 calls/min per API key (prevents botnet cost spikes)
+- **Cost protection**: Only hits for suspicious prompts
+- **Fallback**: Non-blocking—if backend unavailable, execution continues
+
+**Example: Novel attack caught**
+```
+Input:    "explain social engineering vectors for credential theft"
+Layer 1:  ❌ No regex match
+Layer 2:  ✅ LLM classifier: "YES, this is an injection"
+Result:   BLOCKED + logged to database
+```
+
+### Layer 3: Hardened Middleware
+Automatic protections applied to all requests:
+
+**Normalization**: Removes Unicode tricks before pattern matching
+**Jitter**: Random 300-500ms delay masks which layer blocked the attack
+**Rate Limiting**: Backend enforces 5 verifications/min per key
+
+```typescript
+// All automatic—no config needed
+// Attacks from any layer see ~300-500ms latency
+await shield.scanPrompt(userInput); // Returns in 300-500ms
+// (attacker can't tell if Layer 1 or Layer 2 executed)
+```
+
+```typescript
+// Optionally scan raw prompts before LLM call
+try {
+  shield.scanPrompt(userInput);
+  // Safe to call LLM
+} catch (err) {
+  if (err instanceof ShieldBlockError) {
+    console.error('Blocked:', err.event);
+  }
+}
+```
+
+### 3. Signal Sink (Telemetry)
+
+Asynchronous, non-blocking event reporting.
+
+When a block occurs:
+1. Event is queued in-memory
+2. Execution continues (no latency penalty)
+3. Events are batched and sent to your API in the background
+4. Uses `process.nextTick()` for non-blocking dispatch
+
+**No data sent to us**. You own your Signal endpoint (`/api/v1/signal`).
+
+### 4. Manifest Sync (Definition Updates)
+
+Your patterns don't require an npm update to change.
+
+- **Static Manifest**: Shipped with the SDK
+- **Polling**: On instantiation, checks for new version
+- **Stale-While-Revalidate**: Serves cached version while fetching new one in background
+- **Zero-Day Patches**: Deploy new patterns instantly without forcing users to update npm
+
+## Usage
+
+### Basic Setup
+
+```typescript
+import { Tracerney } from '@sandrobuilds/tracerney';
+
+const shield = new Tracerney({
+  baseUrl: process.env.TRACERNY_BACKEND_URL, // e.g., https://myapp.com
+  allowedTools: ['search', 'calculator'],
+  apiKey: process.env.TRACERNY_API_KEY,
+  enableTelemetry: true,
+});
+
+// That's it. Wrap your LLM calls.
+const response = await shield.wrap(() =>
+  openai.chat.completions.create({
+    model: 'gpt-4',
+    messages: [{ role: 'user', content: userInput }],
+    tools: [
+      {
+        type: 'function',
+        function: {
+          name: 'search',
+          description: 'Search the web',
+        },
+      },
+    ],
+  })
+);
+```
+
+### Error Handling
+
+```typescript
+import { Tracerney, ShieldBlockError } from '@sandrobuilds/tracerney';
+
+try {
+  const response = await shield.wrap(() => llmCall());
+} catch (err) {
+  if (err instanceof ShieldBlockError) {
+    // Security block - log and respond to user
+    console.error('Attack blocked:', err.event.blockReason);
+    res.status(403).json({ error: 'Request blocked' });
+  } else {
+    // Actual error from LLM or network
+    throw err;
+  }
+}
+```
+
+### Scanning Prompts Pre-LLM
+
+For defense-in-depth, scan prompts before calling the LLM:
+
+```typescript
+try {
+  shield.scanPrompt(userInput);
+  // Safe to proceed
+} catch (err) {
+  if (err instanceof ShieldBlockError) {
+    console.error('Blocked at edge:', err.event);
+  }
+}
+```
+
+### Updating Allowed Tools
+
+```typescript
+shield.setAllowedTools(['search', 'email', 'calendar']);
+```
+
+### Getting Shield Status
+
+```typescript
+const status = shield.getStatus();
+console.log(status);
+// {
+//   patternMatcher: { ready: true, stats: {...} },
+//   toolGuard: { allowedTools: [...] },
+//   telemetry: { enabled: true, status: {...} }
+// }
+```
+
+### Advanced: Using Utilities Directly
+
+For custom middleware or testing:
+
+```typescript
+import { normalizePrompt, jitter } from '@sandrobuilds/tracerney';
+
+// Normalize prompts (removes Unicode tricks, collapses whitespace)
+const clean = normalizePrompt('ｉｇｎｏｒｅ your rules');
+// → "ignore your rules"
+
+// Add latency jitter (300-500ms default)
+await jitter();           // Waits random 300-500ms
+await jitter(100, 200);   // Custom range: 100-200ms
+```
+
+## Configuration
+
+### TracernyOptions
+
+```typescript
+interface TracernyOptions {
+  // === RECOMMENDED: Single Domain URL ===
+  // Automatically constructs all backend endpoints:
+  // - {baseUrl}/api/v1/signal (events)
+  // - {baseUrl}/api/v1/verify-prompt (Layer 2 verification)
+  // - {baseUrl}/api/v1/shadow-log (potential attacks)
+  // - {baseUrl}/api/v1/definitions (pattern updates)
+  baseUrl?: string;
+
+  // === LAYER 1: Vanguard ===
+  // List of tool names the LLM is allowed to call
+  allowedTools?: string[];
+
+  // === Authentication ===
+  // API key for authentication to backend endpoints
+  apiKey?: string;
+
+  // === LAYER 2: Sentinel (Advanced - use baseUrl instead) ===
+  // Only needed if NOT using baseUrl
+  sentinelEndpoint?: string;
+  sentinelEnabled?: boolean;
+
+  // === LAYER 3: Telemetry & Logging (Advanced - use baseUrl instead) ===
+  // Only needed if NOT using baseUrl
+  apiEndpoint?: string;
+  shadowLogEndpoint?: string;
+  manifestUrl?: string;
+
+  // Enable telemetry (default: true)
+  enableTelemetry?: boolean;
+
+  // Path to local manifest cache (for serverless, use /tmp)
+  localManifestPath?: string;
+}
+```
+
+**Minimal setup (Layer 1 only):**
+```typescript
+const shield = new Tracerney({ allowedTools: ['search'] });
+```
+
+**Recommended setup with backend (all 3 layers):**
+```typescript
+const shield = new Tracerney({
+  baseUrl: 'http://localhost:3000',  // Single domain — paths auto-constructed
+  allowedTools: ['search', 'calculator'],
+  apiKey: process.env.TRACERNY_API_KEY,
+  enableTelemetry: true,
+});
+```
+
+**Advanced setup (individual endpoints):**
+```typescript
+const shield = new Tracerney({
+  allowedTools: ['search', 'calculator'],
+  sentinelEndpoint: 'http://localhost:3000/api/v1/verify-prompt',
+  shadowLogEndpoint: 'http://localhost:3000/api/v1/shadow-log',
+  apiEndpoint: 'http://localhost:3000/api/v1/signal',
+  apiKey: process.env.TRACERNY_API_KEY,
+  enableTelemetry: true,
+});
+```
+
+## Events
+
+Security events have this structure:
+
+```typescript
+interface SecurityEvent {
+  type: 'INJECTION_DETECTED' | 'UNAUTHORIZED_TOOL' | 'BLOCKED_PATTERN';
+  severity: 'low' | 'medium' | 'high' | 'critical';
+  timestamp: number; // Unix ms
+  blockReason: string;
+  metadata: {
+    toolName?: string;
+    patternName?: string;
+    requestSnippet?: string; // First 100 chars, anonymized
+    blockLatencyMs?: number;
+  };
+  anonymized: boolean;
+}
+```
+
+## Backend Endpoints
+
+Your backend should implement these 3 endpoints for full 3-layer defense:
+
+### 1. POST /api/v1/verify-prompt (Layer 2: LLM Sentinel)
+
+**Called by**: SDK when Layer 1 misses
+**Purpose**: Backend LLM verification with rate limiting
+
+```typescript
+// Input: { prompt, keywords?, requestId? }
+// Output: { blocked, confidence, model, latencyMs, remaining }
+// Status 429: Rate limit exceeded
+```
+
+Example (Next.js):
+```typescript
+export async function POST(req: NextRequest) {
+  const { prompt, keywords, requestId } = await req.json();
+
+  // Verify API key
+  const apiKey = req.headers.get('Authorization')?.replace('Bearer ', '');
+  const user = await validateApiKey(apiKey); // Your implementation
+
+  // Check rate limit (5 calls/min per API key)
+  const limiter = getRateLimiter();
+  const { allowed, remaining } = limiter.check(user.id, 5, 60000);
+  if (!allowed) {
+    return NextResponse.json(
+      { blocked: true, reason: 'rate_limit', remaining: 0 },
+      { status: 429 }
+    );
+  }
+
+  // Call OpenRouter LLM with your API key (kept secret on backend)
+  const llmVerdict = await callOpenRouter(prompt, process.env.OPENROUTER_API_KEY);
+
+  return NextResponse.json({
+    blocked: llmVerdict,
+    confidence: llmVerdict ? 0.95 : 0.15,
+    model: 'google/gemini-2.5-flash-lite',
+    latencyMs: Date.now() - startTime,
+    remaining,
+  });
+}
+```
+
+### 2. POST /api/v1/shadow-log (Layer 3: Potential Attacks)
+
+**Called by**: SDK for prompts that pass both Layer 1 + Layer 2
+**Purpose**: Track suspicious-but-allowed input for feedback loop
+
+```typescript
+// Input: { prompt, keywords?, confidence, passed, requestId? }
+// Used to identify novel attack patterns for next version
+```
+
+### 3. POST /api/v1/signal (Layer 3: Blocked Events)
+
+**Called by**: SDK when any layer blocks
+**Purpose**: Security event logging and analytics
+
+```typescript
+interface SignalPayload {
+  events: SecurityEvent[];
+  sdkVersion: string;
+}
+```
+
+Example handler (Next.js):
+```typescript
+export async function POST(req: Request) {
+  const payload = await req.json();
+
+  // Log to your monitoring system
+  console.log(`[Tracerny] ${payload.events.length} security events`);
+  payload.events.forEach((event) => {
+    console.log(`  - ${event.blockReason} (${event.severity})`);
+  });
+
+  // Store in database
+  await db.securityEvents.insertMany(payload.events);
+
+  // Optional: Send alerts for critical blocks
+  if (payload.events.some(e => e.severity === 'CRITICAL')) {
+    await notifySecurityTeam(payload.events);
+  }
+
+  return Response.json({ received: true });
+}
+```
+
+## Latency Profile
+
+**Layer 1 (Vanguard)**: <2ms — Regex pattern matching
+**Layer 2 (Sentinel)**: 200-1500ms — Backend LLM verification (only if Layer 1 misses)
+**Layer 3 (Jitter)**: +300-500ms — Random delay masks which layer executed
+**Total**:
+- Safe prompt Layer 1 only: ~300-500ms (jitter only)
+- Attack Layer 1 hits: ~300-500ms (jitter masks instant block)
+- Novel attack Layer 2 hits: ~500-2000ms (LLM + jitter)
+- No overhead without `sentinelEndpoint`: <3ms (Layer 1 + jitter)
+
+## Provider Support
+
+Designed for provider-agnostic LLM interfaces. Tested with:
+- OpenAI (GPT-4, GPT-3.5)
+- Anthropic (Claude)
+- Others (Azure OpenAI, local models via compatible APIs)
+
+Response structure mapping is automatic for standard provider APIs.
+
+## Graceful Shutdown
+
+```typescript
+process.on('SIGTERM', async () => {
+  shield.destroy();
+  // Ensures any queued events are flushed
+});
+```
+
+## Troubleshooting
+
+### "Cannot find module 'tracerney'"
+Build the SDK first:
+```bash
+npm run build
+```
+
+### "API endpoint not responding"
+Make sure your Signal backend is running:
+```bash
+# In backend directory
+npm run dev
+```
+
+### "Events not appearing in dashboard"
+1. Check `enableTelemetry: true` in config
+2. Verify `apiEndpoint` is reachable: `curl http://localhost:3000/api/v1/stats`
+3. Check browser console for CORS errors
+
+### "Pattern not detecting attacks"
+1. Wait for patterns to load: `shield.getStatus().patternMatcher.ready`
+2. Check if attack matches bundled patterns (20+ vanguard patterns with homoglyph detection)
+3. If Layer 1 misses, enable Layer 2: Set `sentinelEndpoint`
+4. Test directly: `npm install @sandrobuilds/tracerney && node test-layer1.js`
+
+### "Layer 2 verification failing (sentiment not working)"
+1. Check backend is running: `curl http://localhost:3000/api/v1/health`
+2. Verify `OPENROUTER_API_KEY` is set in backend `.env` (not SDK, kept secure on backend)
+3. Check OpenRouter API key is valid: https://openrouter.ai/keys
+4. Look for 429 errors = rate limit hit (wait 60 seconds)
+5. Check backend logs: `cd backend && npm run dev`
+
+### "Getting 429 Rate Limit errors"
+1. **Expected behavior** — 5 verifications per API key per minute
+2. Wait 60+ seconds for window to reset
+3. Or: Use different API key for testing
+4. Or: Change limit in backend: `limiter.check(key, 10, 60000)` (10 calls/min)
+
+### "High latency from shield.wrap()"
+- Layer 1 only (no `sentinelEndpoint`): Should be <3ms + jitter (300-500ms)
+- With Layer 2: 200-1500ms (LLM) + jitter (300-500ms)
+- If higher, check:
+  - Is OpenRouter API slow? (check https://status.openrouter.ai)
+  - Is backend responding? `curl http://localhost:3000/api/v1/verify-prompt`
+  - Profile: `console.time('shield'); await shield.wrap(...); console.timeEnd('shield');`
+
+---
+
+## FAQ
+
+**Q: Will Shield block my legitimate use cases?**
+A: Shield uses regex patterns targeting known injection techniques. False positives are rare. If you see them, add to allowlist or adjust patterns in your Signal backend.
+
+**Q: Does my data go to your servers?**
+A: No. You own the `apiEndpoint` where events are sent. Shield is a runtime library that runs in your process. No data leaves your infrastructure unless you configure it.
+
+**Q: Can I use Shield without telemetry?**
+A: Yes! Set only `allowedTools`:
+```typescript
+const shield = new Tracerney({ allowedTools: ['search'] });
+```
+
+**Q: What if an attack isn't detected?**
+A: Layer 1 catches known patterns. Novel attacks are caught by Layer 2 (backend LLM):
+1. Set `sentinelEndpoint` to enable Layer 2
+2. Provide `OPENROUTER_API_KEY` in backend `.env`
+3. Rate limiting (5 calls/min) prevents botnet cost spikes
+4. Monitor `shadow_log` for potential attacks that pass both layers
+
+**Q: How does Layer 2 rate limiting work?**
+A: Backend enforces 5 verifications per API key per minute:
+- First 5 calls: Returns verification result + `remaining` count
+- Call 6+: Returns HTTP 429 with `blocked: true, reason: 'rate_limit'`
+- Resets every 60 seconds
+
+Example:
+```typescript
+// SDK automatically handles 429
+try {
+  await shield.scanPrompt(userInput);
+} catch (err) {
+  if (err.event.blockReason === 'rate_limit') {
+    return res.status(429).json({ error: 'Too many verification requests' });
+  }
+}
+```
+
+**Q: What's the difference between `apiEndpoint` and `sentinelEndpoint`?**
+A:
+- `sentinelEndpoint`: Layer 2 LLM verification (backend security check)
+- `shadowLogEndpoint`: Layer 3 potential attacks (feedback loop)
+- `apiEndpoint`: Layer 3 security events (blocked attacks)
+
+Use all three for complete observability.
+
+**Q: Does Shield work with streaming responses?**
+A: Yes. `shield.wrap()` works with streaming LLM responses—detection happens after the stream completes.
+
+**Q: Can I rate-limit based on Shield blocks?**
+A: Yes. Use `ShieldBlockError` to track patterns:
+```typescript
+const blocks = new Map();
+try {
+  await shield.wrap(() => llmCall());
+} catch (err) {
+  if (err instanceof ShieldBlockError) {
+    const pattern = err.event.patternName;
+    blocks.set(pattern, (blocks.get(pattern) || 0) + 1);
+    if (blocks.get(pattern) > 5) banIP();
+  }
+}
+```
+
+**Q: What's the difference between `scanPrompt()` and `wrap()`?**
+- `scanPrompt()`: Scans input **before** LLM call (edge defense)
+- `wrap()`: Scans input + validates tool output **after** LLM call (comprehensive defense)
+
+Use both for defense-in-depth.
+
+**Q: How do I update patterns without redeploying?**
+A: Set `manifestUrl` to your backend. The SDK checks for new patterns every 24h and uses stale-while-revalidate caching. Push new patterns to your backend—all clients update automatically.
+
+**Q: Can Shield work offline?**
+A: Yes! Uses bundled 20 vanguard patterns offline. For remote patterns, install fails gracefully and uses cached version.
+
+---
+
+## Resources
+
+- **GitHub**: https://github.com/sandrosaric/tracerney
+- **Issues**: Report bugs or request features
+- **Dashboard**: Built-in real-time threat monitoring at `http://localhost:3000`
+- **Examples**: See `/examples` for Next.js, Express, serverless setups
+
+---
+
+## License
+
+MIT