ship-safe 1.0.1 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +281 -23
- package/ai-defense/cost-protection.md +292 -0
- package/ai-defense/llm-security-checklist.md +324 -0
- package/ai-defense/prompt-injection-patterns.js +283 -0
- package/cli/bin/ship-safe.js +44 -2
- package/cli/commands/fix.js +216 -0
- package/cli/commands/guard.js +297 -0
- package/cli/commands/mcp.js +303 -0
- package/cli/commands/scan.js +231 -39
- package/cli/utils/entropy.js +126 -0
- package/cli/utils/output.js +10 -1
- package/cli/utils/patterns.js +376 -24
- package/configs/firebase/firestore-rules.txt +215 -0
- package/configs/firebase/security-checklist.md +236 -0
- package/configs/firebase/storage-rules.txt +206 -0
- package/configs/ship-safeignore-template +50 -0
- package/configs/supabase/rls-templates.sql +242 -0
- package/configs/supabase/secure-client.ts +225 -0
- package/configs/supabase/security-checklist.md +278 -0
- package/package.json +11 -2
- package/snippets/README.md +89 -25
- package/snippets/api-security/api-security-checklist.md +412 -0
- package/snippets/api-security/cors-config.ts +322 -0
- package/snippets/api-security/input-validation.ts +430 -0
- package/snippets/auth/jwt-checklist.md +322 -0
- package/snippets/rate-limiting/nextjs-middleware.ts +211 -0
- package/snippets/rate-limiting/upstash-ratelimit.ts +229 -0
package/README.md
CHANGED
|
@@ -1,6 +1,17 @@
|
|
|
1
|
-
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src=".github/assets/logo%20ship%20safe.png" alt="Ship Safe Logo" width="180" />
|
|
3
|
+
</p>
|
|
2
4
|
|
|
3
|
-
|
|
5
|
+
<h1 align="center">Ship Safe</h1>
|
|
6
|
+
|
|
7
|
+
<p align="center"><strong>Don't let vibe coding leak your API keys.</strong></p>
|
|
8
|
+
|
|
9
|
+
<p align="center">
|
|
10
|
+
<a href="https://www.npmjs.com/package/ship-safe"><img src="https://badge.fury.io/js/ship-safe.svg" alt="npm version" /></a>
|
|
11
|
+
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT" /></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
---
|
|
4
15
|
|
|
5
16
|
You're shipping fast. You're using AI to write code. You're one `git push` away from exposing your database credentials to the world.
|
|
6
17
|
|
|
@@ -11,9 +22,15 @@ You're shipping fast. You're using AI to write code. You're one `git push` away
|
|
|
11
22
|
## Quick Start
|
|
12
23
|
|
|
13
24
|
```bash
|
|
14
|
-
# Scan
|
|
25
|
+
# Scan for leaked secrets (no install required!)
|
|
15
26
|
npx ship-safe scan .
|
|
16
27
|
|
|
28
|
+
# Auto-generate .env.example from found secrets
|
|
29
|
+
npx ship-safe fix
|
|
30
|
+
|
|
31
|
+
# Block git push if secrets are found
|
|
32
|
+
npx ship-safe guard
|
|
33
|
+
|
|
17
34
|
# Run the launch-day security checklist
|
|
18
35
|
npx ship-safe checklist
|
|
19
36
|
|
|
@@ -21,7 +38,21 @@ npx ship-safe checklist
|
|
|
21
38
|
npx ship-safe init
|
|
22
39
|
```
|
|
23
40
|
|
|
24
|
-
That's it.
|
|
41
|
+
That's it. Five commands to secure your MVP.
|
|
42
|
+
|
|
43
|
+

|
|
44
|
+
|
|
45
|
+
### Let AI Do It For You
|
|
46
|
+
|
|
47
|
+
Copy this prompt to your AI coding assistant:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
Run "npx ship-safe scan ." on my project and fix any secrets you find.
|
|
51
|
+
Then run "npx ship-safe init" to add security configs.
|
|
52
|
+
Explain what you're doing as you go.
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
[More AI prompts for specific frameworks](./AI_SECURITY_PROMPT.md)
|
|
25
56
|
|
|
26
57
|
---
|
|
27
58
|
|
|
@@ -59,7 +90,40 @@ npx ship-safe scan . -v
|
|
|
59
90
|
|
|
60
91
|
**Exit codes:** Returns `1` if secrets found (useful for CI), `0` if clean.
|
|
61
92
|
|
|
62
|
-
**
|
|
93
|
+
**Flags:**
|
|
94
|
+
- `--json` — structured JSON output for CI pipelines
|
|
95
|
+
- `--sarif` — SARIF format for GitHub Code Scanning
|
|
96
|
+
- `--include-tests` — also scan test/spec/fixture files (excluded by default)
|
|
97
|
+
- `-v` — verbose mode
|
|
98
|
+
|
|
99
|
+
**Suppress false positives:**
|
|
100
|
+
```bash
|
|
101
|
+
const apiKey = 'example-key'; // ship-safe-ignore
|
|
102
|
+
```
|
|
103
|
+
Or exclude paths with `.ship-safeignore` (gitignore syntax).
|
|
104
|
+
|
|
105
|
+
**Custom patterns** — create `.ship-safe.json` in your project root:
|
|
106
|
+
```json
|
|
107
|
+
{
|
|
108
|
+
"patterns": [
|
|
109
|
+
{
|
|
110
|
+
"name": "My Internal API Key",
|
|
111
|
+
"pattern": "MYAPP_[A-Z0-9]{32}",
|
|
112
|
+
"severity": "high",
|
|
113
|
+
"description": "Internal key for myapp services."
|
|
114
|
+
}
|
|
115
|
+
]
|
|
116
|
+
}
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
**Detects 50+ secret patterns:**
|
|
120
|
+
- **AI/ML:** OpenAI, Anthropic, Google AI, Cohere, Replicate, Hugging Face
|
|
121
|
+
- **Auth:** Clerk, Auth0, Supabase Auth
|
|
122
|
+
- **Cloud:** AWS, Google Cloud, Azure
|
|
123
|
+
- **Database:** Supabase, PlanetScale, Neon, MongoDB, PostgreSQL, MySQL
|
|
124
|
+
- **Payment:** Stripe, PayPal
|
|
125
|
+
- **Messaging:** Twilio, SendGrid, Resend
|
|
126
|
+
- **And more:** GitHub tokens, private keys, JWTs, generic secrets
|
|
63
127
|
|
|
64
128
|
---
|
|
65
129
|
|
|
@@ -103,6 +167,66 @@ npx ship-safe init -f
|
|
|
103
167
|
|
|
104
168
|
---
|
|
105
169
|
|
|
170
|
+
### `npx ship-safe fix`
|
|
171
|
+
|
|
172
|
+
Scan for secrets and auto-generate a `.env.example` file.
|
|
173
|
+
|
|
174
|
+
```bash
|
|
175
|
+
# Scan and generate .env.example
|
|
176
|
+
npx ship-safe fix
|
|
177
|
+
|
|
178
|
+
# Preview what would be generated without writing it
|
|
179
|
+
npx ship-safe fix --dry-run
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
### `npx ship-safe guard`
|
|
185
|
+
|
|
186
|
+
Install a git hook that blocks pushes if secrets are found. Works with or without Husky.
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
# Install pre-push hook (runs scan before every git push)
|
|
190
|
+
npx ship-safe guard
|
|
191
|
+
|
|
192
|
+
# Install pre-commit hook instead
|
|
193
|
+
npx ship-safe guard --pre-commit
|
|
194
|
+
|
|
195
|
+
# Remove installed hooks
|
|
196
|
+
npx ship-safe guard remove
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
**Suppress false positives:**
|
|
200
|
+
- Add `# ship-safe-ignore` as a comment on a line to skip it
|
|
201
|
+
- Create `.ship-safeignore` (gitignore syntax) to exclude paths
|
|
202
|
+
|
|
203
|
+
---
|
|
204
|
+
|
|
205
|
+
### `npx ship-safe mcp`
|
|
206
|
+
|
|
207
|
+
Start ship-safe as an MCP server so AI editors can call it directly.
|
|
208
|
+
|
|
209
|
+
**Setup (Claude Desktop)** — add to `claude_desktop_config.json`:
|
|
210
|
+
```json
|
|
211
|
+
{
|
|
212
|
+
"mcpServers": {
|
|
213
|
+
"ship-safe": {
|
|
214
|
+
"command": "npx",
|
|
215
|
+
"args": ["ship-safe", "mcp"]
|
|
216
|
+
}
|
|
217
|
+
}
|
|
218
|
+
}
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
Works with Claude Desktop, Cursor, Windsurf, Zed, and any MCP-compatible editor.
|
|
222
|
+
|
|
223
|
+
**Available tools:**
|
|
224
|
+
- `scan_secrets` — scan a directory for leaked secrets
|
|
225
|
+
- `get_checklist` — return the security checklist as structured data
|
|
226
|
+
- `analyze_file` — analyze a single file for issues
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
106
230
|
## What's Inside
|
|
107
231
|
|
|
108
232
|
### [`/checklists`](./checklists)
|
|
@@ -111,19 +235,149 @@ npx ship-safe init -f
|
|
|
111
235
|
|
|
112
236
|
### [`/configs`](./configs)
|
|
113
237
|
**Secure defaults for popular stacks. Drop-in ready.**
|
|
114
|
-
- [Next.js Security Headers](./configs/nextjs-security-headers.js) - CSP, X-Frame-Options, and more
|
|
115
238
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
239
|
+
| Stack | Files |
|
|
240
|
+
|-------|-------|
|
|
241
|
+
| **Next.js** | [Security Headers](./configs/nextjs-security-headers.js) - CSP, X-Frame-Options, HSTS |
|
|
242
|
+
| **Supabase** | [RLS Templates](./configs/supabase/rls-templates.sql) \| [Security Checklist](./configs/supabase/security-checklist.md) \| [Secure Client](./configs/supabase/secure-client.ts) |
|
|
243
|
+
| **Firebase** | [Firestore Rules](./configs/firebase/firestore-rules.txt) \| [Storage Rules](./configs/firebase/storage-rules.txt) \| [Security Checklist](./configs/firebase/security-checklist.md) |
|
|
119
244
|
|
|
120
245
|
### [`/snippets`](./snippets)
|
|
121
246
|
**Copy-paste code blocks for common security patterns.**
|
|
122
|
-
|
|
247
|
+
|
|
248
|
+
| Category | Files |
|
|
249
|
+
|----------|-------|
|
|
250
|
+
| **Rate Limiting** | [Upstash Redis](./snippets/rate-limiting/upstash-ratelimit.ts) \| [Next.js Middleware](./snippets/rate-limiting/nextjs-middleware.ts) |
|
|
251
|
+
| **Authentication** | [JWT Security Checklist](./snippets/auth/jwt-checklist.md) |
|
|
252
|
+
| **API Security** | [CORS Config](./snippets/api-security/cors-config.ts) \| [Input Validation](./snippets/api-security/input-validation.ts) \| [API Checklist](./snippets/api-security/api-security-checklist.md) |
|
|
123
253
|
|
|
124
254
|
### [`/ai-defense`](./ai-defense)
|
|
125
|
-
**Protect your AI features from abuse.**
|
|
126
|
-
|
|
255
|
+
**Protect your AI features from abuse and cost explosions.**
|
|
256
|
+
|
|
257
|
+
| File | Description |
|
|
258
|
+
|------|-------------|
|
|
259
|
+
| [LLM Security Checklist](./ai-defense/llm-security-checklist.md) | Based on OWASP LLM Top 10 - prompt injection, data protection, scope control |
|
|
260
|
+
| [Prompt Injection Patterns](./ai-defense/prompt-injection-patterns.js) | Regex patterns to detect 25+ injection attempts |
|
|
261
|
+
| [Cost Protection Guide](./ai-defense/cost-protection.md) | Prevent $50k surprise bills - rate limits, budget caps, circuit breakers |
|
|
262
|
+
| [System Prompt Armor](./ai-defense/system-prompt-armor.md) | Template for hardened system prompts |
|
|
263
|
+
|
|
264
|
+
### [`/scripts`](./scripts)
|
|
265
|
+
**Automated scanning tools. Run them in CI or locally.**
|
|
266
|
+
- [Secret Scanner](./scripts/scan_secrets.py) - Python version of the secret scanner
|
|
267
|
+
|
|
268
|
+
---
|
|
269
|
+
|
|
270
|
+
## AI/LLM Security
|
|
271
|
+
|
|
272
|
+
Building with AI? Don't let it bankrupt you or get hijacked.
|
|
273
|
+
|
|
274
|
+
### Quick Setup
|
|
275
|
+
|
|
276
|
+
```typescript
|
|
277
|
+
import { containsInjectionAttempt } from './ai-defense/prompt-injection-patterns';
|
|
278
|
+
|
|
279
|
+
async function handleChat(userInput: string) {
|
|
280
|
+
// 1. Check for injection attempts
|
|
281
|
+
const { detected } = containsInjectionAttempt(userInput);
|
|
282
|
+
if (detected) {
|
|
283
|
+
return "I can't process that request.";
|
|
284
|
+
}
|
|
285
|
+
|
|
286
|
+
// 2. Rate limit per user
|
|
287
|
+
const { success } = await ratelimit.limit(userId);
|
|
288
|
+
if (!success) {
|
|
289
|
+
return "Too many requests. Please slow down.";
|
|
290
|
+
}
|
|
291
|
+
|
|
292
|
+
// 3. Check budget before calling
|
|
293
|
+
await checkUserBudget(userId, estimatedCost);
|
|
294
|
+
|
|
295
|
+
// 4. Make the API call with token limits
|
|
296
|
+
const response = await openai.chat.completions.create({
|
|
297
|
+
model: 'gpt-4',
|
|
298
|
+
messages,
|
|
299
|
+
max_tokens: 500, // Hard cap
|
|
300
|
+
});
|
|
301
|
+
|
|
302
|
+
return response;
|
|
303
|
+
}
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
### Cost Protection Layers
|
|
307
|
+
|
|
308
|
+
1. **Token limits** - Cap input/output per request
|
|
309
|
+
2. **Rate limits** - Cap requests per user (10/min)
|
|
310
|
+
3. **Budget caps** - Daily ($1) and monthly ($10) per user
|
|
311
|
+
4. **Circuit breaker** - Disable AI when global budget hit
|
|
312
|
+
5. **Provider limits** - Set hard limits in OpenAI/Anthropic dashboard
|
|
313
|
+
|
|
314
|
+
[Full cost protection guide →](./ai-defense/cost-protection.md)
|
|
315
|
+
|
|
316
|
+
---
|
|
317
|
+
|
|
318
|
+
## Database Security
|
|
319
|
+
|
|
320
|
+
### Supabase RLS Templates
|
|
321
|
+
|
|
322
|
+
```sql
|
|
323
|
+
-- Users can only see their own data
|
|
324
|
+
CREATE POLICY "Users own their data" ON items
|
|
325
|
+
FOR ALL USING (auth.uid() = user_id);
|
|
326
|
+
|
|
327
|
+
-- Read-only public data
|
|
328
|
+
CREATE POLICY "Public read access" ON public_items
|
|
329
|
+
FOR SELECT USING (true);
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
[6 more RLS patterns →](./configs/supabase/rls-templates.sql)
|
|
333
|
+
|
|
334
|
+
### Firebase Security Rules
|
|
335
|
+
|
|
336
|
+
```javascript
|
|
337
|
+
// Users can only access their own documents
|
|
338
|
+
match /users/{userId} {
|
|
339
|
+
allow read, write: if request.auth != null
|
|
340
|
+
&& request.auth.uid == userId;
|
|
341
|
+
}
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
[Full Firestore rules template →](./configs/firebase/firestore-rules.txt)
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
## API Security
|
|
349
|
+
|
|
350
|
+
### CORS (Don't use `*` in production)
|
|
351
|
+
|
|
352
|
+
```typescript
|
|
353
|
+
const ALLOWED_ORIGINS = [
|
|
354
|
+
'https://yourapp.com',
|
|
355
|
+
'https://www.yourapp.com',
|
|
356
|
+
];
|
|
357
|
+
|
|
358
|
+
// Only allow specific origins
|
|
359
|
+
if (origin && ALLOWED_ORIGINS.includes(origin)) {
|
|
360
|
+
headers['Access-Control-Allow-Origin'] = origin;
|
|
361
|
+
}
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
[CORS configs for Next.js, Express, Fastify, Hono →](./snippets/api-security/cors-config.ts)
|
|
365
|
+
|
|
366
|
+
### Input Validation (Zod)
|
|
367
|
+
|
|
368
|
+
```typescript
|
|
369
|
+
const createUserSchema = z.object({
|
|
370
|
+
email: z.string().email().max(255),
|
|
371
|
+
password: z.string().min(8).max(128),
|
|
372
|
+
});
|
|
373
|
+
|
|
374
|
+
const result = createUserSchema.safeParse(body);
|
|
375
|
+
if (!result.success) {
|
|
376
|
+
return Response.json({ error: result.error.issues }, { status: 400 });
|
|
377
|
+
}
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
[Full validation patterns →](./snippets/api-security/input-validation.ts)
|
|
127
381
|
|
|
128
382
|
---
|
|
129
383
|
|
|
@@ -152,11 +406,13 @@ The scan exits with code `1` if secrets are found, failing your build.
|
|
|
152
406
|
|
|
153
407
|
## The 5-Minute Security Checklist
|
|
154
408
|
|
|
155
|
-
1. Run `npx ship-safe scan .` on your project
|
|
156
|
-
2. Run `npx ship-safe init` to add security configs
|
|
157
|
-
3. Add security headers to your Next.js config
|
|
158
|
-
4. Run `npx ship-safe checklist` before launching
|
|
159
|
-
5. If using AI features,
|
|
409
|
+
1. ✅ Run `npx ship-safe scan .` on your project
|
|
410
|
+
2. ✅ Run `npx ship-safe init` to add security configs
|
|
411
|
+
3. ✅ Add security headers to your Next.js config
|
|
412
|
+
4. ✅ Run `npx ship-safe checklist` before launching
|
|
413
|
+
5. ✅ If using AI features, implement [cost protection](./ai-defense/cost-protection.md)
|
|
414
|
+
6. ✅ If using Supabase, check the [RLS checklist](./configs/supabase/security-checklist.md)
|
|
415
|
+
7. ✅ If using Firebase, check the [Firebase checklist](./configs/firebase/security-checklist.md)
|
|
160
416
|
|
|
161
417
|
---
|
|
162
418
|
|
|
@@ -178,15 +434,17 @@ Found a security pattern that saved your app? Share it!
|
|
|
178
434
|
3. Include educational comments explaining *why* it matters
|
|
179
435
|
4. Open a PR
|
|
180
436
|
|
|
437
|
+
See [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines.
|
|
438
|
+
|
|
181
439
|
---
|
|
182
440
|
|
|
183
|
-
##
|
|
441
|
+
## Security Standards Reference
|
|
184
442
|
|
|
185
|
-
|
|
186
|
-
- [
|
|
187
|
-
- [
|
|
188
|
-
- [
|
|
189
|
-
- [ ]
|
|
443
|
+
This toolkit is based on:
|
|
444
|
+
- [OWASP Top 10 Web 2025](https://owasp.org/Top10/)
|
|
445
|
+
- [OWASP Top 10 Mobile 2024](https://owasp.org/www-project-mobile-top-10/)
|
|
446
|
+
- [OWASP LLM Top 10 2025](https://genai.owasp.org/llm-top-10/)
|
|
447
|
+
- [OWASP API Security Top 10 2023](https://owasp.org/API-Security/)
|
|
190
448
|
|
|
191
449
|
---
|
|
192
450
|
|
|
@@ -0,0 +1,292 @@
|
|
|
1
|
+
# AI Cost Protection Guide
|
|
2
|
+
|
|
3
|
+
**Prevent your AI features from bankrupting you.**
|
|
4
|
+
|
|
5
|
+
Real incidents: $50k+ bills from runaway AI usage, abuse, or misconfiguration.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Why Cost Protection Matters
|
|
10
|
+
|
|
11
|
+
| Scenario | Risk |
|
|
12
|
+
|----------|------|
|
|
13
|
+
| Leaked API key | Anyone can rack up charges on your account |
|
|
14
|
+
| No rate limits | Single user sends 10,000 requests |
|
|
15
|
+
| Long responses | GPT-4 response = $0.03-0.12 per request |
|
|
16
|
+
| Infinite loops | Code bug calls AI repeatedly |
|
|
17
|
+
| Viral launch | 10x traffic = 10x costs |
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Layer 1: API Key Security
|
|
22
|
+
|
|
23
|
+
### Keep keys server-side only
|
|
24
|
+
|
|
25
|
+
```typescript
|
|
26
|
+
// BAD: Key in frontend code
|
|
27
|
+
const openai = new OpenAI({ apiKey: 'sk-...' });
|
|
28
|
+
|
|
29
|
+
// GOOD: Key in server environment variable
|
|
30
|
+
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### Scan for leaked keys
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
npx ship-safe scan .
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### Rotate keys periodically
|
|
40
|
+
|
|
41
|
+
- OpenAI: Dashboard > API Keys > Create new > Delete old
|
|
42
|
+
- Anthropic: Console > API Keys > Rotate
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Layer 2: Request Limits
|
|
47
|
+
|
|
48
|
+
### Token limits per request
|
|
49
|
+
|
|
50
|
+
```typescript
|
|
51
|
+
// Limit input
|
|
52
|
+
const MAX_INPUT_CHARS = 2000;
|
|
53
|
+
if (userInput.length > MAX_INPUT_CHARS) {
|
|
54
|
+
return "Message too long";
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// Limit output
|
|
58
|
+
const response = await openai.chat.completions.create({
|
|
59
|
+
model: 'gpt-4',
|
|
60
|
+
messages: messages,
|
|
61
|
+
max_tokens: 500, // Hard cap on response length
|
|
62
|
+
});
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
### Rate limiting per user
|
|
66
|
+
|
|
67
|
+
```typescript
|
|
68
|
+
import { Ratelimit } from '@upstash/ratelimit';
|
|
69
|
+
|
|
70
|
+
const aiRatelimit = new Ratelimit({
|
|
71
|
+
redis,
|
|
72
|
+
limiter: Ratelimit.slidingWindow(10, '1 m'), // 10 requests/minute
|
|
73
|
+
});
|
|
74
|
+
|
|
75
|
+
async function aiHandler(request, userId) {
|
|
76
|
+
const { success } = await aiRatelimit.limit(userId);
|
|
77
|
+
if (!success) {
|
|
78
|
+
return new Response('Too many requests', { status: 429 });
|
|
79
|
+
}
|
|
80
|
+
// Process request
|
|
81
|
+
}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### Global rate limiting
|
|
85
|
+
|
|
86
|
+
```typescript
|
|
87
|
+
const globalRatelimit = new Ratelimit({
|
|
88
|
+
redis,
|
|
89
|
+
limiter: Ratelimit.slidingWindow(1000, '1 h'), // 1000 requests/hour total
|
|
90
|
+
prefix: 'ratelimit:global:ai',
|
|
91
|
+
});
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## Layer 3: Budget Caps
|
|
97
|
+
|
|
98
|
+
### Track usage in database
|
|
99
|
+
|
|
100
|
+
```typescript
|
|
101
|
+
interface AIUsageRecord {
|
|
102
|
+
userId: string;
|
|
103
|
+
model: string;
|
|
104
|
+
inputTokens: number;
|
|
105
|
+
outputTokens: number;
|
|
106
|
+
cost: number;
|
|
107
|
+
timestamp: Date;
|
|
108
|
+
}
|
|
109
|
+
|
|
110
|
+
async function logUsage(usage: AIUsageRecord) {
|
|
111
|
+
await db.aiUsage.create({ data: usage });
|
|
112
|
+
}
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
### Calculate cost before request
|
|
116
|
+
|
|
117
|
+
```typescript
|
|
118
|
+
// Approximate cost calculation
|
|
119
|
+
const COSTS = {
|
|
120
|
+
'gpt-4': { input: 0.03 / 1000, output: 0.06 / 1000 },
|
|
121
|
+
'gpt-4-turbo': { input: 0.01 / 1000, output: 0.03 / 1000 },
|
|
122
|
+
'gpt-3.5-turbo': { input: 0.0005 / 1000, output: 0.0015 / 1000 },
|
|
123
|
+
'claude-3-opus': { input: 0.015 / 1000, output: 0.075 / 1000 },
|
|
124
|
+
'claude-3-sonnet': { input: 0.003 / 1000, output: 0.015 / 1000 },
|
|
125
|
+
};
|
|
126
|
+
|
|
127
|
+
function estimateCost(model: string, inputTokens: number, maxOutputTokens: number) {
|
|
128
|
+
const rates = COSTS[model] || COSTS['gpt-4'];
|
|
129
|
+
return (inputTokens * rates.input) + (maxOutputTokens * rates.output);
|
|
130
|
+
}
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
### Enforce user budget
|
|
134
|
+
|
|
135
|
+
```typescript
|
|
136
|
+
async function checkUserBudget(userId: string, estimatedCost: number) {
|
|
137
|
+
const dailyLimit = 1.00; // $1/day per user
|
|
138
|
+
const monthlyLimit = 10.00; // $10/month per user
|
|
139
|
+
|
|
140
|
+
const today = new Date();
|
|
141
|
+
today.setHours(0, 0, 0, 0);
|
|
142
|
+
|
|
143
|
+
const dailyUsage = await db.aiUsage.aggregate({
|
|
144
|
+
where: { userId, timestamp: { gte: today } },
|
|
145
|
+
_sum: { cost: true },
|
|
146
|
+
});
|
|
147
|
+
|
|
148
|
+
if ((dailyUsage._sum.cost || 0) + estimatedCost > dailyLimit) {
|
|
149
|
+
throw new Error('Daily AI budget exceeded');
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
// Similar check for monthly
|
|
153
|
+
}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### Global budget circuit breaker
|
|
157
|
+
|
|
158
|
+
```typescript
|
|
159
|
+
async function checkGlobalBudget(estimatedCost: number) {
|
|
160
|
+
const monthlyBudget = 500.00; // $500/month total
|
|
161
|
+
|
|
162
|
+
const monthStart = new Date();
|
|
163
|
+
monthStart.setDate(1);
|
|
164
|
+
monthStart.setHours(0, 0, 0, 0);
|
|
165
|
+
|
|
166
|
+
const monthlyUsage = await db.aiUsage.aggregate({
|
|
167
|
+
where: { timestamp: { gte: monthStart } },
|
|
168
|
+
_sum: { cost: true },
|
|
169
|
+
});
|
|
170
|
+
|
|
171
|
+
if ((monthlyUsage._sum.cost || 0) + estimatedCost > monthlyBudget) {
|
|
172
|
+
// CIRCUIT BREAKER: Disable AI features
|
|
173
|
+
await disableAIFeatures();
|
|
174
|
+
await alertAdmins('AI budget exceeded - features disabled');
|
|
175
|
+
throw new Error('Service temporarily unavailable');
|
|
176
|
+
}
|
|
177
|
+
}
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## Layer 4: Provider-Side Limits
|
|
183
|
+
|
|
184
|
+
### OpenAI usage limits
|
|
185
|
+
|
|
186
|
+
1. Go to platform.openai.com
|
|
187
|
+
2. Settings > Limits
|
|
188
|
+
3. Set monthly hard limit
|
|
189
|
+
|
|
190
|
+
### Anthropic usage limits
|
|
191
|
+
|
|
192
|
+
1. Go to console.anthropic.com
|
|
193
|
+
2. Settings > Usage Limits
|
|
194
|
+
3. Set spend limits
|
|
195
|
+
|
|
196
|
+
### Set up billing alerts
|
|
197
|
+
|
|
198
|
+
Most providers support alerts at:
|
|
199
|
+
- 50% of budget
|
|
200
|
+
- 80% of budget
|
|
201
|
+
- 100% of budget
|
|
202
|
+
|
|
203
|
+
---
|
|
204
|
+
|
|
205
|
+
## Layer 5: Monitoring & Alerts
|
|
206
|
+
|
|
207
|
+
### Real-time usage dashboard
|
|
208
|
+
|
|
209
|
+
```typescript
|
|
210
|
+
// Track key metrics
|
|
211
|
+
const metrics = {
|
|
212
|
+
requestsPerMinute: await getRequestsPerMinute(),
|
|
213
|
+
costToday: await getCostToday(),
|
|
214
|
+
costThisMonth: await getCostThisMonth(),
|
|
215
|
+
topUsers: await getTopUsersByUsage(),
|
|
216
|
+
errorRate: await getErrorRate(),
|
|
217
|
+
};
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
### Alert on anomalies
|
|
221
|
+
|
|
222
|
+
```typescript
|
|
223
|
+
async function checkForAnomalies() {
|
|
224
|
+
// Alert if hourly cost exceeds normal
|
|
225
|
+
const hourlyCost = await getHourlyCost();
|
|
226
|
+
const avgHourlyCost = await getAvgHourlyCost();
|
|
227
|
+
|
|
228
|
+
if (hourlyCost > avgHourlyCost * 3) {
|
|
229
|
+
await sendAlert({
|
|
230
|
+
type: 'anomaly',
|
|
231
|
+
message: `Hourly AI cost spike: $${hourlyCost} (avg: $${avgHourlyCost})`,
|
|
232
|
+
severity: 'high',
|
|
233
|
+
});
|
|
234
|
+
}
|
|
235
|
+
|
|
236
|
+
// Alert if single user is abusing
|
|
237
|
+
const topUser = await getTopUserThisHour();
|
|
238
|
+
if (topUser.requests > 100) {
|
|
239
|
+
await sendAlert({
|
|
240
|
+
type: 'abuse',
|
|
241
|
+
message: `User ${topUser.id} made ${topUser.requests} AI requests this hour`,
|
|
242
|
+
severity: 'medium',
|
|
243
|
+
});
|
|
244
|
+
}
|
|
245
|
+
}
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
---
|
|
249
|
+
|
|
250
|
+
## Cost Comparison: Models
|
|
251
|
+
|
|
252
|
+
| Model | Input ($/1M tokens) | Output ($/1M tokens) | Best For |
|
|
253
|
+
|-------|---------------------|----------------------|----------|
|
|
254
|
+
| GPT-4 | $30 | $60 | Complex tasks |
|
|
255
|
+
| GPT-4 Turbo | $10 | $30 | Long context |
|
|
256
|
+
| GPT-3.5 Turbo | $0.50 | $1.50 | Simple tasks |
|
|
257
|
+
| Claude 3 Opus | $15 | $75 | Highest quality |
|
|
258
|
+
| Claude 3 Sonnet | $3 | $15 | Balanced |
|
|
259
|
+
| Claude 3 Haiku | $0.25 | $1.25 | Speed/cost |
|
|
260
|
+
|
|
261
|
+
**Tip:** Use cheaper models for simple tasks, reserve expensive models for complex ones.
|
|
262
|
+
|
|
263
|
+
```typescript
|
|
264
|
+
function selectModel(task: string) {
|
|
265
|
+
const simpleTasks = ['summarize', 'classify', 'extract'];
|
|
266
|
+
const complexTasks = ['code', 'analyze', 'create'];
|
|
267
|
+
|
|
268
|
+
if (simpleTasks.some(t => task.includes(t))) {
|
|
269
|
+
return 'gpt-3.5-turbo'; // Cheap and fast
|
|
270
|
+
}
|
|
271
|
+
return 'gpt-4-turbo'; // Better but pricier
|
|
272
|
+
}
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
---
|
|
276
|
+
|
|
277
|
+
## Quick Implementation Checklist
|
|
278
|
+
|
|
279
|
+
1. [ ] API keys in server-side environment variables only
|
|
280
|
+
2. [ ] Input length limits (e.g., 2000 chars)
|
|
281
|
+
3. [ ] Output token limits (e.g., 500 tokens)
|
|
282
|
+
4. [ ] Rate limiting per user (e.g., 10 requests/minute)
|
|
283
|
+
5. [ ] Daily budget per user (e.g., $1/day)
|
|
284
|
+
6. [ ] Global monthly budget with circuit breaker
|
|
285
|
+
7. [ ] Provider-side hard limits configured
|
|
286
|
+
8. [ ] Billing alerts at 50%, 80%, 100%
|
|
287
|
+
9. [ ] Usage tracking in database
|
|
288
|
+
10. [ ] Anomaly detection and alerting
|
|
289
|
+
|
|
290
|
+
---
|
|
291
|
+
|
|
292
|
+
**Remember: A $50,000 surprise bill is a real risk. Implement these layers before launch.**
|