yinzerflow 0.4.4 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +31 -26
- package/docs/configuration/configuration.md +815 -0
- package/docs/core/core-concepts.md +801 -0
- package/docs/core/error-handling.md +391 -153
- package/docs/core/logging.md +426 -68
- package/docs/modules/body-parsing.md +561 -0
- package/docs/modules/cors.md +369 -0
- package/docs/modules/index.md +125 -0
- package/docs/modules/ip-security.md +280 -0
- package/docs/modules/rate-limiting.md +795 -0
- package/index.d.ts +278 -76
- package/index.js +18 -18
- package/index.js.map +17 -8
- package/package.json +5 -3
- package/docs/configuration/advanced-configuration-options.md +0 -302
- package/docs/configuration/configuration-patterns.md +0 -500
- package/docs/core/context.md +0 -230
- package/docs/core/examples.md +0 -444
- package/docs/core/request.md +0 -161
- package/docs/core/response.md +0 -212
- package/docs/core/routes.md +0 -720
- package/docs/quick-reference.md +0 -346
- package/docs/security/body-parsing.md +0 -296
- package/docs/security/cors.md +0 -189
- package/docs/security/ip-security.md +0 -234
- package/docs/security/security-overview.md +0 -282
- package/docs/start-here.md +0 -184
|
@@ -0,0 +1,795 @@
|
|
|
1
|
+
# 📖 Overview
|
|
2
|
+
|
|
3
|
+
YinzerFlow provides built-in rate limiting to protect your API from abuse and DoS attacks. Rate limiting is <span style="color: #2ecc71">**enabled by default**</span> with sensible limits (100 requests per 15 minutes per IP) using the **Sliding Window Counter** algorithm for accurate request tracking with minimal memory overhead.
|
|
4
|
+
|
|
5
|
+
<span style="color: #3498db">**💡 Tip:**</span> Rate limiting is your first line of defense against DoS attacks and API abuse.
|
|
6
|
+
|
|
7
|
+
**When to use:**
|
|
8
|
+
|
|
9
|
+
- 🌐 Public APIs that need DoS protection
|
|
10
|
+
- 🔑 Authentication endpoints to prevent brute force attacks
|
|
11
|
+
- ⚡ Resource-intensive endpoints that need request throttling
|
|
12
|
+
- 🎯 APIs with tiered access (free vs premium users)
|
|
13
|
+
|
|
14
|
+
**Expected outcomes:**
|
|
15
|
+
|
|
16
|
+
- ✅ Automatic protection against request flooding
|
|
17
|
+
- 📊 Standard rate limit headers inform clients about limits
|
|
18
|
+
- 🎨 Customizable limits per route or globally
|
|
19
|
+
- 💾 Memory-efficient tracking with only ~24 bytes per client
|
|
20
|
+
|
|
21
|
+
<span style="color: #3498db">🔗 For configuration options, see the [Configuration Reference](#configuration-reference).</span>
|
|
22
|
+
|
|
23
|
+
# ⚙️ Usage
|
|
24
|
+
|
|
25
|
+
## 🎛️ Settings
|
|
26
|
+
|
|
27
|
+
### enabled — @default <span style="color: #2ecc71">`true`</span>
|
|
28
|
+
|
|
29
|
+
Enable or disable rate limiting globally or per-route.
|
|
30
|
+
|
|
31
|
+
<span style="color: #e74c3c">**⚠️ Warning:**</span> Disabling rate limiting removes DoS protection from your API.
|
|
32
|
+
|
|
33
|
+
```typescript
|
|
34
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
35
|
+
|
|
36
|
+
// Disable globally
|
|
37
|
+
const app = new YinzerFlow({
|
|
38
|
+
port: 3000,
|
|
39
|
+
rateLimit: { enabled: false }
|
|
40
|
+
});
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
<aside>
|
|
44
|
+
|
|
45
|
+
Options: `boolean`
|
|
46
|
+
|
|
47
|
+
- 🟢 `true`: Rate limiting enabled (default, recommended)
|
|
48
|
+
- 🔴 `false`: Rate limiting disabled
|
|
49
|
+
|
|
50
|
+
</aside>
|
|
51
|
+
|
|
52
|
+
### window — @default <span style="color: #2ecc71">`'15m'`</span> (15 minutes)
|
|
53
|
+
|
|
54
|
+
Time window for rate limiting. Accepts friendly format ('30s', '15m', '2h', '1d') or milliseconds.
|
|
55
|
+
|
|
56
|
+
<span style="color: #3498db">**💡 Tip:**</span> Use friendly formats like `'1m'`, `'15m'`, `'1h'` for better readability.
|
|
57
|
+
|
|
58
|
+
```typescript
|
|
59
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
60
|
+
|
|
61
|
+
const app = new YinzerFlow({
|
|
62
|
+
port: 3000,
|
|
63
|
+
rateLimit: {
|
|
64
|
+
window: '1m', // 1 minute
|
|
65
|
+
max: 60
|
|
66
|
+
}
|
|
67
|
+
});
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
<aside>
|
|
71
|
+
|
|
72
|
+
Options: `TimeString | number`
|
|
73
|
+
|
|
74
|
+
- ⏱️ Friendly format: `'30s'`, `'15m'`, `'2h'`, `'1d'`
|
|
75
|
+
- ⏱️ Milliseconds: `60000` (1 minute)
|
|
76
|
+
- ✅ Default: `'15m'` (900000ms)
|
|
77
|
+
|
|
78
|
+
</aside>
|
|
79
|
+
|
|
80
|
+
### max — @default <span style="color: #2ecc71">`100`</span>
|
|
81
|
+
|
|
82
|
+
Maximum number of requests allowed per window.
|
|
83
|
+
|
|
84
|
+
```typescript
|
|
85
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
86
|
+
|
|
87
|
+
const app = new YinzerFlow({
|
|
88
|
+
port: 3000,
|
|
89
|
+
rateLimit: {
|
|
90
|
+
window: '15m',
|
|
91
|
+
max: 100 // 100 requests per 15 minutes
|
|
92
|
+
}
|
|
93
|
+
});
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
<aside>
|
|
97
|
+
|
|
98
|
+
Options: `number`
|
|
99
|
+
|
|
100
|
+
- 🚨 Minimum: `1`
|
|
101
|
+
- ✅ Recommended: `100` for general APIs
|
|
102
|
+
- 🟢 Default: `100`
|
|
103
|
+
|
|
104
|
+
</aside>
|
|
105
|
+
|
|
106
|
+
### algorithm — @default <span style="color: #2ecc71">`'sliding-window-counter'`</span>
|
|
107
|
+
|
|
108
|
+
Rate limiting algorithm to use.
|
|
109
|
+
|
|
110
|
+
```typescript
|
|
111
|
+
import { YinzerFlow, rateLimitAlgorithm } from 'yinzerflow';
|
|
112
|
+
|
|
113
|
+
const app = new YinzerFlow({
|
|
114
|
+
port: 3000,
|
|
115
|
+
rateLimit: {
|
|
116
|
+
algorithm: rateLimitAlgorithm.slidingWindowCounter
|
|
117
|
+
}
|
|
118
|
+
});
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
<aside>
|
|
122
|
+
|
|
123
|
+
Options: `'sliding-window-counter'`
|
|
124
|
+
|
|
125
|
+
- 🟢 `slidingWindowCounter`: Memory efficient, Redis-ready, 99%+ accurate (default and recommended)
|
|
126
|
+
- 🔮 Future algorithms: `tokenBucket`, `slidingWindowLog`
|
|
127
|
+
|
|
128
|
+
</aside>
|
|
129
|
+
|
|
130
|
+
### standardHeaders — @default <span style="color: #2ecc71">`true`</span>
|
|
131
|
+
|
|
132
|
+
Include standard `RateLimit-*` headers in responses to inform clients about limits.
|
|
133
|
+
|
|
134
|
+
```typescript
|
|
135
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
136
|
+
|
|
137
|
+
const app = new YinzerFlow({
|
|
138
|
+
port: 3000,
|
|
139
|
+
rateLimit: {
|
|
140
|
+
standardHeaders: true // Add RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset headers
|
|
141
|
+
}
|
|
142
|
+
});
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
<aside>
|
|
146
|
+
|
|
147
|
+
Options: `boolean`
|
|
148
|
+
|
|
149
|
+
- 🟢 `true`: Include headers (default, recommended)
|
|
150
|
+
- 🔴 `false`: No rate limit headers
|
|
151
|
+
|
|
152
|
+
Headers added:
|
|
153
|
+
- 📊 `RateLimit-Limit`: Maximum requests per window
|
|
154
|
+
- 📊 `RateLimit-Remaining`: Requests remaining in current window
|
|
155
|
+
- ⏱️ `RateLimit-Reset`: Unix timestamp when window resets
|
|
156
|
+
- 🔄 `Retry-After`: Seconds until client can retry (when limit exceeded)
|
|
157
|
+
|
|
158
|
+
</aside>
|
|
159
|
+
|
|
160
|
+
### keyGenerator — @default <span style="color: #2ecc71">IP address</span>
|
|
161
|
+
|
|
162
|
+
Custom function to generate unique client identifier for rate limiting.
|
|
163
|
+
|
|
164
|
+
<span style="color: #3498db">**💡 Tip:**</span> For authenticated APIs, rate limit by user ID instead of IP address.
|
|
165
|
+
|
|
166
|
+
```typescript
|
|
167
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
168
|
+
|
|
169
|
+
const app = new YinzerFlow({
|
|
170
|
+
port: 3000,
|
|
171
|
+
rateLimit: {
|
|
172
|
+
keyGenerator: (ctx) => {
|
|
173
|
+
// Rate limit by user ID instead of IP
|
|
174
|
+
return ctx.state.userId || ctx.request.ipAddress;
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
});
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
<aside>
|
|
181
|
+
|
|
182
|
+
Options: `(context: Context) => string`
|
|
183
|
+
|
|
184
|
+
- ✅ Default: `(ctx) => ctx.request.ipAddress`
|
|
185
|
+
- 🎯 Common patterns:
|
|
186
|
+
- 🔑 User ID: `ctx.state.userId`
|
|
187
|
+
- 🌐 API key: `ctx.request.headers['x-api-key']`
|
|
188
|
+
- 🎨 Combination: `${tier}:${userId}`
|
|
189
|
+
|
|
190
|
+
</aside>
|
|
191
|
+
|
|
192
|
+
### handler — @default <span style="color: #2ecc71">Pittsburgh-themed message</span>
|
|
193
|
+
|
|
194
|
+
Custom handler function called when rate limit is exceeded.
|
|
195
|
+
|
|
196
|
+
```typescript
|
|
197
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
198
|
+
|
|
199
|
+
const app = new YinzerFlow({
|
|
200
|
+
port: 3000,
|
|
201
|
+
rateLimit: {
|
|
202
|
+
handler: (ctx) => {
|
|
203
|
+
ctx.response.setStatusCode(429);
|
|
204
|
+
return {
|
|
205
|
+
success: false,
|
|
206
|
+
message: 'Too many requests. Please try again later.'
|
|
207
|
+
};
|
|
208
|
+
}
|
|
209
|
+
}
|
|
210
|
+
});
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
<aside>
|
|
214
|
+
|
|
215
|
+
Options: `(context: Context) => unknown`
|
|
216
|
+
|
|
217
|
+
- 🟢 Default: Returns `{ success: false, message: "Yinz are sending too many requests. Slow down, jagoff!" }`
|
|
218
|
+
- 🚨 Automatically sets status code 429
|
|
219
|
+
- 🎨 Can return any JSON-serializable object
|
|
220
|
+
|
|
221
|
+
</aside>
|
|
222
|
+
|
|
223
|
+
### skipSuccessfulRequests — @default <span style="color: #2ecc71">`false`</span>
|
|
224
|
+
|
|
225
|
+
Don't count successful requests (status < 400) toward rate limit.
|
|
226
|
+
|
|
227
|
+
```typescript
|
|
228
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
229
|
+
|
|
230
|
+
const app = new YinzerFlow({
|
|
231
|
+
port: 3000,
|
|
232
|
+
rateLimit: {
|
|
233
|
+
skipSuccessfulRequests: true // Only count errors
|
|
234
|
+
}
|
|
235
|
+
});
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
<aside>
|
|
239
|
+
|
|
240
|
+
Options: `boolean`
|
|
241
|
+
|
|
242
|
+
- 🟢 `false`: Count all requests (default, recommended)
|
|
243
|
+
- 🔴 `true`: Only count failed requests (status >= 400)
|
|
244
|
+
|
|
245
|
+
</aside>
|
|
246
|
+
|
|
247
|
+
### skipFailedRequests — @default <span style="color: #2ecc71">`false`</span>
|
|
248
|
+
|
|
249
|
+
Don't count failed requests (status >= 400) toward rate limit.
|
|
250
|
+
|
|
251
|
+
```typescript
|
|
252
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
253
|
+
|
|
254
|
+
const app = new YinzerFlow({
|
|
255
|
+
port: 3000,
|
|
256
|
+
rateLimit: {
|
|
257
|
+
skipFailedRequests: true // Only count successful requests
|
|
258
|
+
}
|
|
259
|
+
});
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
<aside>
|
|
263
|
+
|
|
264
|
+
Options: `boolean`
|
|
265
|
+
|
|
266
|
+
- 🟢 `false`: Count all requests (default, recommended)
|
|
267
|
+
- 🔴 `true`: Only count successful requests (status < 400)
|
|
268
|
+
|
|
269
|
+
</aside>
|
|
270
|
+
|
|
271
|
+
## 📚 Configuration Reference
|
|
272
|
+
|
|
273
|
+
Full configuration example with all options:
|
|
274
|
+
|
|
275
|
+
```typescript
|
|
276
|
+
import { YinzerFlow, rateLimitAlgorithm } from 'yinzerflow';
|
|
277
|
+
|
|
278
|
+
const app = new YinzerFlow({
|
|
279
|
+
port: 3000,
|
|
280
|
+
rateLimit: {
|
|
281
|
+
enabled: true,
|
|
282
|
+
algorithm: rateLimitAlgorithm.slidingWindowCounter,
|
|
283
|
+
window: '15m',
|
|
284
|
+
max: 100,
|
|
285
|
+
standardHeaders: true,
|
|
286
|
+
skipSuccessfulRequests: false,
|
|
287
|
+
skipFailedRequests: false,
|
|
288
|
+
keyGenerator: (ctx) => ctx.request.ipAddress,
|
|
289
|
+
handler: (ctx) => {
|
|
290
|
+
ctx.response.setStatusCode(429);
|
|
291
|
+
return {
|
|
292
|
+
success: false,
|
|
293
|
+
message: 'Yinz are sending too many requests. Slow down, jagoff!'
|
|
294
|
+
};
|
|
295
|
+
}
|
|
296
|
+
}
|
|
297
|
+
});
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
# ✨ Best Practices
|
|
301
|
+
|
|
302
|
+
- ✅ **Enable by default**: Keep rate limiting enabled in production for security
|
|
303
|
+
- 🔑 **Per-route limits**: Use stricter limits for sensitive endpoints (auth, password reset)
|
|
304
|
+
- 📊 **Standard headers**: Keep `standardHeaders: true` to help well-behaved clients
|
|
305
|
+
- 🎯 **Custom key generator**: Rate limit by user ID for authenticated APIs
|
|
306
|
+
- 📝 **Monitor violations**: Log rate limit exceeded events for security analysis
|
|
307
|
+
- 💬 **Graceful degradation**: Provide helpful error messages when limits are hit
|
|
308
|
+
|
|
309
|
+
# 💻 Examples
|
|
310
|
+
|
|
311
|
+
### Production API
|
|
312
|
+
|
|
313
|
+
**Use Case:** Secure public API with authentication endpoints
|
|
314
|
+
|
|
315
|
+
**Description:** Production-ready configuration with global rate limiting and strict per-route limits for sensitive endpoints.
|
|
316
|
+
|
|
317
|
+
<span style="color: #f39c12">**⚡ Performance:**</span> This configuration balances security and user experience.
|
|
318
|
+
|
|
319
|
+
```typescript
|
|
320
|
+
import { YinzerFlow, rateLimitHook, log } from 'yinzerflow';
|
|
321
|
+
|
|
322
|
+
const app = new YinzerFlow({
|
|
323
|
+
port: 3000,
|
|
324
|
+
rateLimit: {
|
|
325
|
+
enabled: true,
|
|
326
|
+
window: '15m',
|
|
327
|
+
max: 100, // General API limit
|
|
328
|
+
standardHeaders: true
|
|
329
|
+
}
|
|
330
|
+
});
|
|
331
|
+
|
|
332
|
+
// Very strict for login to prevent brute force
|
|
333
|
+
app.post('/api/auth/login',
|
|
334
|
+
{
|
|
335
|
+
beforeRoute: [rateLimitHook({
|
|
336
|
+
window: '15m',
|
|
337
|
+
max: 5, // Only 5 attempts per 15 minutes
|
|
338
|
+
handler: (ctx) => {
|
|
339
|
+
log.warn(`Rate limit exceeded for IP: ${ctx.request.ipAddress}`);
|
|
340
|
+
ctx.response.setStatusCode(429);
|
|
341
|
+
return {
|
|
342
|
+
success: false,
|
|
343
|
+
message: 'Too many login attempts. Please try again later.'
|
|
344
|
+
};
|
|
345
|
+
}
|
|
346
|
+
})]
|
|
347
|
+
},
|
|
348
|
+
async ({ request }) => {
|
|
349
|
+
const { email, password } = request.body;
|
|
350
|
+
return await authenticateUser(email, password);
|
|
351
|
+
}
|
|
352
|
+
);
|
|
353
|
+
|
|
354
|
+
// Strict for password reset
|
|
355
|
+
app.post('/api/auth/reset-password',
|
|
356
|
+
{
|
|
357
|
+
beforeRoute: [rateLimitHook({
|
|
358
|
+
window: '1h',
|
|
359
|
+
max: 3 // Only 3 reset attempts per hour
|
|
360
|
+
})]
|
|
361
|
+
},
|
|
362
|
+
async ({ request }) => {
|
|
363
|
+
const { email } = request.body;
|
|
364
|
+
return await sendPasswordResetEmail(email);
|
|
365
|
+
}
|
|
366
|
+
);
|
|
367
|
+
|
|
368
|
+
// Lower limit for expensive search endpoint
|
|
369
|
+
app.post('/api/search',
|
|
370
|
+
{
|
|
371
|
+
beforeRoute: [rateLimitHook({
|
|
372
|
+
window: '1m',
|
|
373
|
+
max: 10 // 10 searches per minute
|
|
374
|
+
})]
|
|
375
|
+
},
|
|
376
|
+
async ({ request }) => {
|
|
377
|
+
return await performExpensiveSearch(request.body);
|
|
378
|
+
}
|
|
379
|
+
);
|
|
380
|
+
|
|
381
|
+
await app.listen();
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
### Custom Key Generator (User-Based)
|
|
385
|
+
|
|
386
|
+
**Use Case:** Rate limit authenticated users by user ID
|
|
387
|
+
|
|
388
|
+
**Description:** Production API that rate limits by user ID instead of IP address for better accuracy with authenticated users.
|
|
389
|
+
|
|
390
|
+
```typescript
|
|
391
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
392
|
+
|
|
393
|
+
const app = new YinzerFlow({
|
|
394
|
+
port: 3000,
|
|
395
|
+
rateLimit: {
|
|
396
|
+
window: '1h',
|
|
397
|
+
max: 1000, // 1000 requests per hour per user
|
|
398
|
+
keyGenerator: (ctx) => {
|
|
399
|
+
// Rate limit by user ID if authenticated, otherwise by IP
|
|
400
|
+
return ctx.state.userId || ctx.request.ipAddress;
|
|
401
|
+
}
|
|
402
|
+
}
|
|
403
|
+
});
|
|
404
|
+
|
|
405
|
+
// Middleware to extract user ID from JWT
|
|
406
|
+
app.beforeAll([
|
|
407
|
+
async (ctx) => {
|
|
408
|
+
const token = ctx.request.headers.authorization?.replace('Bearer ', '');
|
|
409
|
+
if (token) {
|
|
410
|
+
const decoded = await verifyJWT(token);
|
|
411
|
+
ctx.state.userId = decoded.userId;
|
|
412
|
+
}
|
|
413
|
+
}
|
|
414
|
+
]);
|
|
415
|
+
|
|
416
|
+
app.get('/api/user/profile', async ({ state }) => {
|
|
417
|
+
return await getUserProfile(state.userId);
|
|
418
|
+
});
|
|
419
|
+
|
|
420
|
+
await app.listen();
|
|
421
|
+
```
|
|
422
|
+
|
|
423
|
+
### Tiered Access (Free vs Premium)
|
|
424
|
+
|
|
425
|
+
**Use Case:** Different rate limits for free and premium users
|
|
426
|
+
|
|
427
|
+
**Description:** API with tiered access where premium users get higher limits than free users.
|
|
428
|
+
|
|
429
|
+
```typescript
|
|
430
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
431
|
+
|
|
432
|
+
const app = new YinzerFlow({
|
|
433
|
+
port: 3000,
|
|
434
|
+
rateLimit: {
|
|
435
|
+
window: '1m',
|
|
436
|
+
max: 10, // Base limit for free users
|
|
437
|
+
keyGenerator: (ctx) => {
|
|
438
|
+
const tier = ctx.state.userTier || 'free';
|
|
439
|
+
const userId = ctx.state.userId || ctx.request.ipAddress;
|
|
440
|
+
return `${tier}:${userId}`;
|
|
441
|
+
}
|
|
442
|
+
}
|
|
443
|
+
});
|
|
444
|
+
|
|
445
|
+
// Middleware to determine user tier
|
|
446
|
+
app.beforeAll([
|
|
447
|
+
async (ctx) => {
|
|
448
|
+
if (ctx.state.userId) {
|
|
449
|
+
ctx.state.userTier = await getUserTier(ctx.state.userId);
|
|
450
|
+
}
|
|
451
|
+
}
|
|
452
|
+
]);
|
|
453
|
+
|
|
454
|
+
// Premium endpoints can have higher per-route limits
|
|
455
|
+
app.get('/api/premium/analytics',
|
|
456
|
+
{
|
|
457
|
+
beforeRoute: [rateLimitHook({
|
|
458
|
+
window: '1m',
|
|
459
|
+
max: 100 // Higher limit for premium endpoint
|
|
460
|
+
})]
|
|
461
|
+
},
|
|
462
|
+
async ({ state }) => {
|
|
463
|
+
return await getPremiumAnalytics(state.userId);
|
|
464
|
+
}
|
|
465
|
+
);
|
|
466
|
+
|
|
467
|
+
await app.listen();
|
|
468
|
+
```
|
|
469
|
+
|
|
470
|
+
## 🚀 Performance Notes
|
|
471
|
+
|
|
472
|
+
YinzerFlow's rate limiter is designed for high performance:
|
|
473
|
+
|
|
474
|
+
- ⚡ **O(1) lookups**: Uses Map for constant-time access
|
|
475
|
+
- 🎯 **Minimal overhead**: Adds only ~0.1-0.5ms per request
|
|
476
|
+
- 💾 **Memory efficient**: Only 3 numbers per client (~24 bytes) vs 100+ timestamps (~800 bytes) for sliding window log
|
|
477
|
+
- 🔄 **No blocking**: Fully synchronous algorithm with no async operations needed for rate checks
|
|
478
|
+
- 🧹 **Automatic cleanup**: Windows naturally expire without explicit cleanup loops
|
|
479
|
+
|
|
480
|
+
<span style="color: #f39c12">**⚡ Performance:**</span> Memory comparison:
|
|
481
|
+
|
|
482
|
+
- **Sliding Window Counter** (YinzerFlow): ~24 bytes per client (3 numbers)
|
|
483
|
+
- **Sliding Window Log**: ~800 bytes per client (100+ timestamps)
|
|
484
|
+
- **Savings**: 33x less memory usage
|
|
485
|
+
|
|
486
|
+
**Algorithm complexity:**
|
|
487
|
+
|
|
488
|
+
- ⚡ All operations: O(1) constant time
|
|
489
|
+
- 💾 Memory per client: O(1) constant space
|
|
490
|
+
- ✅ No background cleanup needed
|
|
491
|
+
|
|
492
|
+
<span style="color: #2ecc71">**✅ Result:**</span> For most applications, rate limiting overhead is negligible (< 1%) compared to actual request processing.
|
|
493
|
+
|
|
494
|
+
## 🔒 Security Notes
|
|
495
|
+
|
|
496
|
+
YinzerFlow implements several security measures to prevent abuse while maintaining excellent performance:
|
|
497
|
+
|
|
498
|
+
### 🛡️ Sliding Window Counter Algorithm
|
|
499
|
+
|
|
500
|
+
- **Problem**: Fixed window algorithms allow burst traffic at window boundaries (100 requests at 11:59, 100 more at 12:00). Traditional sliding window log algorithms store every timestamp, consuming excessive memory.
|
|
501
|
+
- **YinzerFlow Solution**: Uses **Sliding Window Counter** algorithm which provides 99%+ accuracy while using 33x less memory. Stores only 3 numbers per client (current count, previous count, window start) instead of 100+ timestamps.
|
|
502
|
+
|
|
503
|
+
### 🛡️ Per-IP Protection
|
|
504
|
+
|
|
505
|
+
- **Problem**: Without per-client tracking, a single abusive client can consume all available resources.
|
|
506
|
+
- **YinzerFlow Solution**: Default key generator uses IP address from YinzerFlow's IP security system, ensuring accurate client identification even behind proxies and load balancers. <span style="color: #3498db">🔗 For proxy configuration, see [IP Security](./ip-security.md)</span>.
|
|
507
|
+
|
|
508
|
+
### 🛡️ Memory Efficiency
|
|
509
|
+
|
|
510
|
+
- **Problem**: Storing rate limit data for every client can cause memory exhaustion in high-traffic scenarios.
|
|
511
|
+
- **YinzerFlow Solution**: Sliding window counter uses only ~24 bytes per client, and windows automatically expire without explicit cleanup loops.
|
|
512
|
+
|
|
513
|
+
### 🛡️ Standard Headers
|
|
514
|
+
|
|
515
|
+
- **Problem**: Clients hitting rate limits repeatedly waste server resources.
|
|
516
|
+
- **YinzerFlow Solution**: Standard `RateLimit-*` headers inform clients about limits, helping well-behaved clients avoid violations.
|
|
517
|
+
|
|
518
|
+
### 🛡️ Per-Route Limits
|
|
519
|
+
|
|
520
|
+
- **Problem**: One-size-fits-all limits don't work for different endpoint types (public vs authenticated, read vs write).
|
|
521
|
+
- **YinzerFlow Solution**: Per-route rate limiting with `rateLimitHook()` allows strict limits for sensitive endpoints while keeping generous limits for general API usage.
|
|
522
|
+
|
|
523
|
+
### 🛡️ Custom Key Generators
|
|
524
|
+
|
|
525
|
+
- **Problem**: IP-based limiting doesn't work well for authenticated APIs where users might share IPs (corporate networks, VPNs).
|
|
526
|
+
- **YinzerFlow Solution**: Custom key generators enable rate limiting by user ID, API key, or any other identifier.
|
|
527
|
+
|
|
528
|
+
### 🛡️ Retry-After Header
|
|
529
|
+
|
|
530
|
+
- **Problem**: Clients don't know when they can retry, leading to repeated failed requests.
|
|
531
|
+
- **YinzerFlow Solution**: Automatic `Retry-After` header tells clients exactly when to retry, reducing unnecessary traffic.
|
|
532
|
+
|
|
533
|
+
### 🛡️ Integration with IP Security
|
|
534
|
+
|
|
535
|
+
- **Problem**: Rate limiters using untrusted IP headers can be bypassed by spoofing X-Forwarded-For.
|
|
536
|
+
- **YinzerFlow Solution**: Uses YinzerFlow's IP security system with trusted proxy validation, preventing IP spoofing attacks.
|
|
537
|
+
|
|
538
|
+
<span style="color: #2ecc71">**✅ Result:**</span> These security measures ensure YinzerFlow's rate limiting provides robust protection against DoS attacks and API abuse while maintaining excellent performance.
|
|
539
|
+
|
|
540
|
+
## 🔧 Troubleshooting
|
|
541
|
+
|
|
542
|
+
### Rate limit exceeded immediately on first request
|
|
543
|
+
|
|
544
|
+
**Symptom:** First request to endpoint returns <span style="color: #e74c3c">429 status code</span>.
|
|
545
|
+
|
|
546
|
+
**Cause:** Clock skew or incorrect window configuration.
|
|
547
|
+
|
|
548
|
+
<span style="color: #2ecc71">**✅ Fix:**</span>
|
|
549
|
+
```typescript
|
|
550
|
+
// Ensure window is set correctly
|
|
551
|
+
const app = new YinzerFlow({
|
|
552
|
+
port: 3000,
|
|
553
|
+
rateLimit: {
|
|
554
|
+
window: '15m', // Use friendly format
|
|
555
|
+
max: 100
|
|
556
|
+
}
|
|
557
|
+
});
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
### Rate limit headers not appearing
|
|
561
|
+
|
|
562
|
+
**Symptom:** Responses don't include `RateLimit-*` headers.
|
|
563
|
+
|
|
564
|
+
**Cause:** `standardHeaders` is set to <span style="color: #e74c3c">`false`</span>.
|
|
565
|
+
|
|
566
|
+
<span style="color: #2ecc71">**✅ Fix:**</span>
|
|
567
|
+
```typescript
|
|
568
|
+
const app = new YinzerFlow({
|
|
569
|
+
port: 3000,
|
|
570
|
+
rateLimit: {
|
|
571
|
+
standardHeaders: true // Enable headers
|
|
572
|
+
}
|
|
573
|
+
});
|
|
574
|
+
```
|
|
575
|
+
|
|
576
|
+
### Custom handler not being called
|
|
577
|
+
|
|
578
|
+
**Symptom:** Custom rate limit handler is ignored.
|
|
579
|
+
|
|
580
|
+
**Cause:** Handler must set status code and return response.
|
|
581
|
+
|
|
582
|
+
<span style="color: #2ecc71">**✅ Fix:**</span>
|
|
583
|
+
```typescript
|
|
584
|
+
const app = new YinzerFlow({
|
|
585
|
+
port: 3000,
|
|
586
|
+
rateLimit: {
|
|
587
|
+
handler: (ctx) => {
|
|
588
|
+
ctx.response.setStatusCode(429); // Must set status code
|
|
589
|
+
return {
|
|
590
|
+
success: false,
|
|
591
|
+
message: 'Too many requests'
|
|
592
|
+
};
|
|
593
|
+
}
|
|
594
|
+
}
|
|
595
|
+
});
|
|
596
|
+
```
|
|
597
|
+
|
|
598
|
+
### Different users sharing same rate limit
|
|
599
|
+
|
|
600
|
+
**Symptom:** Authenticated users with different IDs are being rate limited together.
|
|
601
|
+
|
|
602
|
+
**Cause:** Default key generator uses IP address, not user ID. This commonly happens when users are:
|
|
603
|
+
|
|
604
|
+
- 🌐 Behind the same corporate VPN
|
|
605
|
+
- 🏢 On the same corporate network/proxy
|
|
606
|
+
- 🌐 Using shared office internet
|
|
607
|
+
- 🔄 Behind CGNAT (Carrier-Grade NAT) in some ISPs
|
|
608
|
+
|
|
609
|
+
<span style="color: #2ecc71">**✅ Fix:**</span> Use a custom key generator to rate limit by user ID instead of IP address.
|
|
610
|
+
|
|
611
|
+
<span style="color: #3498db">**💡 Tip:**</span> For authenticated APIs, always rate limit by user ID instead of IP address.
|
|
612
|
+
```typescript
|
|
613
|
+
const app = new YinzerFlow({
|
|
614
|
+
port: 3000,
|
|
615
|
+
rateLimit: {
|
|
616
|
+
keyGenerator: (ctx) => {
|
|
617
|
+
// Rate limit by user ID for authenticated users, fall back to IP
|
|
618
|
+
return ctx.state.userId || ctx.request.ipAddress;
|
|
619
|
+
}
|
|
620
|
+
}
|
|
621
|
+
});
|
|
622
|
+
|
|
623
|
+
// Middleware to extract user ID from authentication
|
|
624
|
+
app.beforeAll([
|
|
625
|
+
async (ctx) => {
|
|
626
|
+
const token = ctx.request.headers.authorization?.replace('Bearer ', '');
|
|
627
|
+
if (token) {
|
|
628
|
+
const decoded = await verifyJWT(token);
|
|
629
|
+
ctx.state.userId = decoded.userId;
|
|
630
|
+
}
|
|
631
|
+
}
|
|
632
|
+
]);
|
|
633
|
+
```
|
|
634
|
+
|
|
635
|
+
### Per-route rate limit not working
|
|
636
|
+
|
|
637
|
+
**Symptom:** Per-route `rateLimitHook` doesn't seem to apply.
|
|
638
|
+
|
|
639
|
+
**Cause:** Per-route limits are ADDITIVE to global limits. Both must pass.
|
|
640
|
+
|
|
641
|
+
<span style="color: #f39c12">**⚡ Important:**</span> Client must pass BOTH global AND route-specific limits.
|
|
642
|
+
|
|
643
|
+
<span style="color: #2ecc71">**✅ Fix:**</span>
|
|
644
|
+
```typescript
|
|
645
|
+
import { YinzerFlow, rateLimitHook } from 'yinzerflow';
|
|
646
|
+
|
|
647
|
+
// Per-route limit is ADDITIVE to global limit
|
|
648
|
+
// Client must pass both global (100/15m) AND route-specific (5/15m) limits
|
|
649
|
+
app.post('/api/auth/login',
|
|
650
|
+
{
|
|
651
|
+
beforeRoute: [rateLimitHook({
|
|
652
|
+
window: '15m',
|
|
653
|
+
max: 5
|
|
654
|
+
})]
|
|
655
|
+
},
|
|
656
|
+
async () => ({ success: true })
|
|
657
|
+
);
|
|
658
|
+
```
|
|
659
|
+
|
|
660
|
+
### High memory usage in production
|
|
661
|
+
|
|
662
|
+
**Symptom:** Memory usage grows over time with many clients.
|
|
663
|
+
|
|
664
|
+
**Cause:** Rate limit data is stored in memory per client.
|
|
665
|
+
|
|
666
|
+
<span style="color: #2ecc71">**✅ Expected:**</span> This is normal behavior. Sliding window counter uses only ~24 bytes per client.
|
|
667
|
+
|
|
668
|
+
<span style="color: #3498db">**💡 Tip:**</span> For distributed systems with millions of clients, use Redis-based storage for distributed rate limiting.
|
|
669
|
+
|
|
670
|
+
```typescript
|
|
671
|
+
// Current memory usage is minimal:
|
|
672
|
+
// 💾 1 million clients = ~24 MB of memory
|
|
673
|
+
// ✅ This is acceptable for most applications
|
|
674
|
+
```
|
|
675
|
+
|
|
676
|
+
## 🔴 Redis Store for Distributed Rate Limiting
|
|
677
|
+
|
|
678
|
+
For production applications with multiple server instances, YinzerFlow supports Redis-based rate limiting storage for distributed rate limiting across your entire infrastructure.
|
|
679
|
+
|
|
680
|
+
### 🚀 Quick Start with Redis
|
|
681
|
+
|
|
682
|
+
```typescript
|
|
683
|
+
import { YinzerFlow } from 'yinzerflow';
|
|
684
|
+
import { RateLimiter } from 'yinzerflow';
|
|
685
|
+
import { createClient } from 'redis';
|
|
686
|
+
|
|
687
|
+
// Create Redis client
|
|
688
|
+
const redis = createClient({
|
|
689
|
+
url: 'redis://localhost:6379'
|
|
690
|
+
});
|
|
691
|
+
await redis.connect();
|
|
692
|
+
|
|
693
|
+
// Create rate limiter with Redis store
|
|
694
|
+
const limiter = new RateLimiter({
|
|
695
|
+
algorithm: 'sliding-window-counter',
|
|
696
|
+
window: '15m',
|
|
697
|
+
max: 100,
|
|
698
|
+
keyGenerator: (ctx) => ctx.request.ipAddress,
|
|
699
|
+
handler: (ctx) => ({
|
|
700
|
+
success: false,
|
|
701
|
+
message: 'Rate limit exceeded'
|
|
702
|
+
})
|
|
703
|
+
}, {
|
|
704
|
+
type: 'redis',
|
|
705
|
+
redis: {
|
|
706
|
+
client: redis,
|
|
707
|
+
keyPrefix: 'myapp:rate_limit:',
|
|
708
|
+
defaultTtl: 3600
|
|
709
|
+
}
|
|
710
|
+
});
|
|
711
|
+
|
|
712
|
+
// Use with YinzerFlow
|
|
713
|
+
const app = new YinzerFlow({
|
|
714
|
+
port: 3000,
|
|
715
|
+
rateLimit: { enabled: false } // Handle manually
|
|
716
|
+
});
|
|
717
|
+
|
|
718
|
+
app.beforeAll(async (ctx) => {
|
|
719
|
+
const result = limiter.check(ctx);
|
|
720
|
+
if (!result.allowed) {
|
|
721
|
+
ctx.response.setStatusCode(429);
|
|
722
|
+
return limiter.config.handler(ctx);
|
|
723
|
+
}
|
|
724
|
+
});
|
|
725
|
+
```
|
|
726
|
+
|
|
727
|
+
### 🐉 DragonflyDB Alternative
|
|
728
|
+
|
|
729
|
+
<span style="color: #3498db">**💡 Tip:**</span> Consider using [DragonflyDB](https://www.dragonflydb.io/) as a Redis alternative. DragonflyDB is a modern, high-performance in-memory database that's Redis-compatible but offers better performance through parallel request processing and lower memory usage.
|
|
730
|
+
|
|
731
|
+
```typescript
|
|
732
|
+
// Works with the same Redis clients
|
|
733
|
+
const redis = createClient({
|
|
734
|
+
url: 'redis://localhost:6379' // DragonflyDB uses same protocol
|
|
735
|
+
});
|
|
736
|
+
```
|
|
737
|
+
|
|
738
|
+
### 📋 Features
|
|
739
|
+
|
|
740
|
+
- **🔄 Distributed**: Works across multiple server instances
|
|
741
|
+
- **⚡ High Performance**: Redis/DragonflyDB-optimized for speed
|
|
742
|
+
- **🛡️ Automatic Expiration**: Keys expire automatically to prevent memory leaks
|
|
743
|
+
- **🔧 Algorithm Agnostic**: Works with any rate limiting algorithm
|
|
744
|
+
- **📊 JSON Serialization**: Handles complex data structures
|
|
745
|
+
- **🚨 Error Handling**: Graceful fallback on connection errors
|
|
746
|
+
|
|
747
|
+
### ⚙️ Configuration
|
|
748
|
+
|
|
749
|
+
```typescript
|
|
750
|
+
interface RedisStoreConfig {
|
|
751
|
+
client: RedisClient; // Redis client instance (required)
|
|
752
|
+
keyPrefix?: string; // Key prefix (default: 'rate_limit:')
|
|
753
|
+
defaultTtl?: number; // Default TTL in seconds (default: 3600)
|
|
754
|
+
debug?: boolean; // Enable debug logging (default: false)
|
|
755
|
+
}
|
|
756
|
+
```
|
|
757
|
+
|
|
758
|
+
### 🔧 Usage Examples
|
|
759
|
+
|
|
760
|
+
#### User-Based Rate Limiting
|
|
761
|
+
|
|
762
|
+
```typescript
|
|
763
|
+
const userLimiter = new RateLimiter({
|
|
764
|
+
algorithm: 'sliding-window-counter',
|
|
765
|
+
window: '1h',
|
|
766
|
+
max: 10000,
|
|
767
|
+
keyGenerator: (ctx) => {
|
|
768
|
+
// Extract user ID from JWT, session, etc.
|
|
769
|
+
const userId = ctx.request.headers['x-user-id'] || 'anonymous';
|
|
770
|
+
return `user:${userId}`;
|
|
771
|
+
},
|
|
772
|
+
handler: (ctx) => ({
|
|
773
|
+
success: false,
|
|
774
|
+
message: 'User rate limit exceeded'
|
|
775
|
+
})
|
|
776
|
+
}, {
|
|
777
|
+
type: 'redis',
|
|
778
|
+
redis: {
|
|
779
|
+
client: redis,
|
|
780
|
+
keyPrefix: 'myapp:user_limit:',
|
|
781
|
+
defaultTtl: 7200 // 2 hours
|
|
782
|
+
}
|
|
783
|
+
});
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
### 🛡️ Security Considerations
|
|
787
|
+
|
|
788
|
+
- **Key Prefixing**: Use unique prefixes (`myapp:rate_limit:`) to avoid conflicts
|
|
789
|
+
- **TTL Configuration**: Set TTL longer than your rate limit windows (1-hour TTL for 15-minute windows)
|
|
790
|
+
|
|
791
|
+
### 📊 Performance
|
|
792
|
+
|
|
793
|
+
- **Memory**: ~50-100 bytes per IP including Redis overhead
|
|
794
|
+
- **Latency**: ~0.1-0.5ms local, ~1-10ms remote
|
|
795
|
+
- **Throughput**: 100k+ operations/second with single Redis/DragonflyDB
|