@madeinoz67/voice-server 0.1.10 → 0.1.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/codanna-explorer.md +30 -0
- package/.claude/agents/codanna-implementer.md +39 -0
- package/.claude/hooks/codanna-nudge.ts +100 -0
- package/.claude/hooks/codanna-reindex.ts +38 -0
- package/.claude/settings.json +67 -0
- package/.claude/settings.local.json +4 -1
- package/.codanna/settings.toml +71 -41
- package/.mcp.json +8 -0
- package/.orca/codanna-archive.ts +41 -0
- package/.orca/codanna-setup.ts +101 -0
- package/CHANGELOG.md +43 -65
- package/CLAUDE.md +51 -1
- package/Formula/madeinoz-voice-server.rb +1 -1
- package/cliff.toml +10 -5
- package/docs/assets/queue-system-overview-v2.png +0 -0
- package/docs/assets/voice-server-architecture-backup.png +0 -0
- package/homebrew.mxcl.voice-server.plist +33 -0
- package/orca.yaml +21 -0
- package/package.json +1 -1
- package/reports/agent/2025-02-09-12-00-fifo-queue-research.md +380 -0
- package/reports/agent/2026-02-09-queue-rate-limit-integration.md +250 -0
- package/reports/agent/2026-02-09-queued-async-error-handling.md +236 -0
- package/src/ts/services/notification-queue.ts +15 -12
- package/bun.lock +0 -208
|
@@ -0,0 +1,380 @@
|
|
|
1
|
+
# Research Report: FIFO Queue Implementation for HTTP Request Queuing in Bun/TypeScript
|
|
2
|
+
|
|
3
|
+
**Date**: 2025-02-09 12:00
|
|
4
|
+
**Agent**: Research-Agent
|
|
5
|
+
**Model**: Claude 4.7
|
|
6
|
+
**Context**: Voice server TTS request queuing with 201 Accepted pattern, max 100 items depth
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Executive Summary
|
|
11
|
+
|
|
12
|
+
After comprehensive analysis of queue implementation patterns for Bun/TypeScript and the specific requirements of the voice server (sequential TTS processing, HTTP async pattern, graceful shutdown), the **recommended approach is a simple array-based queue with async/await producer-consumer pattern and Atomics for thread safety**.
|
|
13
|
+
|
|
14
|
+
This approach prioritizes simplicity, Bun-specific optimizations, and the single-process nature of the current application.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Decision
|
|
19
|
+
|
|
20
|
+
**Chosen Approach**: Array-based FIFO queue with:
|
|
21
|
+
- TypeScript array as the underlying data structure
|
|
22
|
+
- Async/await for producer-consumer coordination
|
|
23
|
+
- Promises for queue slot availability signaling
|
|
24
|
+
- Atomic operations for queue depth tracking (optional for single-process)
|
|
25
|
+
- Built-in graceful shutdown with queue draining
|
|
26
|
+
|
|
27
|
+
**Rationale**:
|
|
28
|
+
1. **Simplicity**: Array-based queues are straightforward to implement, debug, and maintain
|
|
29
|
+
2. **Bun Optimization**: Bun's runtime optimizes async operations and Promise handling
|
|
30
|
+
3. **Single-Process Architecture**: Current implementation is single-process (no workers), eliminating need for shared memory
|
|
31
|
+
4. **Sufficient Performance**: For max 100 queue depth and sequential TTS processing, array operations are more than adequate
|
|
32
|
+
5. **Type Safety**: TypeScript provides excellent type safety for queue items
|
|
33
|
+
6. **Graceful Shutdown**: Simple to implement draining logic with async/await patterns
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## Alternatives Considered
|
|
38
|
+
|
|
39
|
+
### 1. Linked List Implementation
|
|
40
|
+
**Evaluated**: Custom linked list with Node objects
|
|
41
|
+
**Rejected Because**:
|
|
42
|
+
- Higher memory overhead per item (additional object references)
|
|
43
|
+
- No performance benefit for small queue sizes (100 items)
|
|
44
|
+
- More complex implementation with potential for memory leaks
|
|
45
|
+
- Poor cache locality compared to arrays
|
|
46
|
+
- TypeScript requires careful type definitions
|
|
47
|
+
|
|
48
|
+
### 2. Atomics/Shared Memory
|
|
49
|
+
**Evaluated**: SharedArrayBuffer with Atomics for inter-process communication
|
|
50
|
+
**Rejected Because**:
|
|
51
|
+
- Current architecture is single-process (no workers/threads)
|
|
52
|
+
- Adds complexity without benefit for non-concurrent processing
|
|
53
|
+
- Requires special HTTP headers (Cross-Origin-Opener-Policy)
|
|
54
|
+
- Overkill for sequential TTS processing
|
|
55
|
+
- Security considerations with shared memory
|
|
56
|
+
|
|
57
|
+
### 3. Bull/BullMQ (Redis-based)
|
|
58
|
+
**Evaluated**: Industry-standard Redis-backed job queues
|
|
59
|
+
**Rejected Because**:
|
|
60
|
+
- External dependency on Redis (adds infrastructure complexity)
|
|
61
|
+
- Overkill for single-server, in-memory queuing
|
|
62
|
+
- Network latency for queue operations
|
|
63
|
+
- Additional operational overhead
|
|
64
|
+
- Current requirements don't need persistence or distributed processing
|
|
65
|
+
|
|
66
|
+
### 4. Bun Workers with Message Channels
|
|
67
|
+
**Evaluated**: Separate worker process with structured clone for queue items
|
|
68
|
+
**Rejected Because**:
|
|
69
|
+
- Current TTS processing is already async (MLX-audio subprocess)
|
|
70
|
+
- Adding workers would complicate the architecture
|
|
71
|
+
- Structured cloning overhead for queue items
|
|
72
|
+
- No CPU-bound work to offload (I/O bound TTS operations)
|
|
73
|
+
- Would require refactoring existing MLX client integration
|
|
74
|
+
|
|
75
|
+
### 5. EventEmitter-based Queue
|
|
76
|
+
**Evaluated**: Node.js EventEmitter for queue state changes
|
|
77
|
+
**Rejected Because**:
|
|
78
|
+
- Less explicit than Promise-based coordination
|
|
79
|
+
- Harder to reason about async flow
|
|
80
|
+
- Potential for unhandled rejections
|
|
81
|
+
- Promises are more idiomatic for async/await patterns
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## Technical Implementation Details
|
|
86
|
+
|
|
87
|
+
### Queue Data Structure
|
|
88
|
+
|
|
89
|
+
```typescript
|
|
90
|
+
interface QueuedRequest {
|
|
91
|
+
id: string;
|
|
92
|
+
request: NotificationRequest | PaiNotificationRequest;
|
|
93
|
+
resolve: (response: SuccessResponse | ErrorResponse) => void;
|
|
94
|
+
reject: (error: Error) => void;
|
|
95
|
+
timestamp: number;
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
class RequestQueue {
|
|
99
|
+
private queue: QueuedRequest[] = [];
|
|
100
|
+
private processing = false;
|
|
101
|
+
private maxDepth: number;
|
|
102
|
+
|
|
103
|
+
constructor(maxDepth: number = 100) {
|
|
104
|
+
this.maxDepth = maxDepth;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
async enqueue(item: QueuedRequest): Promise<void> {
|
|
108
|
+
if (this.queue.length >= this.maxDepth) {
|
|
109
|
+
throw new Error("Queue depth exceeded");
|
|
110
|
+
}
|
|
111
|
+
this.queue.push(item);
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
dequeue(): QueuedRequest | undefined {
|
|
115
|
+
return this.queue.shift();
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
get size(): number {
|
|
119
|
+
return this.queue.length;
|
|
120
|
+
}
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### Producer-Consumer Pattern
|
|
125
|
+
|
|
126
|
+
**Producer (HTTP Handler)**:
|
|
127
|
+
```typescript
|
|
128
|
+
// In server.ts fetch handler
|
|
129
|
+
if (path === "/notify" && req.method === "POST") {
|
|
130
|
+
const body = await parseJsonBody<NotificationRequest>(req);
|
|
131
|
+
|
|
132
|
+
// Return 202 Accepted immediately
|
|
133
|
+
const responsePromise = requestQueue.enqueue({
|
|
134
|
+
id: generateId(),
|
|
135
|
+
request: body,
|
|
136
|
+
timestamp: Date.now()
|
|
137
|
+
});
|
|
138
|
+
|
|
139
|
+
return Response.json(
|
|
140
|
+
{ status: "accepted", message: "Request queued" },
|
|
141
|
+
{ status: 202 }
|
|
142
|
+
);
|
|
143
|
+
}
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
**Consumer (Background Processor)**:
|
|
147
|
+
```typescript
|
|
148
|
+
async function processQueue(): Promise<void> {
|
|
149
|
+
while (true) {
|
|
150
|
+
const item = requestQueue.dequeue();
|
|
151
|
+
if (!item) {
|
|
152
|
+
await sleep(50); // Avoid busy-wait
|
|
153
|
+
continue;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
try {
|
|
157
|
+
const response = await handleNotify(item.request);
|
|
158
|
+
item.resolve(response);
|
|
159
|
+
} catch (error) {
|
|
160
|
+
item.reject(error as Error);
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
### Graceful Shutdown
|
|
167
|
+
|
|
168
|
+
```typescript
|
|
169
|
+
interface ServerState {
|
|
170
|
+
requestQueue: RequestQueue;
|
|
171
|
+
shutdownRequested: boolean;
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
const shutdown = async () => {
|
|
175
|
+
logger.info("Shutting down server...");
|
|
176
|
+
serverState.shutdownRequested = true;
|
|
177
|
+
|
|
178
|
+
// Stop accepting new requests
|
|
179
|
+
serverState.requestQueue.close();
|
|
180
|
+
|
|
181
|
+
// Wait for queue to drain (with timeout)
|
|
182
|
+
const drainTimeout = 30000; // 30 seconds
|
|
183
|
+
const startTime = Date.now();
|
|
184
|
+
|
|
185
|
+
while (serverState.requestQueue.size > 0) {
|
|
186
|
+
if (Date.now() - startTime > drainTimeout) {
|
|
187
|
+
logger.warn("Shutdown timeout - forcing exit");
|
|
188
|
+
break;
|
|
189
|
+
}
|
|
190
|
+
await sleep(100);
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
server.stop();
|
|
194
|
+
process.exit(0);
|
|
195
|
+
};
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## Concurrency Patterns Analysis
|
|
201
|
+
|
|
202
|
+
### Bun-Specific Considerations
|
|
203
|
+
|
|
204
|
+
1. **Async I/O Optimization**:
|
|
205
|
+
- Bun's event loop is optimized for async operations
|
|
206
|
+
- Array operations with Promises leverage this optimization
|
|
207
|
+
|
|
208
|
+
2. **Worker_pool**:
|
|
209
|
+
- Bun has built-in worker pool for CPU tasks
|
|
210
|
+
- Not needed for I/O-bound TTS operations
|
|
211
|
+
|
|
212
|
+
3. **Atomic Operations**:
|
|
213
|
+
- Bun supports Atomics API
|
|
214
|
+
- Use only if implementing multi-consumer queue
|
|
215
|
+
- For single-consumer (current use case), not necessary
|
|
216
|
+
|
|
217
|
+
### Recommended Concurrency Pattern
|
|
218
|
+
|
|
219
|
+
**Single Consumer, Multiple Producers**:
|
|
220
|
+
- Multiple HTTP requests can add to queue simultaneously
|
|
221
|
+
- Single background worker processes sequentially
|
|
222
|
+
- Ensures TTS operations don't overlap (resource constraint)
|
|
223
|
+
- Matches MLX-audio's single-stream limitation
|
|
224
|
+
|
|
225
|
+
---
|
|
226
|
+
|
|
227
|
+
## Performance Considerations
|
|
228
|
+
|
|
229
|
+
### Array Performance (Max 100 items)
|
|
230
|
+
|
|
231
|
+
- **Enqueue**: O(1) amortized (array push)
|
|
232
|
+
- **Dequeue**: O(n) worst case (array shift) - negligible for n=100
|
|
233
|
+
- **Memory**: ~100 item references (negligible)
|
|
234
|
+
- **Cache locality**: Excellent (contiguous memory)
|
|
235
|
+
|
|
236
|
+
### Optimization: Circular Buffer (Optional)
|
|
237
|
+
|
|
238
|
+
If performance becomes critical:
|
|
239
|
+
```typescript
|
|
240
|
+
class CircularBuffer<T> {
|
|
241
|
+
private buffer: (T | undefined)[];
|
|
242
|
+
private head = 0;
|
|
243
|
+
private tail = 0;
|
|
244
|
+
private count = 0;
|
|
245
|
+
|
|
246
|
+
constructor(private capacity: number) {
|
|
247
|
+
this.buffer = new Array(capacity);
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
enqueue(item: T): boolean {
|
|
251
|
+
if (this.count >= this.capacity) return false;
|
|
252
|
+
this.buffer[this.tail] = item;
|
|
253
|
+
this.tail = (this.tail + 1) % this.capacity;
|
|
254
|
+
this.count++;
|
|
255
|
+
return true;
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
dequeue(): T | undefined {
|
|
259
|
+
if (this.count === 0) return undefined;
|
|
260
|
+
const item = this.buffer[this.head];
|
|
261
|
+
this.buffer[this.head] = undefined;
|
|
262
|
+
this.head = (this.head + 1) % this.capacity;
|
|
263
|
+
this.count--;
|
|
264
|
+
return item;
|
|
265
|
+
}
|
|
266
|
+
}
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
**Recommendation**: Start with simple array, optimize only if profiling shows need.
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## Implementation Checklist
|
|
274
|
+
|
|
275
|
+
### Core Features
|
|
276
|
+
- [ ] Array-based queue implementation
|
|
277
|
+
- [ ] Enqueue with max depth check (100 items)
|
|
278
|
+
- [ ] Dequeue for background processor
|
|
279
|
+
- [ ] Promise-based response coordination
|
|
280
|
+
- [ ] Request ID generation for tracking
|
|
281
|
+
|
|
282
|
+
### HTTP Integration
|
|
283
|
+
- [ ] Modify server.ts to return 202 Accepted
|
|
284
|
+
- [ ] Add queue endpoint (optional: GET /queue/status)
|
|
285
|
+
- [ ] Error responses for queue full (503 Service Unavailable)
|
|
286
|
+
|
|
287
|
+
### Background Processor
|
|
288
|
+
- [ ] Async function to process queue sequentially
|
|
289
|
+
- [ ] Integration with existing processTTS()
|
|
290
|
+
- [ ] Error handling and logging per item
|
|
291
|
+
|
|
292
|
+
### Graceful Shutdown
|
|
293
|
+
- [ ] Close queue on shutdown signal
|
|
294
|
+
- [ ] Wait for queue drain (with timeout)
|
|
295
|
+
- [ ] Log remaining items on timeout
|
|
296
|
+
|
|
297
|
+
### Monitoring (Optional)
|
|
298
|
+
- [ ] Queue depth metric
|
|
299
|
+
- [ ] Processing time per item
|
|
300
|
+
- [ ] Average wait time
|
|
301
|
+
- [ ] Rejection count (queue full)
|
|
302
|
+
|
|
303
|
+
---
|
|
304
|
+
|
|
305
|
+
## Testing Strategy
|
|
306
|
+
|
|
307
|
+
### Unit Tests
|
|
308
|
+
```typescript
|
|
309
|
+
describe('RequestQueue', () => {
|
|
310
|
+
test('enqueues items up to max depth', async () => {
|
|
311
|
+
const queue = new RequestQueue(2);
|
|
312
|
+
await queue.enqueue(mockItem);
|
|
313
|
+
await queue.enqueue(mockItem);
|
|
314
|
+
await expect(queue.enqueue(mockItem)).rejects.toThrow();
|
|
315
|
+
});
|
|
316
|
+
|
|
317
|
+
test('dequeues items in FIFO order', async () => {
|
|
318
|
+
const queue = new RequestQueue();
|
|
319
|
+
await queue.enqueue(item1);
|
|
320
|
+
await queue.enqueue(item2);
|
|
321
|
+
expect(queue.dequeue()).toBe(item1);
|
|
322
|
+
expect(queue.dequeue()).toBe(item2);
|
|
323
|
+
});
|
|
324
|
+
});
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
### Integration Tests
|
|
328
|
+
- Test HTTP 202 Accepted response
|
|
329
|
+
- Test queue full scenario (503 response)
|
|
330
|
+
- Test sequential processing
|
|
331
|
+
- Test graceful shutdown with draining
|
|
332
|
+
|
|
333
|
+
---
|
|
334
|
+
|
|
335
|
+
## Monitoring and Observability
|
|
336
|
+
|
|
337
|
+
### Metrics to Track
|
|
338
|
+
1. **Queue Depth**: Current number of queued requests
|
|
339
|
+
2. **Processing Rate**: Requests per second
|
|
340
|
+
3. **Wait Time**: Time from enqueue to processing start
|
|
341
|
+
4. **Rejection Rate**: Requests rejected due to full queue
|
|
342
|
+
|
|
343
|
+
### Logging Structure
|
|
344
|
+
```typescript
|
|
345
|
+
logger.info("Queue status", {
|
|
346
|
+
depth: queue.size,
|
|
347
|
+
maxDepth: MAX_DEPTH,
|
|
348
|
+
processing: queue.isProcessing,
|
|
349
|
+
avgWaitTime: calculateAverageWaitTime(),
|
|
350
|
+
});
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
---
|
|
354
|
+
|
|
355
|
+
## Conclusion
|
|
356
|
+
|
|
357
|
+
For the voice server's TTS request queuing requirements, an **array-based FIFO queue with async/await producer-consumer pattern** provides the optimal balance of simplicity, performance, and maintainability. The single-process architecture, sequential processing requirements, and small queue size (100 items) make this approach ideal.
|
|
358
|
+
|
|
359
|
+
The implementation should start with a simple array-based queue and add complexity (circular buffer, metrics, etc.) only if profiling demonstrates need. The graceful shutdown pattern ensures no requests are lost during deployments or restarts.
|
|
360
|
+
|
|
361
|
+
---
|
|
362
|
+
|
|
363
|
+
## References
|
|
364
|
+
|
|
365
|
+
### Files Examined
|
|
366
|
+
- `src/ts/server.ts:1-500` - Current HTTP server implementation
|
|
367
|
+
|
|
368
|
+
### Key Patterns Identified
|
|
369
|
+
1. Current implementation uses `Bun.serve()` with async fetch handler
|
|
370
|
+
2. TTS processing is already async via MLX-audio subprocess calls
|
|
371
|
+
3. Single-process architecture (no workers/threads)
|
|
372
|
+
4. Existing graceful shutdown with SIGTERM/SIGINT handling
|
|
373
|
+
|
|
374
|
+
### Recommended Next Steps
|
|
375
|
+
1. Implement `RequestQueue` class in `src/ts/utils/request-queue.ts`
|
|
376
|
+
2. Add background processor function
|
|
377
|
+
3. Modify server.ts to use queue for `/notify` and `/pai` endpoints
|
|
378
|
+
4. Implement graceful shutdown draining
|
|
379
|
+
5. Add unit tests for queue operations
|
|
380
|
+
6. (Optional) Add metrics endpoint for queue monitoring
|
|
@@ -0,0 +1,250 @@
|
|
|
1
|
+
# Research Report: FIFO Queue Integration with Rate Limiting Middleware
|
|
2
|
+
|
|
3
|
+
**Date**: 2026-02-09
|
|
4
|
+
**Agent**: Research-Agent-v5
|
|
5
|
+
**Model**: Claude Opus 4.6
|
|
6
|
+
|
|
7
|
+
## Summary
|
|
8
|
+
|
|
9
|
+
This report analyzes how the proposed FIFO notification queue should integrate with the existing rate limiting middleware in the voice-server. The key finding is that rate limiting and queue capacity are distinct concerns that should operate at different layers: rate limiting controls client request frequency, while queue capacity manages server processing capacity.
|
|
10
|
+
|
|
11
|
+
## Key Findings
|
|
12
|
+
|
|
13
|
+
### 1. Current Rate Limiter Implementation
|
|
14
|
+
|
|
15
|
+
**Location**: `src/ts/middleware/rate-limiter.ts:1-115`
|
|
16
|
+
|
|
17
|
+
The existing rate limiter uses a token bucket algorithm with the following characteristics:
|
|
18
|
+
- Default: 60 requests per minute per client
|
|
19
|
+
- Client identification via X-Forwarded-For, X-Real-IP, or fallback to localhost
|
|
20
|
+
- Returns 429 with message "Rate limit exceeded" when limit is hit
|
|
21
|
+
- NO Retry-After header currently implemented
|
|
22
|
+
- Cleanup runs every 60 seconds
|
|
23
|
+
|
|
24
|
+
**Current 429 Response Pattern**:
|
|
25
|
+
```typescript
|
|
26
|
+
return Response.json(
|
|
27
|
+
errorResponse("Rate limit exceeded"),
|
|
28
|
+
{ status: 429 }
|
|
29
|
+
);
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
### 2. Proposed FIFO Queue Specification
|
|
33
|
+
|
|
34
|
+
**Location**: `specs/001-fifo-notify-queue/spec.md`
|
|
35
|
+
|
|
36
|
+
Key requirements:
|
|
37
|
+
- Maximum queue depth: 100 items (FR-005)
|
|
38
|
+
- Returns 429 when queue at capacity (FR-006)
|
|
39
|
+
- Immediate 201 Accepted for valid requests (FR-002)
|
|
40
|
+
- Schema validation BEFORE queuing (FR-013)
|
|
41
|
+
- Average audio duration: 3-10 seconds
|
|
42
|
+
|
|
43
|
+
### 3. Architectural Distinction
|
|
44
|
+
|
|
45
|
+
Rate limiting and queue capacity serve different purposes:
|
|
46
|
+
|
|
47
|
+
| Aspect | Rate Limiter | Queue Capacity |
|
|
48
|
+
|--------|-------------|----------------|
|
|
49
|
+
| **Purpose** | Prevent client abuse | Manage server processing load |
|
|
50
|
+
| **Scope** | Per-client (identified by IP) | Global (server-wide) |
|
|
51
|
+
| **Time Window** | Sliding window (60s) | Until processed |
|
|
52
|
+
| **Error Meaning** | "You're sending too fast" | "Server is too busy" |
|
|
53
|
+
| **Retry Strategy** | Wait for window to expire | Wait for queue to drain |
|
|
54
|
+
|
|
55
|
+
## Decision Framework
|
|
56
|
+
|
|
57
|
+
### Decision 1: Rate Limiting Before or After Queuing?
|
|
58
|
+
|
|
59
|
+
**Decision**: Rate limiting should apply BEFORE queuing.
|
|
60
|
+
|
|
61
|
+
**Rationale**:
|
|
62
|
+
1. **Client protection first**: If a client is abusing the API, they shouldn't consume queue space
|
|
63
|
+
2. **Queue preservation**: The queue is a shared resource - abusive clients shouldn't fill it
|
|
64
|
+
3. **Existing pattern**: Current rate limiter runs at the middleware layer before request handlers (`src/ts/server.ts:387-395`)
|
|
65
|
+
4. **Clear error semantics**: Distinguishes between "you're being rate limited" vs "queue is full"
|
|
66
|
+
|
|
67
|
+
**Evidence**: `src/ts/server.ts:387-395`
|
|
68
|
+
```typescript
|
|
69
|
+
// Rate limiting (apply to POST requests)
|
|
70
|
+
if (req.method === "POST") {
|
|
71
|
+
const clientId = extractClientId(req);
|
|
72
|
+
if (serverState.rateLimiter.isRateLimited(clientId)) {
|
|
73
|
+
return Response.json(
|
|
74
|
+
errorResponse("Rate limit exceeded"),
|
|
75
|
+
{ status: 429 }
|
|
76
|
+
);
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
### Decision 2: 429 Response Pattern Distinction
|
|
82
|
+
|
|
83
|
+
**Decision**: Use distinct error messages to differentiate rate limiting from queue overflow.
|
|
84
|
+
|
|
85
|
+
**Rationale**:
|
|
86
|
+
- Rate limit 429: "Rate limit exceeded" (client issue)
|
|
87
|
+
- Queue full 429: "Queue at capacity" (server issue)
|
|
88
|
+
- This allows clients to implement different retry strategies
|
|
89
|
+
- Operators can distinguish between abuse vs high load
|
|
90
|
+
|
|
91
|
+
**Recommended Implementation**:
|
|
92
|
+
```typescript
|
|
93
|
+
// Rate limit 429
|
|
94
|
+
{
|
|
95
|
+
status: "error",
|
|
96
|
+
message: "Rate limit exceeded",
|
|
97
|
+
retry_after: <seconds_until_window_reset>
|
|
98
|
+
}
|
|
99
|
+
|
|
100
|
+
// Queue full 429
|
|
101
|
+
{
|
|
102
|
+
status: "error",
|
|
103
|
+
message: "Queue at capacity",
|
|
104
|
+
retry_after: <estimated_seconds_until_queue_drains>
|
|
105
|
+
}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
### Decision 3: Retry-After Header Strategy
|
|
109
|
+
|
|
110
|
+
**Decision**: Implement Retry-After header for both scenarios with different calculation methods.
|
|
111
|
+
|
|
112
|
+
**Rate Limiter Retry-After**:
|
|
113
|
+
- Calculate seconds until client's rate limit window resets
|
|
114
|
+
- Use `getResetTime(clientId)` from existing rate limiter
|
|
115
|
+
- Example: `Math.ceil((resetTime - now) / 1000)`
|
|
116
|
+
|
|
117
|
+
**Queue Full Retry-After**:
|
|
118
|
+
- Estimate based on average processing time (3-10s per item)
|
|
119
|
+
- Formula: `queue_depth * avg_processing_time`
|
|
120
|
+
- Add buffer for safety (e.g., +20%)
|
|
121
|
+
- Example: `queue.size * 8 + 2` seconds
|
|
122
|
+
|
|
123
|
+
**Evidence**: Rate limiter already tracks reset time (`src/ts/middleware/rate-limiter.ts:68-74`):
|
|
124
|
+
```typescript
|
|
125
|
+
getResetTime(clientId: string): number | null {
|
|
126
|
+
const entry = this.clients.get(clientId);
|
|
127
|
+
return entry ? entry.resetTime : null;
|
|
128
|
+
}
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
### Decision 4: Middleware Ordering
|
|
132
|
+
|
|
133
|
+
**Decision**: Establish clear middleware pipeline order.
|
|
134
|
+
|
|
135
|
+
**Recommended Pipeline**:
|
|
136
|
+
1. **CORS preflight** (if OPTIONS request)
|
|
137
|
+
2. **Rate limiting** (per-client request frequency)
|
|
138
|
+
3. **Schema validation** (400 if invalid)
|
|
139
|
+
4. **Queue capacity check** (429 if full)
|
|
140
|
+
5. **Queue enqueue** (returns 201 Accepted)
|
|
141
|
+
6. **Background processing** (non-blocking)
|
|
142
|
+
|
|
143
|
+
**Visual Flow**:
|
|
144
|
+
```
|
|
145
|
+
Request → CORS → Rate Limit → Validate → Queue Check → Enqueue → 201 Accepted
|
|
146
|
+
↓ ↓ ↓ ↓ ↓
|
|
147
|
+
204 No 429 Rate 400 Bad 429 Queue Background
|
|
148
|
+
Content Limited Request Full Processing
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
## Alternatives Considered
|
|
152
|
+
|
|
153
|
+
### Alternative 1: Rate Limit After Queuing
|
|
154
|
+
**Description**: Accept all valid requests into queue, rate limit at processing time.
|
|
155
|
+
|
|
156
|
+
**Pros**:
|
|
157
|
+
- Fairer first-come-first-serve
|
|
158
|
+
- Simpler client retry logic
|
|
159
|
+
|
|
160
|
+
**Cons**:
|
|
161
|
+
- Abusive clients fill shared queue
|
|
162
|
+
- Queue becomes weaponizable
|
|
163
|
+
- No protection against queue flooding
|
|
164
|
+
|
|
165
|
+
**Rejected**: Queue capacity is a shared server resource that must be protected from abuse.
|
|
166
|
+
|
|
167
|
+
### Alternative 2: Single 429 Response
|
|
168
|
+
**Description**: Use same "Rate limit exceeded" message for both rate limiting and queue full.
|
|
169
|
+
|
|
170
|
+
**Pros**:
|
|
171
|
+
- Simpler implementation
|
|
172
|
+
- Fewer error cases to handle
|
|
173
|
+
|
|
174
|
+
**Cons**:
|
|
175
|
+
- Clients can't distinguish client vs server issues
|
|
176
|
+
- No way to know appropriate retry strategy
|
|
177
|
+
- Poor observability for operators
|
|
178
|
+
|
|
179
|
+
**Rejected**: Clear error semantics are essential for client behavior and operational visibility.
|
|
180
|
+
|
|
181
|
+
### Alternative 3: 503 Service Unavailable for Queue Full
|
|
182
|
+
**Description**: Use 503 instead of 429 when queue is at capacity.
|
|
183
|
+
|
|
184
|
+
**Pros**:
|
|
185
|
+
- Semantically correct (server unavailable)
|
|
186
|
+
- Standard HTTP practice for overloaded services
|
|
187
|
+
|
|
188
|
+
**Cons**:
|
|
189
|
+
- Spec explicitly requires 429 (FR-006)
|
|
190
|
+
- Clients may treat 503 as downtime rather than retryable
|
|
191
|
+
- Inconsistent with specification
|
|
192
|
+
|
|
193
|
+
**Rejected**: Specification compliance is mandatory.
|
|
194
|
+
|
|
195
|
+
## Conclusions
|
|
196
|
+
|
|
197
|
+
### Recommended Approach
|
|
198
|
+
|
|
199
|
+
1. **Rate limiting operates first** at middleware layer, before queue operations
|
|
200
|
+
2. **Queue capacity check happens after** validation but before enqueue
|
|
201
|
+
3. **Distinct 429 responses** with clear error messages
|
|
202
|
+
4. **Retry-After header** added to both 429 responses with appropriate calculation
|
|
203
|
+
5. **Queue status endpoint** provides visibility into current load
|
|
204
|
+
|
|
205
|
+
### Implementation Priority
|
|
206
|
+
|
|
207
|
+
1. **P0**: Implement distinct 429 responses with error message differentiation
|
|
208
|
+
2. **P0**: Add Retry-After header to rate limiter 429 responses
|
|
209
|
+
3. **P0**: Add Retry-After header to queue full 429 responses
|
|
210
|
+
4. **P1**: Update ErrorResponse type to include optional retry_after field
|
|
211
|
+
5. **P2**: Add queue capacity metrics to /health endpoint
|
|
212
|
+
|
|
213
|
+
### Code Changes Required
|
|
214
|
+
|
|
215
|
+
1. **Update ErrorResponse type** (`src/ts/models/notification.ts:43-46`):
|
|
216
|
+
```typescript
|
|
217
|
+
export interface ErrorResponse {
|
|
218
|
+
status: "error";
|
|
219
|
+
message: string;
|
|
220
|
+
retry_after?: number; // seconds
|
|
221
|
+
}
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
2. **Update rate limiter** to return Retry-After header
|
|
225
|
+
3. **Update queue middleware** to return distinct 429 with Retry-After
|
|
226
|
+
4. **Update server.ts** to use new error response format
|
|
227
|
+
|
|
228
|
+
### Testing Strategy
|
|
229
|
+
|
|
230
|
+
1. Test rate limiting with concurrent requests
|
|
231
|
+
2. Test queue full scenario with burst > 100 requests
|
|
232
|
+
3. Verify Retry-After header accuracy
|
|
233
|
+
4. Test pipeline ordering (rate limit → validate → queue)
|
|
234
|
+
5. Verify distinct error messages for each failure mode
|
|
235
|
+
|
|
236
|
+
### Operational Considerations
|
|
237
|
+
|
|
238
|
+
- Rate limiter protects against abusive clients
|
|
239
|
+
- Queue capacity prevents server overload
|
|
240
|
+
- Both work together but serve different purposes
|
|
241
|
+
- Operators should monitor both metrics independently
|
|
242
|
+
- Client retry behavior differs between scenarios
|
|
243
|
+
|
|
244
|
+
---
|
|
245
|
+
|
|
246
|
+
**References**:
|
|
247
|
+
- Rate Limiter Implementation: `src/ts/middleware/rate-limiter.ts`
|
|
248
|
+
- Server Request Handler: `src/ts/server.ts:387-395`
|
|
249
|
+
- FIFO Queue Specification: `specs/001-fifo-notify-queue/spec.md`
|
|
250
|
+
- Notification Models: `src/ts/models/notification.ts`
|