@madeinoz67/voice-server 0.1.11 → 0.1.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/codanna-explorer.md +30 -0
- package/.claude/agents/codanna-implementer.md +39 -0
- package/.claude/hooks/codanna-nudge.ts +100 -0
- package/.claude/hooks/codanna-reindex.ts +38 -0
- package/.claude/settings.json +67 -0
- package/.claude/settings.local.json +4 -1
- package/.codanna/settings.toml +71 -41
- package/.mcp.json +8 -0
- package/.orca/codanna-archive.ts +41 -0
- package/.orca/codanna-setup.ts +101 -0
- package/CHANGELOG.md +48 -1
- package/CLAUDE.md +51 -1
- package/Formula/madeinoz-voice-server.rb +1 -1
- package/docs/assets/queue-system-overview-v2.png +0 -0
- package/docs/assets/voice-server-architecture-backup.png +0 -0
- package/orca.yaml +21 -0
- package/package.json +1 -1
- package/reports/agent/2025-02-09-12-00-fifo-queue-research.md +380 -0
- package/reports/agent/2026-02-09-queue-rate-limit-integration.md +250 -0
- package/reports/agent/2026-02-09-queued-async-error-handling.md +236 -0
- package/src/ts/services/notification-queue.ts +15 -12
- package/bun.lock +0 -208
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,53 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
|
5
5
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
## [Unreleased]
|
|
7
7
|
|
|
8
|
+
### Bug Fixes
|
|
9
|
+
|
|
10
|
+
- Allow empty/missing voice_id in notifications ([2479428](247942826cc9262dc1402b8e299c02bb5275c94b))
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
### Documentation
|
|
14
|
+
|
|
15
|
+
- Add queue diagrams and agent reports ([d4be95d](d4be95df90e396d2bf182e1b900da289049da1eb))
|
|
16
|
+
|
|
17
|
+
- Ban `codanna index .` in agent guidance ([2650a51](2650a51c8c0799deb48b83fa47e54ad6fb76cf12))
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
### Miscellaneous Tasks
|
|
21
|
+
|
|
22
|
+
- Bump eslint from 10.0.0 to 10.0.1 ([ff97687](ff97687b3acb15158be44dd21e449a36b18515c4))
|
|
23
|
+
|
|
24
|
+
- Bump @types/node from 25.2.3 to 25.3.0 ([b930f7b](b930f7bc5a788295b73c09430e291d3ffc218a94))
|
|
25
|
+
|
|
26
|
+
- Bump @typescript-eslint/parser from 8.56.0 to 8.56.1 ([f8f2c69](f8f2c6904395f213990a61f5bbc9a074d3171531))
|
|
27
|
+
|
|
28
|
+
- Add GitHub MCP permissions to local settings ([1641ec3](1641ec3581e763149da2035b3ca91c5f599dcba6))
|
|
29
|
+
|
|
30
|
+
- Prune dead python paths, drop stale pyproject ref, rebuild index for 0.17.1 ([9e1497f](9e1497f8804ed33d36449051994b3fef8ca0eb71))
|
|
31
|
+
|
|
32
|
+
- Install hooks, agents, MCP binding, orca setup, worktree-safe config ([d6d3cf8](d6d3cf8114dd0c34dd44d01d0bc48226cb8bde63))
|
|
33
|
+
|
|
34
|
+
- Bump flatted ([9ea0207](9ea0207f25c59d8561381a7d59f689a77cd873b1))
|
|
35
|
+
|
|
36
|
+
- Bump @types/node from 25.3.0 to 25.6.0 ([b9a5f72](b9a5f72ffada0f0a00760292d721596f97640900))
|
|
37
|
+
|
|
38
|
+
- Bump eslint from 10.0.1 to 10.2.0 ([a18acb9](a18acb9b2ce93acdd88ebbe0d00d62f931bb6c06))
|
|
39
|
+
|
|
40
|
+
- Bump @typescript-eslint/eslint-plugin ([e46bc7d](e46bc7db99c423a1a416d8799cea3e55beb62ea6))
|
|
41
|
+
|
|
42
|
+
- Bump typescript-eslint from 8.56.0 to 8.58.1 ([ec2887e](ec2887e2f1bb10c90536b9713ce8c49d457bd435))
|
|
43
|
+
|
|
44
|
+
- Bump bun-types from 1.3.9 to 1.3.12 ([75cdd0b](75cdd0b53b84bbade97b347596eea66f4a32a4ff))
|
|
45
|
+
|
|
46
|
+
- Bump the npm_and_yarn group across 1 directory with 3 updates ([494a67b](494a67b3028d4b3d6312d94375955a1a6a36137d))
|
|
47
|
+
|
|
48
|
+
- Sync ProjectSetup update (grep nudge, safe index, doc collections) ([7522f4d](7522f4da18195aa0d0c2ee4b9e54e637a6599a06))
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
[Unreleased]: https://github.com/madeinoz67/madeinoz-voice-server/compare/trunk...HEAD
|
|
53
|
+
## [0.1.11] - 2026-02-17
|
|
54
|
+
|
|
8
55
|
### Documentation
|
|
9
56
|
|
|
10
57
|
- FIFO queue documentation with visual diagrams ([53f4dd8](53f4dd8118c55f10dcbe55d42b054cd803731d4f))
|
|
@@ -27,7 +74,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
27
74
|
|
|
28
75
|
|
|
29
76
|
|
|
30
|
-
[
|
|
77
|
+
[0.1.7]: https://github.com/madeinoz67/madeinoz-voice-server/compare/v0.1.7...v0.1.11
|
|
31
78
|
## [0.1.7] - 2026-02-09
|
|
32
79
|
|
|
33
80
|
### Ci
|
package/CLAUDE.md
CHANGED
|
@@ -42,10 +42,60 @@ bun run typecheck # Type check only
|
|
|
42
42
|
|
|
43
43
|
## Notes
|
|
44
44
|
|
|
45
|
-
- Codanna indexing is enabled for `src
|
|
45
|
+
- Codanna indexing is enabled for `src/ts` and `tests/ts` directories
|
|
46
46
|
- Semantic search is available for code queries
|
|
47
47
|
- Documentation is built with MkDocs Material theme
|
|
48
48
|
|
|
49
|
+
## Code navigation: Codanna
|
|
50
|
+
|
|
51
|
+
This repo is indexed by Codanna (local symbol graph + semantic search).
|
|
52
|
+
|
|
53
|
+
If the location of a symbol or the shape of a module is not already known, find it with
|
|
54
|
+
Codanna before reading files or grepping. A file already named in the issue or the error
|
|
55
|
+
is not a search: read it. A hook nudges once per session when a search starts with `rg` or `grep`
|
|
56
|
+
before Codanna has been used, so the rule is not something to remember.
|
|
57
|
+
|
|
58
|
+
### Which interface
|
|
59
|
+
- **Main session:** use the `mcp__codanna__*` tools. The server runs `serve --watch` from the main checkout and stays current as files change.
|
|
60
|
+
- **Subagents:** use the `codanna` CLI through Bash (the `x-ray` skill wraps it). The CLI reads the index in its current directory, so it stays correct inside worktrees.
|
|
61
|
+
- **Whole-codebase shape** (module map, hubs, which modules talk to each other, history): use the `graph` skill.
|
|
62
|
+
|
|
63
|
+
### Workflow
|
|
64
|
+
1. **Rephrase first.** Search for what is meant, not the literal words (e.g. "the login stuff" → "authentication session token validation").
|
|
65
|
+
2. **Locate:** semantic search with context, limit 5. Focus on results with score > 0.6. Use `find_symbol` for exact names and `search_symbols` for partial ones.
|
|
66
|
+
3. **Read narrowly:** read only the returned line range (`sed -n 'A,Bp' file` or Read with offset/limit).
|
|
67
|
+
4. **Expand:** follow 1–2 important relationships per result (`retrieve describe`, callers, calls). Run `analyze_impact` **before** changing any public symbol, signature, or shared function.
|
|
68
|
+
5. **Docs:** `search_documents` for project docs and markdown.
|
|
69
|
+
6. **Refine or fall back:** if the scores are weak, rephrase once. Use `rg` for exact strings, config keys, HTML/CSS/templates, and any language Codanna doesn't parse.
|
|
70
|
+
|
|
71
|
+
CLI equivalents:
|
|
72
|
+
- `codanna mcp semantic_search_with_context query:"..." limit:5 --json`
|
|
73
|
+
- `codanna mcp find_symbol <Name> --json`
|
|
74
|
+
- `codanna mcp find_callers <Name> --json`
|
|
75
|
+
- `codanna mcp analyze_impact <Name> --json`
|
|
76
|
+
- `codanna retrieve describe <Name|symbol_id:N>`
|
|
77
|
+
- `codanna retrieve calls <Name>`
|
|
78
|
+
|
|
79
|
+
Add `lang:<language>` to a search to narrow it by language.
|
|
80
|
+
|
|
81
|
+
### Rules
|
|
82
|
+
- Codanna results are navigation hints, not ground truth. Confirm against the file before editing.
|
|
83
|
+
- Before renaming or changing a signature, list every caller in your plan.
|
|
84
|
+
- If a Codanna command exits with code 7 (stale index), run `codanna index` in the current checkout and retry.
|
|
85
|
+
- Never pass `.` to `codanna index`. It registers the cwd as a root and rewrites `.codanna/settings.toml` with an absolute path, double-indexing the tree (32 files/413 symbols vs 18/322 here). Bare `codanna index` or `codanna index -f` rebuild the configured roots only.
|
|
86
|
+
- Prefer `symbol_id:N` over names when available; it avoids ambiguity.
|
|
87
|
+
|
|
88
|
+
### Orca worktrees
|
|
89
|
+
- In Orca, each agent is a full Claude Code session whose working directory is its own worktree. Orca's `scripts.setup` builds that worktree's Codanna index, and the `.mcp.json` server indexes the worktree, so `mcp__codanna__*` tools are correct there.
|
|
90
|
+
- If a Codanna tool reports a missing or stale index (exit code 7) right after the worktree opens, setup may still be running: wait, or run `codanna index` once.
|
|
91
|
+
- Never share `.codanna/` between worktrees (symlink or `sharedDirectories`).
|
|
92
|
+
|
|
93
|
+
### Subagents and worktrees
|
|
94
|
+
- A subagent in `.claude/worktrees/*` must **not** use `mcp__codanna__*`. Those tools read the main checkout's index, which doesn't match the worktree's files.
|
|
95
|
+
- Worktree subagents build their own index first (`codanna index` from the worktree root), then use the CLI.
|
|
96
|
+
- Never point the CLI at the main checkout from inside a worktree (`--config ../..`, `cd` to the repo root). Worktree isolation blocks it anyway.
|
|
97
|
+
- Delegate read-only investigation to `codanna-explorer` and isolated edits to `codanna-implementer`.
|
|
98
|
+
|
|
49
99
|
## Recent Changes
|
|
50
100
|
- 001-fifo-notify-queue: Added TypeScript 5.x (Bun runtime) + Bun native APIs, MLX-audio CLI (existing)
|
|
51
101
|
- 001-qwen-tts: Added [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
|
|
Binary file
|
|
Binary file
|
package/orca.yaml
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# orca.yaml (repo root) — Orca project config with Codanna bootstrap (Bun scripts)
|
|
2
|
+
# author: Stephen Eaton
|
|
3
|
+
#
|
|
4
|
+
# setup runs in each new worktree right after Orca creates it.
|
|
5
|
+
# archive runs before Orca deletes a worktree.
|
|
6
|
+
# Script env: ORCA_ROOT_PATH (primary checkout), ORCA_WORKTREE_PATH, ORCA_WORKSPACE_NAME.
|
|
7
|
+
# Requires bun and codanna on PATH.
|
|
8
|
+
#
|
|
9
|
+
# Do NOT add .codanna (or .codanna/index) to worktree.sharedDirectories.
|
|
10
|
+
# Shared directories are symlinked, so every worktree would read and write ONE
|
|
11
|
+
# index built from the primary checkout's files, which is wrong for every branch.
|
|
12
|
+
|
|
13
|
+
scripts:
|
|
14
|
+
setup: |
|
|
15
|
+
bun "$ORCA_ROOT_PATH/.orca/codanna-setup.ts"
|
|
16
|
+
archive: |
|
|
17
|
+
bun "$ORCA_ROOT_PATH/.orca/codanna-archive.ts"
|
|
18
|
+
|
|
19
|
+
# worktree:
|
|
20
|
+
# sharedDirectories: # optional: share heavy, branch-independent caches only
|
|
21
|
+
# - node_modules
|
package/package.json
CHANGED
|
@@ -0,0 +1,380 @@
|
|
|
1
|
+
# Research Report: FIFO Queue Implementation for HTTP Request Queuing in Bun/TypeScript
|
|
2
|
+
|
|
3
|
+
**Date**: 2025-02-09 12:00
|
|
4
|
+
**Agent**: Research-Agent
|
|
5
|
+
**Model**: Claude 4.7
|
|
6
|
+
**Context**: Voice server TTS request queuing with 201 Accepted pattern, max 100 items depth
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Executive Summary
|
|
11
|
+
|
|
12
|
+
After comprehensive analysis of queue implementation patterns for Bun/TypeScript and the specific requirements of the voice server (sequential TTS processing, HTTP async pattern, graceful shutdown), the **recommended approach is a simple array-based queue with async/await producer-consumer pattern and Atomics for thread safety**.
|
|
13
|
+
|
|
14
|
+
This approach prioritizes simplicity, Bun-specific optimizations, and the single-process nature of the current application.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Decision
|
|
19
|
+
|
|
20
|
+
**Chosen Approach**: Array-based FIFO queue with:
|
|
21
|
+
- TypeScript array as the underlying data structure
|
|
22
|
+
- Async/await for producer-consumer coordination
|
|
23
|
+
- Promises for queue slot availability signaling
|
|
24
|
+
- Atomic operations for queue depth tracking (optional for single-process)
|
|
25
|
+
- Built-in graceful shutdown with queue draining
|
|
26
|
+
|
|
27
|
+
**Rationale**:
|
|
28
|
+
1. **Simplicity**: Array-based queues are straightforward to implement, debug, and maintain
|
|
29
|
+
2. **Bun Optimization**: Bun's runtime optimizes async operations and Promise handling
|
|
30
|
+
3. **Single-Process Architecture**: Current implementation is single-process (no workers), eliminating need for shared memory
|
|
31
|
+
4. **Sufficient Performance**: For max 100 queue depth and sequential TTS processing, array operations are more than adequate
|
|
32
|
+
5. **Type Safety**: TypeScript provides excellent type safety for queue items
|
|
33
|
+
6. **Graceful Shutdown**: Simple to implement draining logic with async/await patterns
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## Alternatives Considered
|
|
38
|
+
|
|
39
|
+
### 1. Linked List Implementation
|
|
40
|
+
**Evaluated**: Custom linked list with Node objects
|
|
41
|
+
**Rejected Because**:
|
|
42
|
+
- Higher memory overhead per item (additional object references)
|
|
43
|
+
- No performance benefit for small queue sizes (100 items)
|
|
44
|
+
- More complex implementation with potential for memory leaks
|
|
45
|
+
- Poor cache locality compared to arrays
|
|
46
|
+
- TypeScript requires careful type definitions
|
|
47
|
+
|
|
48
|
+
### 2. Atomics/Shared Memory
|
|
49
|
+
**Evaluated**: SharedArrayBuffer with Atomics for inter-process communication
|
|
50
|
+
**Rejected Because**:
|
|
51
|
+
- Current architecture is single-process (no workers/threads)
|
|
52
|
+
- Adds complexity without benefit for non-concurrent processing
|
|
53
|
+
- Requires special HTTP headers (Cross-Origin-Opener-Policy)
|
|
54
|
+
- Overkill for sequential TTS processing
|
|
55
|
+
- Security considerations with shared memory
|
|
56
|
+
|
|
57
|
+
### 3. Bull/BullMQ (Redis-based)
|
|
58
|
+
**Evaluated**: Industry-standard Redis-backed job queues
|
|
59
|
+
**Rejected Because**:
|
|
60
|
+
- External dependency on Redis (adds infrastructure complexity)
|
|
61
|
+
- Overkill for single-server, in-memory queuing
|
|
62
|
+
- Network latency for queue operations
|
|
63
|
+
- Additional operational overhead
|
|
64
|
+
- Current requirements don't need persistence or distributed processing
|
|
65
|
+
|
|
66
|
+
### 4. Bun Workers with Message Channels
|
|
67
|
+
**Evaluated**: Separate worker process with structured clone for queue items
|
|
68
|
+
**Rejected Because**:
|
|
69
|
+
- Current TTS processing is already async (MLX-audio subprocess)
|
|
70
|
+
- Adding workers would complicate the architecture
|
|
71
|
+
- Structured cloning overhead for queue items
|
|
72
|
+
- No CPU-bound work to offload (I/O bound TTS operations)
|
|
73
|
+
- Would require refactoring existing MLX client integration
|
|
74
|
+
|
|
75
|
+
### 5. EventEmitter-based Queue
|
|
76
|
+
**Evaluated**: Node.js EventEmitter for queue state changes
|
|
77
|
+
**Rejected Because**:
|
|
78
|
+
- Less explicit than Promise-based coordination
|
|
79
|
+
- Harder to reason about async flow
|
|
80
|
+
- Potential for unhandled rejections
|
|
81
|
+
- Promises are more idiomatic for async/await patterns
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
85
|
+
## Technical Implementation Details
|
|
86
|
+
|
|
87
|
+
### Queue Data Structure
|
|
88
|
+
|
|
89
|
+
```typescript
|
|
90
|
+
interface QueuedRequest {
|
|
91
|
+
id: string;
|
|
92
|
+
request: NotificationRequest | PaiNotificationRequest;
|
|
93
|
+
resolve: (response: SuccessResponse | ErrorResponse) => void;
|
|
94
|
+
reject: (error: Error) => void;
|
|
95
|
+
timestamp: number;
|
|
96
|
+
}
|
|
97
|
+
|
|
98
|
+
class RequestQueue {
|
|
99
|
+
private queue: QueuedRequest[] = [];
|
|
100
|
+
private processing = false;
|
|
101
|
+
private maxDepth: number;
|
|
102
|
+
|
|
103
|
+
constructor(maxDepth: number = 100) {
|
|
104
|
+
this.maxDepth = maxDepth;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
async enqueue(item: QueuedRequest): Promise<void> {
|
|
108
|
+
if (this.queue.length >= this.maxDepth) {
|
|
109
|
+
throw new Error("Queue depth exceeded");
|
|
110
|
+
}
|
|
111
|
+
this.queue.push(item);
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
dequeue(): QueuedRequest | undefined {
|
|
115
|
+
return this.queue.shift();
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
get size(): number {
|
|
119
|
+
return this.queue.length;
|
|
120
|
+
}
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### Producer-Consumer Pattern
|
|
125
|
+
|
|
126
|
+
**Producer (HTTP Handler)**:
|
|
127
|
+
```typescript
|
|
128
|
+
// In server.ts fetch handler
|
|
129
|
+
if (path === "/notify" && req.method === "POST") {
|
|
130
|
+
const body = await parseJsonBody<NotificationRequest>(req);
|
|
131
|
+
|
|
132
|
+
// Return 202 Accepted immediately
|
|
133
|
+
const responsePromise = requestQueue.enqueue({
|
|
134
|
+
id: generateId(),
|
|
135
|
+
request: body,
|
|
136
|
+
timestamp: Date.now()
|
|
137
|
+
});
|
|
138
|
+
|
|
139
|
+
return Response.json(
|
|
140
|
+
{ status: "accepted", message: "Request queued" },
|
|
141
|
+
{ status: 202 }
|
|
142
|
+
);
|
|
143
|
+
}
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
**Consumer (Background Processor)**:
|
|
147
|
+
```typescript
|
|
148
|
+
async function processQueue(): Promise<void> {
|
|
149
|
+
while (true) {
|
|
150
|
+
const item = requestQueue.dequeue();
|
|
151
|
+
if (!item) {
|
|
152
|
+
await sleep(50); // Avoid busy-wait
|
|
153
|
+
continue;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
try {
|
|
157
|
+
const response = await handleNotify(item.request);
|
|
158
|
+
item.resolve(response);
|
|
159
|
+
} catch (error) {
|
|
160
|
+
item.reject(error as Error);
|
|
161
|
+
}
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
### Graceful Shutdown
|
|
167
|
+
|
|
168
|
+
```typescript
|
|
169
|
+
interface ServerState {
|
|
170
|
+
requestQueue: RequestQueue;
|
|
171
|
+
shutdownRequested: boolean;
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
const shutdown = async () => {
|
|
175
|
+
logger.info("Shutting down server...");
|
|
176
|
+
serverState.shutdownRequested = true;
|
|
177
|
+
|
|
178
|
+
// Stop accepting new requests
|
|
179
|
+
serverState.requestQueue.close();
|
|
180
|
+
|
|
181
|
+
// Wait for queue to drain (with timeout)
|
|
182
|
+
const drainTimeout = 30000; // 30 seconds
|
|
183
|
+
const startTime = Date.now();
|
|
184
|
+
|
|
185
|
+
while (serverState.requestQueue.size > 0) {
|
|
186
|
+
if (Date.now() - startTime > drainTimeout) {
|
|
187
|
+
logger.warn("Shutdown timeout - forcing exit");
|
|
188
|
+
break;
|
|
189
|
+
}
|
|
190
|
+
await sleep(100);
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
server.stop();
|
|
194
|
+
process.exit(0);
|
|
195
|
+
};
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## Concurrency Patterns Analysis
|
|
201
|
+
|
|
202
|
+
### Bun-Specific Considerations
|
|
203
|
+
|
|
204
|
+
1. **Async I/O Optimization**:
|
|
205
|
+
- Bun's event loop is optimized for async operations
|
|
206
|
+
- Array operations with Promises leverage this optimization
|
|
207
|
+
|
|
208
|
+
2. **Worker_pool**:
|
|
209
|
+
- Bun has built-in worker pool for CPU tasks
|
|
210
|
+
- Not needed for I/O-bound TTS operations
|
|
211
|
+
|
|
212
|
+
3. **Atomic Operations**:
|
|
213
|
+
- Bun supports Atomics API
|
|
214
|
+
- Use only if implementing multi-consumer queue
|
|
215
|
+
- For single-consumer (current use case), not necessary
|
|
216
|
+
|
|
217
|
+
### Recommended Concurrency Pattern
|
|
218
|
+
|
|
219
|
+
**Single Consumer, Multiple Producers**:
|
|
220
|
+
- Multiple HTTP requests can add to queue simultaneously
|
|
221
|
+
- Single background worker processes sequentially
|
|
222
|
+
- Ensures TTS operations don't overlap (resource constraint)
|
|
223
|
+
- Matches MLX-audio's single-stream limitation
|
|
224
|
+
|
|
225
|
+
---
|
|
226
|
+
|
|
227
|
+
## Performance Considerations
|
|
228
|
+
|
|
229
|
+
### Array Performance (Max 100 items)
|
|
230
|
+
|
|
231
|
+
- **Enqueue**: O(1) amortized (array push)
|
|
232
|
+
- **Dequeue**: O(n) worst case (array shift) - negligible for n=100
|
|
233
|
+
- **Memory**: ~100 item references (negligible)
|
|
234
|
+
- **Cache locality**: Excellent (contiguous memory)
|
|
235
|
+
|
|
236
|
+
### Optimization: Circular Buffer (Optional)
|
|
237
|
+
|
|
238
|
+
If performance becomes critical:
|
|
239
|
+
```typescript
|
|
240
|
+
class CircularBuffer<T> {
|
|
241
|
+
private buffer: (T | undefined)[];
|
|
242
|
+
private head = 0;
|
|
243
|
+
private tail = 0;
|
|
244
|
+
private count = 0;
|
|
245
|
+
|
|
246
|
+
constructor(private capacity: number) {
|
|
247
|
+
this.buffer = new Array(capacity);
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
enqueue(item: T): boolean {
|
|
251
|
+
if (this.count >= this.capacity) return false;
|
|
252
|
+
this.buffer[this.tail] = item;
|
|
253
|
+
this.tail = (this.tail + 1) % this.capacity;
|
|
254
|
+
this.count++;
|
|
255
|
+
return true;
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
dequeue(): T | undefined {
|
|
259
|
+
if (this.count === 0) return undefined;
|
|
260
|
+
const item = this.buffer[this.head];
|
|
261
|
+
this.buffer[this.head] = undefined;
|
|
262
|
+
this.head = (this.head + 1) % this.capacity;
|
|
263
|
+
this.count--;
|
|
264
|
+
return item;
|
|
265
|
+
}
|
|
266
|
+
}
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
**Recommendation**: Start with simple array, optimize only if profiling shows need.
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## Implementation Checklist
|
|
274
|
+
|
|
275
|
+
### Core Features
|
|
276
|
+
- [ ] Array-based queue implementation
|
|
277
|
+
- [ ] Enqueue with max depth check (100 items)
|
|
278
|
+
- [ ] Dequeue for background processor
|
|
279
|
+
- [ ] Promise-based response coordination
|
|
280
|
+
- [ ] Request ID generation for tracking
|
|
281
|
+
|
|
282
|
+
### HTTP Integration
|
|
283
|
+
- [ ] Modify server.ts to return 202 Accepted
|
|
284
|
+
- [ ] Add queue endpoint (optional: GET /queue/status)
|
|
285
|
+
- [ ] Error responses for queue full (503 Service Unavailable)
|
|
286
|
+
|
|
287
|
+
### Background Processor
|
|
288
|
+
- [ ] Async function to process queue sequentially
|
|
289
|
+
- [ ] Integration with existing processTTS()
|
|
290
|
+
- [ ] Error handling and logging per item
|
|
291
|
+
|
|
292
|
+
### Graceful Shutdown
|
|
293
|
+
- [ ] Close queue on shutdown signal
|
|
294
|
+
- [ ] Wait for queue drain (with timeout)
|
|
295
|
+
- [ ] Log remaining items on timeout
|
|
296
|
+
|
|
297
|
+
### Monitoring (Optional)
|
|
298
|
+
- [ ] Queue depth metric
|
|
299
|
+
- [ ] Processing time per item
|
|
300
|
+
- [ ] Average wait time
|
|
301
|
+
- [ ] Rejection count (queue full)
|
|
302
|
+
|
|
303
|
+
---
|
|
304
|
+
|
|
305
|
+
## Testing Strategy
|
|
306
|
+
|
|
307
|
+
### Unit Tests
|
|
308
|
+
```typescript
|
|
309
|
+
describe('RequestQueue', () => {
|
|
310
|
+
test('enqueues items up to max depth', async () => {
|
|
311
|
+
const queue = new RequestQueue(2);
|
|
312
|
+
await queue.enqueue(mockItem);
|
|
313
|
+
await queue.enqueue(mockItem);
|
|
314
|
+
await expect(queue.enqueue(mockItem)).rejects.toThrow();
|
|
315
|
+
});
|
|
316
|
+
|
|
317
|
+
test('dequeues items in FIFO order', async () => {
|
|
318
|
+
const queue = new RequestQueue();
|
|
319
|
+
await queue.enqueue(item1);
|
|
320
|
+
await queue.enqueue(item2);
|
|
321
|
+
expect(queue.dequeue()).toBe(item1);
|
|
322
|
+
expect(queue.dequeue()).toBe(item2);
|
|
323
|
+
});
|
|
324
|
+
});
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
### Integration Tests
|
|
328
|
+
- Test HTTP 202 Accepted response
|
|
329
|
+
- Test queue full scenario (503 response)
|
|
330
|
+
- Test sequential processing
|
|
331
|
+
- Test graceful shutdown with draining
|
|
332
|
+
|
|
333
|
+
---
|
|
334
|
+
|
|
335
|
+
## Monitoring and Observability
|
|
336
|
+
|
|
337
|
+
### Metrics to Track
|
|
338
|
+
1. **Queue Depth**: Current number of queued requests
|
|
339
|
+
2. **Processing Rate**: Requests per second
|
|
340
|
+
3. **Wait Time**: Time from enqueue to processing start
|
|
341
|
+
4. **Rejection Rate**: Requests rejected due to full queue
|
|
342
|
+
|
|
343
|
+
### Logging Structure
|
|
344
|
+
```typescript
|
|
345
|
+
logger.info("Queue status", {
|
|
346
|
+
depth: queue.size,
|
|
347
|
+
maxDepth: MAX_DEPTH,
|
|
348
|
+
processing: queue.isProcessing,
|
|
349
|
+
avgWaitTime: calculateAverageWaitTime(),
|
|
350
|
+
});
|
|
351
|
+
```
|
|
352
|
+
|
|
353
|
+
---
|
|
354
|
+
|
|
355
|
+
## Conclusion
|
|
356
|
+
|
|
357
|
+
For the voice server's TTS request queuing requirements, an **array-based FIFO queue with async/await producer-consumer pattern** provides the optimal balance of simplicity, performance, and maintainability. The single-process architecture, sequential processing requirements, and small queue size (100 items) make this approach ideal.
|
|
358
|
+
|
|
359
|
+
The implementation should start with a simple array-based queue and add complexity (circular buffer, metrics, etc.) only if profiling demonstrates need. The graceful shutdown pattern ensures no requests are lost during deployments or restarts.
|
|
360
|
+
|
|
361
|
+
---
|
|
362
|
+
|
|
363
|
+
## References
|
|
364
|
+
|
|
365
|
+
### Files Examined
|
|
366
|
+
- `src/ts/server.ts:1-500` - Current HTTP server implementation
|
|
367
|
+
|
|
368
|
+
### Key Patterns Identified
|
|
369
|
+
1. Current implementation uses `Bun.serve()` with async fetch handler
|
|
370
|
+
2. TTS processing is already async via MLX-audio subprocess calls
|
|
371
|
+
3. Single-process architecture (no workers/threads)
|
|
372
|
+
4. Existing graceful shutdown with SIGTERM/SIGINT handling
|
|
373
|
+
|
|
374
|
+
### Recommended Next Steps
|
|
375
|
+
1. Implement `RequestQueue` class in `src/ts/utils/request-queue.ts`
|
|
376
|
+
2. Add background processor function
|
|
377
|
+
3. Modify server.ts to use queue for `/notify` and `/pai` endpoints
|
|
378
|
+
4. Implement graceful shutdown draining
|
|
379
|
+
5. Add unit tests for queue operations
|
|
380
|
+
6. (Optional) Add metrics endpoint for queue monitoring
|