queen-mq 0.2.11 → 0.2.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -175,6 +175,37 @@ FILE_BUFFER_DIR=/custom/path ./bin/queen-server
175
175
  | **Auto-Ack** | Fire-and-forget events, notifications | Tasks requiring retry logic |
176
176
  | **Failover** | Everything (automatic) | N/A - always beneficial |
177
177
 
178
+ ## Architecture
179
+
180
+ Queen uses a high-performance **acceptor/worker pattern** with uWebSockets, combining non-blocking I/O for HTTP/WebSocket with a dedicated thread pool for database operations.
181
+
182
+ **View the interactive architecture diagram:** [architecture.svg](./assets/architecture.svg)
183
+
184
+ **Key Components:**
185
+ - **UWS Acceptor**: Single thread listening on port 6632, round-robin distributes to workers
186
+ - **UWS Workers**: N event loop threads (default: 10) handling HTTP routes and WebSocket
187
+ - **Response Timers**: Per-worker timers (25ms tick) drain response queue back to clients
188
+ - **DB ThreadPool**: Separate pool for blocking PostgreSQL operations
189
+ - **Poll Workers**: 2 reserved threads for long-polling with adaptive backoff (100ms→2000ms)
190
+ - **Poll Intention Registry**: Thread-safe store for long-poll requests
191
+ - **Database Pool**: 150 shared PostgreSQL connections (libpq) with mutex/condition variable
192
+ - **Response Queue**: Thread-safe queue decoupling DB results from event loop responses
193
+
194
+ **Request Flow:**
195
+ 1. Client → Acceptor → Worker (event loop)
196
+ 2. Worker registers response, submits job to DB ThreadPool
197
+ 3. DB thread executes query, pushes result to Response Queue
198
+ 4. Worker's response timer drains queue, sends HTTP response
199
+
200
+ **Long-Polling Flow:**
201
+ 1. No immediate messages? Register intention in Registry
202
+ 2. Poll Workers wake every 50ms, group intentions by queue/partition/consumer
203
+ 3. Rate-limited DB queries (100ms initial, exponential backoff to 2s)
204
+ 4. Messages distributed to waiting clients via Response Queue
205
+ 5. Timeouts detected by Poll Workers, send 204 No Content
206
+
207
+ This architecture provides high concurrency, efficient connection pooling, and minimal latency for both immediate and long-polling requests.
208
+
178
209
  ## Webapp
179
210
 
180
211
  A modern Vue 3 web interface for managing and monitoring Queen MQ.
@@ -244,10 +275,6 @@ You can use Queen directly from HTTP without the JS client.
244
275
  ### Server Startup Timing (Critical)
245
276
  **Issue:** Worker initialization timeout (30s → 3600s) now matches file buffer recovery timeout. This is a temporary fix.
246
277
 
247
- **Problem:** During startup, if the file buffer has many events to recover (e.g., after a long PostgreSQL outage), Worker 0 performs blocking recovery that can take up to 1 hour. The acceptor waits for all workers to initialize before starting to accept connections.
248
-
249
- **Current Fix:** Worker initialization timeout increased to 3600s to prevent premature timeout.
250
-
251
278
  **Better Solution Needed:**
252
279
  - Make recovery non-blocking while preserving FIFO ordering guarantees
253
280
  - Implement progressive readiness with memory-buffered queue during recovery
@@ -255,10 +282,10 @@ You can use Queen directly from HTTP without the JS client.
255
282
  - See: `server/src/services/file_buffer.cpp:212` (MAX_STARTUP_RECOVERY_SECONDS)
256
283
  - See: `server/src/acceptor_server.cpp:1876` (worker initialization timeout)
257
284
 
285
+
258
286
  ### Other TODO Items
259
287
  - retention jobs
260
288
  - reconsume
261
- - fix frontend
262
- - pg async
263
- - pg reconnect
264
- - memory leak?
289
+ - new client
290
+ - auth
291
+ - streaming engine
@@ -50,7 +50,7 @@ async function consumer(consumerId, partition) {
50
50
  try {
51
51
  for await (const messages of q.takeBatch(target, { //`namespace:benchmark`,
52
52
  wait: true,
53
- timeout: 15000,
53
+ timeout: 30000,
54
54
  batch: BATCH_SIZE,
55
55
  idleTimeout: 5000
56
56
  })) {
@@ -3,7 +3,7 @@ import fs from 'fs';
3
3
 
4
4
  const TOTAL_MESSAGES = 1000000;
5
5
  const PARTITIONS = 10;
6
- const MAX_BATCH_SIZE = 10000; // Reduced from 10000
6
+ const MAX_BATCH_SIZE = 100; // Reduced from 10000
7
7
  const PUSH_PARALLEL = 10; // Reduced from 10 to avoid lock contention
8
8
  const QUEUE_NAME = 'benchmark-queue-001';
9
9
 
@@ -1244,7 +1244,11 @@ class PipelineBuilder {
1244
1244
  await tx.commit();
1245
1245
  } else {
1246
1246
  // Default: just ACK the messages
1247
- await this.#client.ack(messages, true);
1247
+ // Extract consumer group from queue address if present (format: queue@group)
1248
+ const atIndex = this.#queue.lastIndexOf('@');
1249
+ const consumerGroup = atIndex > 0 ? this.#queue.substring(atIndex + 1) : null;
1250
+ const context = consumerGroup ? { group: consumerGroup } : {};
1251
+ await this.#client.ack(messages, true, context);
1248
1252
  }
1249
1253
 
1250
1254
  return {
@@ -40,7 +40,7 @@ The test suite is organized into focused, modular files:
40
40
  - Subscription modes (all vs new messages)
41
41
  - Consumer group isolation
42
42
 
43
- #### 5. Edge Cases (`edge-case-tests.js`)
43
+ #### 5. Edge Cases (`edge-case-tests.js` & `partition-transaction-tests.js`)
44
44
  - Empty and null payloads
45
45
  - Very large payloads
46
46
  - Concurrent push/take operations
@@ -48,6 +48,11 @@ The test suite is organized into focused, modular files:
48
48
  - Lease expiration and redelivery
49
49
  - SQL injection prevention
50
50
  - XSS prevention
51
+ - **Partition-scoped transaction_id handling** (`partition-transaction-tests.js`):
52
+ - Duplicate transaction IDs across partitions with correct ACK targeting
53
+ - Batch ACK with duplicate transaction IDs
54
+ - DLQ operations with duplicate transaction IDs
55
+ - Analytics API with duplicate transaction IDs
51
56
 
52
57
  #### 6. Advanced Patterns (`advanced-pattern-tests.js`)
53
58
  - Multi-stage pipeline workflow
@@ -138,20 +143,36 @@ for await (const msg of client.take('myqueue/partition@group', { limit: 10 })) {
138
143
  cd /Users/alice/Work/queen
139
144
 
140
145
  # Run all tests
141
- nvm use 22 && node src/test/test-new.js
146
+ nvm use 22 && node client-js/test/test-new.js
142
147
 
143
148
  # Run specific test category
144
- node src/test/test-new.js core # Core features only
145
- node src/test/test-new.js partition # Partition locking tests only
146
- node src/test/test-new.js enterprise # Enterprise features only
147
- node src/test/test-new.js bus # Bus mode tests only
148
- node src/test/test-new.js edge # Edge cases only
149
- node src/test/test-new.js advanced # Advanced patterns only
149
+ node client-js/test/test-new.js core # Core features only
150
+ node client-js/test/test-new.js partition # Partition locking tests only
151
+ node client-js/test/test-new.js enterprise # Enterprise features only
152
+ node client-js/test/test-new.js bus # Bus mode tests only
153
+ node client-js/test/test-new.js edge # Edge cases (includes partition-scoped txn_id tests)
154
+ node client-js/test/test-new.js advanced # Advanced patterns only
150
155
 
151
156
  # Show help
152
- node src/test/test-new.js help
157
+ node client-js/test/test-new.js help
153
158
  ```
154
159
 
160
+ ### Testing Partition-Scoped Transaction ID Fix
161
+
162
+ The `edge` test category includes critical tests for partition-scoped transaction_id handling:
163
+
164
+ ```bash
165
+ # Run all edge case tests (includes partition transaction_id tests)
166
+ node client-js/test/test-new.js edge
167
+ ```
168
+
169
+ These tests verify that:
170
+ - ✅ The same `transaction_id` can exist in multiple partitions
171
+ - ✅ ACK operations correctly target messages using BOTH `partition_id` AND `transaction_id`
172
+ - ✅ Batch ACK operations scope correctly by partition
173
+ - ✅ DLQ operations only affect the intended partition
174
+ - ✅ Analytics API (`GET /api/v1/messages/:partitionId/:transactionId`) returns the correct message
175
+
155
176
  ### Available Test Categories
156
177
 
157
178
  - **`core`** - Core features (queue creation, push, take, ack, delayed processing, FIFO)
@@ -0,0 +1,482 @@
1
+ /**
2
+ * Partition-Scoped Transaction ID Tests
3
+ *
4
+ * Tests that transaction_id uniqueness is properly scoped to partitions.
5
+ * With the constraint UNIQUE(partition_id, transaction_id), the same transaction_id
6
+ * can exist in multiple partitions. This tests that ACK operations correctly
7
+ * identify and operate on the right message by using BOTH partition_id AND transaction_id.
8
+ */
9
+
10
+ import { startTest, passTest, failTest, sleep, dbPool, getMessageCount, log, TEST_CONFIG } from './utils.js';
11
+
12
+ /**
13
+ * Test: Duplicate transaction IDs across partitions with correct ACK targeting
14
+ *
15
+ * This is the critical test that verifies the fix for partition-scoped transaction_ids.
16
+ * Without the fix, ACKing a message in one partition could accidentally ACK
17
+ * a message with the same transaction_id in a different partition.
18
+ */
19
+ export async function testDuplicateTransactionIdsAcrossPartitions(client) {
20
+ startTest('Duplicate Transaction IDs Across Partitions - ACK Targeting', 'edge');
21
+
22
+ try {
23
+ const queue = 'test-duplicate-txn-ids';
24
+ const partition1 = 'partition-A';
25
+ const partition2 = 'partition-B';
26
+ const sharedTxnId = 'shared-txn-12345';
27
+
28
+ // Configure queue
29
+ await client.queue(queue, { leaseTime: 30 });
30
+
31
+ // CRITICAL: Push messages with the SAME transaction_id to DIFFERENT partitions
32
+ log(`Pushing message with txn=${sharedTxnId} to ${partition1}`, 'test');
33
+ await client.push(`${queue}/${partition1}`,
34
+ { data: 'Message in Partition A', partition: partition1 },
35
+ { transactionId: sharedTxnId }
36
+ );
37
+
38
+ log(`Pushing message with txn=${sharedTxnId} to ${partition2}`, 'test');
39
+ await client.push(`${queue}/${partition2}`,
40
+ { data: 'Message in Partition B', partition: partition2 },
41
+ { transactionId: sharedTxnId }
42
+ );
43
+
44
+ await sleep(100);
45
+
46
+ // Verify both messages exist in database
47
+ const msgCountA = await getMessageCount(queue, partition1);
48
+ const msgCountB = await getMessageCount(queue, partition2);
49
+
50
+ if (msgCountA !== 1) {
51
+ throw new Error(`Expected 1 message in ${partition1}, found ${msgCountA}`);
52
+ }
53
+ if (msgCountB !== 1) {
54
+ throw new Error(`Expected 1 message in ${partition2}, found ${msgCountB}`);
55
+ }
56
+
57
+ log('✓ Both messages with same transaction_id exist in different partitions', 'success');
58
+
59
+ // Take message from partition A
60
+ log(`Taking message from ${partition1}`, 'test');
61
+ let messageA = null;
62
+ for await (const msg of client.take(`${queue}/${partition1}`, { limit: 1 })) {
63
+ messageA = msg;
64
+ break;
65
+ }
66
+
67
+ if (!messageA) {
68
+ throw new Error('Failed to take message from partition A');
69
+ }
70
+
71
+ if (messageA.transactionId !== sharedTxnId) {
72
+ throw new Error(`Expected transaction_id ${sharedTxnId}, got ${messageA.transactionId}`);
73
+ }
74
+
75
+ if (messageA.data.partition !== partition1) {
76
+ throw new Error(`Expected partition ${partition1}, got ${messageA.data.partition}`);
77
+ }
78
+
79
+ log(`✓ Received correct message from ${partition1}: ${JSON.stringify(messageA.data)}`, 'success');
80
+
81
+ // ACK the message from partition A
82
+ log(`ACKing message in ${partition1} with txn=${sharedTxnId}`, 'test');
83
+ await client.ack(messageA, true);
84
+
85
+ await sleep(200);
86
+
87
+ // CRITICAL CHECK: Verify message in partition A is acknowledged
88
+ const result = await dbPool.query(`
89
+ SELECT
90
+ m.id,
91
+ m.transaction_id,
92
+ p.name as partition_name,
93
+ pc.last_consumed_id,
94
+ CASE
95
+ WHEN m.id <= pc.last_consumed_id AND DATE_TRUNC('milliseconds', m.created_at) <= DATE_TRUNC('milliseconds', pc.last_consumed_created_at)
96
+ THEN 'completed'
97
+ ELSE 'pending'
98
+ END as status
99
+ FROM queen.messages m
100
+ JOIN queen.partitions p ON p.id = m.partition_id
101
+ JOIN queen.queues q ON q.id = p.queue_id
102
+ LEFT JOIN queen.partition_consumers pc ON pc.partition_id = p.id AND pc.consumer_group = '__QUEUE_MODE__'
103
+ WHERE q.name = $1 AND m.transaction_id = $2
104
+ ORDER BY p.name
105
+ `, [queue, sharedTxnId]);
106
+
107
+ if (result.rows.length !== 2) {
108
+ throw new Error(`Expected 2 messages in database, found ${result.rows.length}`);
109
+ }
110
+
111
+ const msgA = result.rows.find(r => r.partition_name === partition1);
112
+ const msgB = result.rows.find(r => r.partition_name === partition2);
113
+
114
+ if (!msgA || !msgB) {
115
+ throw new Error('Could not find both messages in query result');
116
+ }
117
+
118
+ log(`Message A (${partition1}): status=${msgA.status}`, 'test');
119
+ log(`Message B (${partition2}): status=${msgB.status}`, 'test');
120
+
121
+ // CRITICAL ASSERTION: Message in partition A should be completed
122
+ if (msgA.status !== 'completed') {
123
+ throw new Error(`Message in ${partition1} should be completed, but status is: ${msgA.status}`);
124
+ }
125
+
126
+ // CRITICAL ASSERTION: Message in partition B should still be pending
127
+ if (msgB.status !== 'pending') {
128
+ throw new Error(`Message in ${partition2} should still be pending, but status is: ${msgB.status}`);
129
+ }
130
+
131
+ log(`✓ ACK correctly targeted only ${partition1}, ${partition2} remains pending`, 'success');
132
+
133
+ // Now take and ACK message from partition B
134
+ log(`Taking message from ${partition2}`, 'test');
135
+ let messageB = null;
136
+ for await (const msg of client.take(`${queue}/${partition2}`, { limit: 1 })) {
137
+ messageB = msg;
138
+ break;
139
+ }
140
+
141
+ if (!messageB) {
142
+ throw new Error('Failed to take message from partition B');
143
+ }
144
+
145
+ if (messageB.transactionId !== sharedTxnId) {
146
+ throw new Error(`Expected transaction_id ${sharedTxnId}, got ${messageB.transactionId}`);
147
+ }
148
+
149
+ if (messageB.data.partition !== partition2) {
150
+ throw new Error(`Expected partition ${partition2}, got ${messageB.data.partition}`);
151
+ }
152
+
153
+ log(`✓ Received correct message from ${partition2}: ${JSON.stringify(messageB.data)}`, 'success');
154
+
155
+ // ACK the message from partition B
156
+ log(`ACKing message in ${partition2} with txn=${sharedTxnId}`, 'test');
157
+ await client.ack(messageB, true);
158
+
159
+ await sleep(200);
160
+
161
+ // Verify both messages are now completed
162
+ const finalResult = await dbPool.query(`
163
+ SELECT
164
+ p.name as partition_name,
165
+ CASE
166
+ WHEN m.id <= pc.last_consumed_id AND DATE_TRUNC('milliseconds', m.created_at) <= DATE_TRUNC('milliseconds', pc.last_consumed_created_at)
167
+ THEN 'completed'
168
+ ELSE 'pending'
169
+ END as status
170
+ FROM queen.messages m
171
+ JOIN queen.partitions p ON p.id = m.partition_id
172
+ JOIN queen.queues q ON q.id = p.queue_id
173
+ LEFT JOIN queen.partition_consumers pc ON pc.partition_id = p.id AND pc.consumer_group = '__QUEUE_MODE__'
174
+ WHERE q.name = $1 AND m.transaction_id = $2
175
+ ORDER BY p.name
176
+ `, [queue, sharedTxnId]);
177
+
178
+ const finalMsgA = finalResult.rows.find(r => r.partition_name === partition1);
179
+ const finalMsgB = finalResult.rows.find(r => r.partition_name === partition2);
180
+
181
+ if (finalMsgA.status !== 'completed' || finalMsgB.status !== 'completed') {
182
+ throw new Error('Both messages should be completed after individual ACKs');
183
+ }
184
+
185
+ log('✓ Both messages correctly ACKed independently', 'success');
186
+
187
+ passTest('Partition-scoped transaction_id ACK targeting works correctly');
188
+ } catch (error) {
189
+ failTest(error);
190
+ }
191
+ }
192
+
193
+ /**
194
+ * Test: Batch ACK with duplicate transaction IDs across partitions
195
+ *
196
+ * Tests that batch acknowledgment correctly scopes to partition_id
197
+ */
198
+ export async function testBatchAckWithDuplicateTransactionIds(client) {
199
+ startTest('Batch ACK with Duplicate Transaction IDs', 'edge');
200
+
201
+ try {
202
+ const queue = 'test-batch-ack-duplicate';
203
+ const partition1 = 'partition-X';
204
+ const partition2 = 'partition-Y';
205
+
206
+ await client.queue(queue, { leaseTime: 30 });
207
+
208
+ // Push 3 messages to partition1 with specific transaction IDs
209
+ const txnIds = ['batch-1', 'batch-2', 'batch-3'];
210
+
211
+ for (const txnId of txnIds) {
212
+ await client.push(`${queue}/${partition1}`,
213
+ { data: `P1-${txnId}` },
214
+ { transactionId: txnId }
215
+ );
216
+
217
+ // Push same transaction_id to partition2
218
+ await client.push(`${queue}/${partition2}`,
219
+ { data: `P2-${txnId}` },
220
+ { transactionId: txnId }
221
+ );
222
+ }
223
+
224
+ await sleep(500); // Give more time for messages to be ready
225
+
226
+ // Take all messages from partition1 (batch size 10 to get all 3)
227
+ const messagesP1 = [];
228
+ for await (const msg of client.take(`${queue}/${partition1}`, { batch: 10, limit: 10 })) {
229
+ messagesP1.push(msg);
230
+ if (messagesP1.length >= 3) break;
231
+ }
232
+
233
+ if (messagesP1.length !== 3) {
234
+ log(`DEBUG: Only got ${messagesP1.length} messages. Transaction IDs: ${messagesP1.map(m => m.transactionId).join(', ')}`, 'warning');
235
+
236
+ // Check database to see what's there
237
+ const dbCheck = await dbPool.query(`
238
+ SELECT m.transaction_id, p.name as partition_name, m.created_at
239
+ FROM queen.messages m
240
+ JOIN queen.partitions p ON p.id = m.partition_id
241
+ JOIN queen.queues q ON q.id = p.queue_id
242
+ WHERE q.name = $1
243
+ ORDER BY p.name, m.created_at
244
+ `, [queue]);
245
+
246
+ log(`DEBUG: Database has ${dbCheck.rows.length} messages: ${JSON.stringify(dbCheck.rows)}`, 'warning');
247
+ throw new Error(`Expected 3 messages from partition1, got ${messagesP1.length}`);
248
+ }
249
+
250
+ log(`Received ${messagesP1.length} messages from ${partition1}`, 'test');
251
+
252
+ // Batch ACK all messages from partition1
253
+ for (const msg of messagesP1) {
254
+ await client.ack(msg, true);
255
+ }
256
+
257
+ await sleep(200);
258
+
259
+ // Verify partition1 messages are completed but partition2 messages are still pending
260
+ for (const txnId of txnIds) {
261
+ const result = await dbPool.query(`
262
+ SELECT
263
+ p.name as partition_name,
264
+ CASE
265
+ WHEN m.id <= pc.last_consumed_id AND DATE_TRUNC('milliseconds', m.created_at) <= DATE_TRUNC('milliseconds', pc.last_consumed_created_at)
266
+ THEN 'completed'
267
+ ELSE 'pending'
268
+ END as status
269
+ FROM queen.messages m
270
+ JOIN queen.partitions p ON p.id = m.partition_id
271
+ JOIN queen.queues q ON q.id = p.queue_id
272
+ LEFT JOIN queen.partition_consumers pc ON pc.partition_id = p.id AND pc.consumer_group = '__QUEUE_MODE__'
273
+ WHERE q.name = $1 AND m.transaction_id = $2
274
+ ORDER BY p.name
275
+ `, [queue, txnId]);
276
+
277
+ const p1Status = result.rows.find(r => r.partition_name === partition1)?.status;
278
+ const p2Status = result.rows.find(r => r.partition_name === partition2)?.status;
279
+
280
+ if (p1Status !== 'completed') {
281
+ throw new Error(`${partition1} message with txn=${txnId} should be completed, got ${p1Status}`);
282
+ }
283
+
284
+ if (p2Status !== 'pending') {
285
+ throw new Error(`${partition2} message with txn=${txnId} should be pending, got ${p2Status}`);
286
+ }
287
+ }
288
+
289
+ log(`✓ Batch ACK in ${partition1} did not affect ${partition2}`, 'success');
290
+
291
+ passTest('Batch ACK correctly scopes by partition_id');
292
+ } catch (error) {
293
+ failTest(error);
294
+ }
295
+ }
296
+
297
+ /**
298
+ * Test: DLQ with duplicate transaction IDs
299
+ *
300
+ * Tests that moving messages to DLQ correctly scopes by partition_id
301
+ */
302
+ export async function testDLQWithDuplicateTransactionIds(client) {
303
+ startTest('DLQ with Duplicate Transaction IDs', 'edge');
304
+
305
+ try {
306
+ const queue = 'test-dlq-duplicate';
307
+ const partition1 = 'partition-DLQ-A';
308
+ const partition2 = 'partition-DLQ-B';
309
+ const txnId = 'dlq-shared-txn';
310
+
311
+ await client.queue(queue, { retryLimit: 0 }); // Immediate DLQ
312
+
313
+ // Push same transaction_id to both partitions
314
+ await client.push(`${queue}/${partition1}`, { data: 'P1-data' }, { transactionId: txnId });
315
+ await client.push(`${queue}/${partition2}`, { data: 'P2-data' }, { transactionId: txnId });
316
+
317
+ await sleep(100);
318
+
319
+ // Take from partition1 and fail it
320
+ let msgP1 = null;
321
+ for await (const msg of client.take(`${queue}/${partition1}`, { limit: 1 })) {
322
+ msgP1 = msg;
323
+ break;
324
+ }
325
+
326
+ if (!msgP1) {
327
+ throw new Error('Failed to take message from partition1');
328
+ }
329
+
330
+ // ACK as failed - should go to DLQ
331
+ await client.ack(msgP1, false, { error: 'Test error for P1' });
332
+
333
+ await sleep(200);
334
+
335
+ // Check DLQ - only partition1 message should be there
336
+ const dlqResult = await dbPool.query(`
337
+ SELECT
338
+ p.name as partition_name,
339
+ dlq.error_message
340
+ FROM queen.dead_letter_queue dlq
341
+ JOIN queen.messages m ON dlq.message_id = m.id
342
+ JOIN queen.partitions p ON p.id = m.partition_id
343
+ JOIN queen.queues q ON q.id = p.queue_id
344
+ WHERE q.name = $1 AND m.transaction_id = $2
345
+ `, [queue, txnId]);
346
+
347
+ if (dlqResult.rows.length !== 1) {
348
+ throw new Error(`Expected 1 message in DLQ, found ${dlqResult.rows.length}`);
349
+ }
350
+
351
+ if (dlqResult.rows[0].partition_name !== partition1) {
352
+ throw new Error(`Expected DLQ message from ${partition1}, got ${dlqResult.rows[0].partition_name}`);
353
+ }
354
+
355
+ if (!dlqResult.rows[0].error_message.includes('Test error for P1')) {
356
+ throw new Error(`Unexpected error message: ${dlqResult.rows[0].error_message}`);
357
+ }
358
+
359
+ log(`✓ DLQ correctly scoped to ${partition1}`, 'success');
360
+
361
+ // Verify partition2 message is still pending
362
+ let msgP2 = null;
363
+ for await (const msg of client.take(`${queue}/${partition2}`, { limit: 1 })) {
364
+ msgP2 = msg;
365
+ break;
366
+ }
367
+
368
+ if (!msgP2) {
369
+ throw new Error('Message in partition2 should still be available');
370
+ }
371
+
372
+ if (msgP2.transactionId !== txnId) {
373
+ throw new Error(`Expected transaction_id ${txnId} in P2, got ${msgP2.transactionId}`);
374
+ }
375
+
376
+ log(`✓ ${partition2} message with same txn_id is still available`, 'success');
377
+
378
+ passTest('DLQ correctly scopes by partition_id');
379
+ } catch (error) {
380
+ failTest(error);
381
+ }
382
+ }
383
+
384
+ /**
385
+ * Test: Analytics API with duplicate transaction IDs
386
+ *
387
+ * Tests that the GET /api/v1/messages/:partitionId/:transactionId endpoint
388
+ * returns the correct message when the same transaction_id exists in multiple partitions
389
+ */
390
+ export async function testAnalyticsAPIWithDuplicateTransactionIds(client) {
391
+ startTest('Analytics API with Duplicate Transaction IDs', 'edge');
392
+
393
+ try {
394
+ const queue = 'test-analytics-duplicate';
395
+ const partition1 = 'analytics-P1';
396
+ const partition2 = 'analytics-P2';
397
+ const txnId = 'analytics-shared-txn';
398
+
399
+ await client.queue(queue, {});
400
+
401
+ // Push messages with same transaction_id to both partitions
402
+ await client.push(`${queue}/${partition1}`,
403
+ { unique: 'data-from-P1', timestamp: Date.now() },
404
+ { transactionId: txnId }
405
+ );
406
+
407
+ await client.push(`${queue}/${partition2}`,
408
+ { unique: 'data-from-P2', timestamp: Date.now() + 1000 },
409
+ { transactionId: txnId }
410
+ );
411
+
412
+ await sleep(100);
413
+
414
+ // Get partition IDs from database
415
+ const partitionResult = await dbPool.query(`
416
+ SELECT p.id, p.name
417
+ FROM queen.partitions p
418
+ JOIN queen.queues q ON q.id = p.queue_id
419
+ WHERE q.name = $1
420
+ ORDER BY p.name
421
+ `, [queue]);
422
+
423
+ const p1Id = partitionResult.rows.find(r => r.name === partition1)?.id;
424
+ const p2Id = partitionResult.rows.find(r => r.name === partition2)?.id;
425
+
426
+ if (!p1Id || !p2Id) {
427
+ throw new Error('Failed to get partition IDs');
428
+ }
429
+
430
+ log(`Partition IDs: ${partition1}=${p1Id}, ${partition2}=${p2Id}`, 'test');
431
+
432
+ // Fetch message using Analytics API (via HTTP)
433
+ const baseUrl = TEST_CONFIG.baseUrls[0];
434
+
435
+ // Fetch from partition1
436
+ const response1 = await fetch(`${baseUrl}/api/v1/messages/${p1Id}/${txnId}`);
437
+ if (!response1.ok) {
438
+ throw new Error(`Failed to fetch message from partition1: ${response1.statusText}`);
439
+ }
440
+ const data1 = await response1.json();
441
+
442
+ if (data1.transactionId !== txnId) {
443
+ throw new Error(`Expected transactionId ${txnId}, got ${data1.transactionId}`);
444
+ }
445
+
446
+ if (data1.partition !== partition1) {
447
+ throw new Error(`Expected partition ${partition1}, got ${data1.partition}`);
448
+ }
449
+
450
+ if (data1.payload.unique !== 'data-from-P1') {
451
+ throw new Error(`Expected data from P1, got: ${JSON.stringify(data1.payload)}`);
452
+ }
453
+
454
+ log(`✓ Analytics API correctly returned message from ${partition1}`, 'success');
455
+
456
+ // Fetch from partition2
457
+ const response2 = await fetch(`${baseUrl}/api/v1/messages/${p2Id}/${txnId}`);
458
+ if (!response2.ok) {
459
+ throw new Error(`Failed to fetch message from partition2: ${response2.statusText}`);
460
+ }
461
+ const data2 = await response2.json();
462
+
463
+ if (data2.transactionId !== txnId) {
464
+ throw new Error(`Expected transactionId ${txnId}, got ${data2.transactionId}`);
465
+ }
466
+
467
+ if (data2.partition !== partition2) {
468
+ throw new Error(`Expected partition ${partition2}, got ${data2.partition}`);
469
+ }
470
+
471
+ if (data2.payload.unique !== 'data-from-P2') {
472
+ throw new Error(`Expected data from P2, got: ${JSON.stringify(data2.payload)}`);
473
+ }
474
+
475
+ log(`✓ Analytics API correctly returned message from ${partition2}`, 'success');
476
+
477
+ passTest('Analytics API correctly distinguishes messages by partition_id');
478
+ } catch (error) {
479
+ failTest(error);
480
+ }
481
+ }
482
+
@@ -80,6 +80,13 @@ import {
80
80
  testXSSPrevention
81
81
  } from './edge-case-tests.js';
82
82
 
83
+ import {
84
+ testDuplicateTransactionIdsAcrossPartitions,
85
+ testBatchAckWithDuplicateTransactionIds,
86
+ testDLQWithDuplicateTransactionIds,
87
+ testAnalyticsAPIWithDuplicateTransactionIds
88
+ } from './partition-transaction-tests.js';
89
+
83
90
  import {
84
91
  testMultiStagePipeline,
85
92
  testFanOutFanIn,
@@ -274,6 +281,12 @@ async function runTests() {
274
281
  await runTest(() => testLeaseExpiration(client));
275
282
  await runTest(() => testSQLInjectionPrevention(client));
276
283
  await runTest(() => testXSSPrevention(client));
284
+
285
+ // Partition-scoped transaction_id tests
286
+ await runTest(() => testDuplicateTransactionIdsAcrossPartitions(client));
287
+ await runTest(() => testBatchAckWithDuplicateTransactionIds(client));
288
+ await runTest(() => testDLQWithDuplicateTransactionIds(client));
289
+ await runTest(() => testAnalyticsAPIWithDuplicateTransactionIds(client));
277
290
  }
278
291
 
279
292
  // ============================================
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "queen-mq",
3
- "version": "0.2.11",
3
+ "version": "0.2.22",
4
4
  "type": "module",
5
5
  "description": "High-performance C++ message queue backed by PostgreSQL",
6
6
  "main": "client-js/client/index.js",