queen-mq 0.2.0 → 0.2.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +186 -763
- package/{src → client-js}/benchmark/consumer.js +6 -4
- package/{src → client-js}/benchmark/producer.js +9 -4
- package/{src → client-js}/client/client.js +277 -7
- package/{src → client-js}/client/index.js +1 -4
- package/{src → client-js}/client/utils/http.js +3 -2
- package/{src → client-js}/client/utils/retry.js +7 -1
- package/{src → client-js}/test/README.md +30 -9
- package/{src → client-js}/test/core-tests.js +7 -2
- package/{src → client-js}/test/edge-case-tests.js +2 -1
- package/client-js/test/human.js +162 -0
- package/client-js/test/partition-transaction-tests.js +482 -0
- package/client-js/test/qos0-tests.js +334 -0
- package/{src → client-js}/test/test-new.js +55 -0
- package/package.json +8 -17
- package/init-db.js +0 -20
- package/src/cluster-server.js +0 -242
- package/src/config.js +0 -229
- package/src/database/connection.js +0 -129
- package/src/database/poolManager.js +0 -199
- package/src/database/schema-v2.sql +0 -278
- package/src/managers/eventManager.js +0 -59
- package/src/managers/queueManagerOptimized.js +0 -1634
- package/src/managers/resourceCache.js +0 -96
- package/src/managers/systemEventManager.js +0 -132
- package/src/routes/ack.js +0 -26
- package/src/routes/configure.js +0 -46
- package/src/routes/messages.js +0 -368
- package/src/routes/pop.js +0 -69
- package/src/routes/push.js +0 -28
- package/src/routes/resources.js +0 -330
- package/src/routes/status.js +0 -1037
- package/src/server.js +0 -1633
- package/src/test/MIGRATION_ISSUES.md +0 -174
- package/src/test/test.js +0 -4524
- package/src/utils/streaming.js +0 -231
- package/src/webapp-dist/assets/Analytics-DJrE4Rz2.css +0 -1
- package/src/webapp-dist/assets/Analytics-HEMXKmfJ.js +0 -1
- package/src/webapp-dist/assets/ConfirmDialog-YEoCdE7x.js +0 -1
- package/src/webapp-dist/assets/ConsumerGroups-CC5VJywp.css +0 -1
- package/src/webapp-dist/assets/ConsumerGroups-D8Z20HhZ.js +0 -1
- package/src/webapp-dist/assets/Dashboard-BRFOz6it.css +0 -1
- package/src/webapp-dist/assets/Dashboard-Bpx9I32h.js +0 -1
- package/src/webapp-dist/assets/LoadingSpinner-BPttXcbs.js +0 -1
- package/src/webapp-dist/assets/Messages-BOfVHKma.css +0 -1
- package/src/webapp-dist/assets/Messages-EKURCbLl.js +0 -1
- package/src/webapp-dist/assets/QueueDetail-BXXA7xoZ.css +0 -1
- package/src/webapp-dist/assets/QueueDetail-Bmxg0KiA.js +0 -1
- package/src/webapp-dist/assets/Queues-CvrJkrZT.css +0 -1
- package/src/webapp-dist/assets/Queues-yUx9_3-9.js +0 -1
- package/src/webapp-dist/assets/StatusBadge-C1bBtXHh.css +0 -1
- package/src/webapp-dist/assets/StatusBadge-C6N84XmO.js +0 -1
- package/src/webapp-dist/assets/analytics-BapTekLM.js +0 -1
- package/src/webapp-dist/assets/colors-5z6Q_kUr.js +0 -18
- package/src/webapp-dist/assets/index-B7w2bKT6.css +0 -1
- package/src/webapp-dist/assets/index-CUSnwYTH.js +0 -31
- package/src/webapp-dist/assets/messages-CRV6E-xU.js +0 -1
- package/src/webapp-dist/assets/queen-logo-blue.svg +0 -210
- package/src/webapp-dist/assets/queen-logo-cyan.svg +0 -210
- package/src/webapp-dist/assets/queen-logo-indigo.svg +0 -210
- package/src/webapp-dist/assets/queen-logo-orange.svg +0 -210
- package/src/webapp-dist/assets/queen-logo-pink.svg +0 -210
- package/src/webapp-dist/assets/queen-logo-purple.svg +0 -210
- package/src/webapp-dist/assets/queen-logo-rose.svg +0 -239
- package/src/webapp-dist/assets/queen-logo.svg +0 -263
- package/src/webapp-dist/assets/queues-BdUbxfBC.js +0 -1
- package/src/webapp-dist/assets/resources-Bs9U-ceF.js +0 -1
- package/src/webapp-dist/index.html +0 -15
- package/src/websocket/wsServer.js +0 -228
- /package/{src → client-js}/benchmark/consumer_multi.js +0 -0
- /package/{src → client-js}/benchmark/producer_multi.js +0 -0
- /package/{src → client-js}/client/utils/loadBalancer.js +0 -0
- /package/{src → client-js}/services/encryptionService.js +0 -0
- /package/{src → client-js}/services/evictionService.js +0 -0
- /package/{src → client-js}/services/retentionService.js +0 -0
- /package/{src → client-js}/services/startupSync.js +0 -0
- /package/{src → client-js}/test/advanced-client-tests.js +0 -0
- /package/{src → client-js}/test/advanced-pattern-tests.js +0 -0
- /package/{src → client-js}/test/bus-mode-tests.js +0 -0
- /package/{src → client-js}/test/enterprise-tests.js +0 -0
- /package/{src → client-js}/test/partition-locking-tests.js +0 -0
- /package/{src → client-js}/test/utils.js +0 -0
- /package/{src → client-js}/test/window-buffer-test.js +0 -0
- /package/{src → client-js}/utils/logger.js +0 -0
- /package/{src → client-js}/utils/uuid.js +0 -0
|
@@ -1,174 +0,0 @@
|
|
|
1
|
-
# Test Migration Issues and Fixes
|
|
2
|
-
|
|
3
|
-
## Summary
|
|
4
|
-
|
|
5
|
-
When migrating tests from the old `queenClient` interface to the new minimalist `Queen` client interface, we discovered client bugs that needed fixing. The system is transactional - ACKs immediately release partition locks when awaited.
|
|
6
|
-
|
|
7
|
-
## Client Bugs Fixed
|
|
8
|
-
|
|
9
|
-
### 1. Null Payload Handling in `push()`
|
|
10
|
-
**Issue**: `typeof null === 'object'` is `true` in JavaScript, causing the code to try to access properties on `null` payloads.
|
|
11
|
-
|
|
12
|
-
**Location**: `src/client/client.js` line 153
|
|
13
|
-
|
|
14
|
-
**Fix**:
|
|
15
|
-
```javascript
|
|
16
|
-
// Before
|
|
17
|
-
const isMessageObject = typeof item === 'object' &&
|
|
18
|
-
(item._payload || item._transactionId || item._traceId);
|
|
19
|
-
|
|
20
|
-
// After
|
|
21
|
-
const isMessageObject = typeof item === 'object' && item !== null &&
|
|
22
|
-
(item._payload || item._transactionId || item._traceId);
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
**Why**: Tests correctly validate edge cases including null payloads. The client must handle them gracefully.
|
|
26
|
-
|
|
27
|
-
### 2. Null Handling in `ack()`
|
|
28
|
-
**Issue**: Same null-checking issue when extracting transaction ID from message object.
|
|
29
|
-
|
|
30
|
-
**Location**: `src/client/client.js` line 330
|
|
31
|
-
|
|
32
|
-
**Fix**:
|
|
33
|
-
```javascript
|
|
34
|
-
// Before
|
|
35
|
-
const transactionId = typeof message === 'object' ?
|
|
36
|
-
(message.transactionId || message.id) :
|
|
37
|
-
message;
|
|
38
|
-
|
|
39
|
-
// After
|
|
40
|
-
const transactionId = typeof message === 'object' && message !== null ?
|
|
41
|
-
(message.transactionId || message.id) :
|
|
42
|
-
message;
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
### 3. Consumer Group Parsing for Namespace/Task Addresses
|
|
46
|
-
**Issue**: The `#parseAddress()` method didn't extract consumer groups from namespace/task patterns like `namespace:billing@group-a`.
|
|
47
|
-
|
|
48
|
-
**Location**: `src/client/client.js` line 77-101
|
|
49
|
-
|
|
50
|
-
**Fix**: Extract `@group` portion before parsing namespace/task segments:
|
|
51
|
-
```javascript
|
|
52
|
-
#parseAddress(address) {
|
|
53
|
-
if (address.includes('namespace:') || address.includes('task:')) {
|
|
54
|
-
const parts = {};
|
|
55
|
-
|
|
56
|
-
// Extract consumer group if present (after @)
|
|
57
|
-
let workingAddress = address;
|
|
58
|
-
const atIndex = address.lastIndexOf('@');
|
|
59
|
-
if (atIndex > 0) {
|
|
60
|
-
parts.consumerGroup = address.substring(atIndex + 1);
|
|
61
|
-
workingAddress = address.substring(0, atIndex);
|
|
62
|
-
}
|
|
63
|
-
|
|
64
|
-
const segments = workingAddress.split('/');
|
|
65
|
-
for (const segment of segments) {
|
|
66
|
-
if (segment.startsWith('namespace:')) {
|
|
67
|
-
parts.namespace = segment.substring(10);
|
|
68
|
-
} else if (segment.startsWith('task:')) {
|
|
69
|
-
parts.task = segment.substring(5);
|
|
70
|
-
}
|
|
71
|
-
}
|
|
72
|
-
return parts;
|
|
73
|
-
}
|
|
74
|
-
// ...
|
|
75
|
-
}
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
### 4. Defensive Filtering of Null Messages
|
|
79
|
-
**Issue**: Server might return null values in the `messages` array response.
|
|
80
|
-
|
|
81
|
-
**Location**: `src/client/client.js` in `take()` method
|
|
82
|
-
|
|
83
|
-
**Fix**: Filter out null/undefined messages when yielding:
|
|
84
|
-
```javascript
|
|
85
|
-
// Yield each message (filter out any null/undefined values)
|
|
86
|
-
for (const message of result.messages) {
|
|
87
|
-
if (message) { // Skip null/undefined messages
|
|
88
|
-
yield message;
|
|
89
|
-
count++;
|
|
90
|
-
if (limit && count >= limit) return;
|
|
91
|
-
}
|
|
92
|
-
}
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
## Test Fixes
|
|
96
|
-
|
|
97
|
-
### 1. Consumer Group Isolation Test
|
|
98
|
-
**Problem**: Test was ACKing all messages as completed for both groups, but the test expects group2 to have some failed messages to verify isolation.
|
|
99
|
-
|
|
100
|
-
**Solution**: Actually fail the first 2 messages for group2:
|
|
101
|
-
```javascript
|
|
102
|
-
// Fail first 2 messages for group2, complete the rest
|
|
103
|
-
for (let i = 0; i < group2Messages.length; i++) {
|
|
104
|
-
if (i < 2) {
|
|
105
|
-
await client.ack(group2Messages[i], false, {
|
|
106
|
-
error: 'Test failure',
|
|
107
|
-
group: 'group2'
|
|
108
|
-
});
|
|
109
|
-
} else {
|
|
110
|
-
await client.ack(group2Messages[i], true, { group: 'group2' });
|
|
111
|
-
}
|
|
112
|
-
}
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
### 2. Empty and Null Payloads Test
|
|
116
|
-
**Problem**: If server returns null message objects in the array, the test would crash.
|
|
117
|
-
|
|
118
|
-
**Solution**: Add defensive check (though client now filters these):
|
|
119
|
-
```javascript
|
|
120
|
-
for await (const msg of client.take(queue, { limit: 10 })) {
|
|
121
|
-
if (!msg) {
|
|
122
|
-
throw new Error('Received null/undefined message from take()');
|
|
123
|
-
}
|
|
124
|
-
messages.push(msg);
|
|
125
|
-
await client.ack(msg);
|
|
126
|
-
}
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
## Key Learnings
|
|
130
|
-
|
|
131
|
-
### System is Transactional ✅
|
|
132
|
-
|
|
133
|
-
The Queen system is **transactional**. When you `await client.ack(msg)`:
|
|
134
|
-
- The ACK completes atomically
|
|
135
|
-
- The partition lock is released immediately
|
|
136
|
-
- No "timing issues" or delays needed
|
|
137
|
-
|
|
138
|
-
### Correct Pattern
|
|
139
|
-
```javascript
|
|
140
|
-
// ✅ This works correctly:
|
|
141
|
-
for await (const msg of client.take(queue, { limit: 10 })) {
|
|
142
|
-
await client.ack(msg); // Partition unlocked when this completes
|
|
143
|
-
}
|
|
144
|
-
|
|
145
|
-
// Next consumer can immediately access the partition:
|
|
146
|
-
for await (const msg of client.take(queue, { limit: 10 })) {
|
|
147
|
-
// Gets messages right away
|
|
148
|
-
}
|
|
149
|
-
```
|
|
150
|
-
|
|
151
|
-
### Edge Case Handling
|
|
152
|
-
|
|
153
|
-
1. **Null payloads are valid**: Tests should validate that the system handles `null`, `undefined`, empty objects, and empty strings as payloads.
|
|
154
|
-
|
|
155
|
-
2. **Client must be defensive**: Check for `typeof x === 'object' && x !== null` when you need to access properties.
|
|
156
|
-
|
|
157
|
-
3. **Server responses should be filtered**: If server returns null messages in an array, filter them out client-side.
|
|
158
|
-
|
|
159
|
-
### Test Design Principles
|
|
160
|
-
|
|
161
|
-
1. **Test real edge cases**: null, undefined, empty strings, large payloads, etc.
|
|
162
|
-
|
|
163
|
-
2. **Don't add artificial delays**: The system is transactional. If you need a delay, something is wrong.
|
|
164
|
-
|
|
165
|
-
3. **Verify isolation**: When testing consumer groups, actually test that different groups can have different statuses for the same messages.
|
|
166
|
-
|
|
167
|
-
4. **Use appropriate batch sizes**: Match batch size to expected message count for efficient testing.
|
|
168
|
-
|
|
169
|
-
## Final Results
|
|
170
|
-
|
|
171
|
-
After fixing client bugs and test issues:
|
|
172
|
-
- 33/33 tests should pass
|
|
173
|
-
- No timing sensitivity
|
|
174
|
-
- Clean, transactional behavior throughout
|