@aztec/p2p 0.0.1-commit.9ee6fcc6 → 0.0.1-commit.9ef841308

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (117) hide show
  1. package/dest/client/factory.d.ts +1 -1
  2. package/dest/client/factory.d.ts.map +1 -1
  3. package/dest/client/factory.js +6 -5
  4. package/dest/client/test/tx_proposal_collector/proposal_tx_collector_worker.js +14 -3
  5. package/dest/index.d.ts +1 -2
  6. package/dest/index.d.ts.map +1 -1
  7. package/dest/index.js +0 -1
  8. package/dest/mem_pools/index.d.ts +1 -2
  9. package/dest/mem_pools/index.d.ts.map +1 -1
  10. package/dest/mem_pools/tx_pool_v2/tx_metadata.d.ts +1 -1
  11. package/dest/mem_pools/tx_pool_v2/tx_metadata.d.ts.map +1 -1
  12. package/dest/mem_pools/tx_pool_v2/tx_metadata.js +5 -1
  13. package/dest/services/discv5/discV5_service.d.ts +1 -1
  14. package/dest/services/discv5/discV5_service.d.ts.map +1 -1
  15. package/dest/services/discv5/discV5_service.js +4 -2
  16. package/dest/services/libp2p/libp2p_service.d.ts +7 -2
  17. package/dest/services/libp2p/libp2p_service.d.ts.map +1 -1
  18. package/dest/services/libp2p/libp2p_service.js +115 -34
  19. package/dest/services/peer-manager/peer_manager.d.ts +6 -2
  20. package/dest/services/peer-manager/peer_manager.d.ts.map +1 -1
  21. package/dest/services/peer-manager/peer_manager.js +18 -6
  22. package/dest/services/peer-manager/peer_scoring.d.ts +5 -2
  23. package/dest/services/peer-manager/peer_scoring.d.ts.map +1 -1
  24. package/dest/services/peer-manager/peer_scoring.js +28 -10
  25. package/dest/services/reqresp/batch-tx-requester/batch_tx_requester.d.ts +1 -1
  26. package/dest/services/reqresp/batch-tx-requester/batch_tx_requester.d.ts.map +1 -1
  27. package/dest/services/reqresp/batch-tx-requester/batch_tx_requester.js +3 -0
  28. package/dest/services/reqresp/rate-limiter/rate_limiter.d.ts +5 -4
  29. package/dest/services/reqresp/rate-limiter/rate_limiter.d.ts.map +1 -1
  30. package/dest/services/reqresp/rate-limiter/rate_limiter.js +10 -8
  31. package/dest/services/service.d.ts +7 -1
  32. package/dest/services/service.d.ts.map +1 -1
  33. package/dest/services/tx_collection/file_store_tx_source.d.ts +5 -4
  34. package/dest/services/tx_collection/file_store_tx_source.d.ts.map +1 -1
  35. package/dest/services/tx_collection/file_store_tx_source.js +39 -29
  36. package/dest/services/tx_collection/tx_source.d.ts +6 -5
  37. package/dest/services/tx_collection/tx_source.d.ts.map +1 -1
  38. package/dest/services/tx_collection/tx_source.js +9 -7
  39. package/dest/test-helpers/mock-pubsub.d.ts +6 -1
  40. package/dest/test-helpers/mock-pubsub.d.ts.map +1 -1
  41. package/dest/test-helpers/mock-pubsub.js +9 -1
  42. package/dest/testbench/p2p_client_testbench_worker.js +41 -12
  43. package/dest/testbench/worker_client_manager.d.ts +1 -1
  44. package/dest/testbench/worker_client_manager.d.ts.map +1 -1
  45. package/dest/testbench/worker_client_manager.js +0 -1
  46. package/dest/util.d.ts +8 -3
  47. package/dest/util.d.ts.map +1 -1
  48. package/dest/util.js +2 -9
  49. package/package.json +14 -14
  50. package/src/client/factory.ts +8 -3
  51. package/src/client/test/tx_proposal_collector/proposal_tx_collector_worker.ts +15 -3
  52. package/src/index.ts +0 -1
  53. package/src/mem_pools/index.ts +0 -3
  54. package/src/mem_pools/tx_pool_v2/tx_metadata.ts +7 -1
  55. package/src/msg_validators/proposal_validator/README.md +1 -1
  56. package/src/services/discv5/discV5_service.ts +4 -2
  57. package/src/services/libp2p/libp2p_service.ts +117 -43
  58. package/src/services/peer-manager/peer_manager.ts +21 -6
  59. package/src/services/peer-manager/peer_scoring.ts +21 -5
  60. package/src/services/reqresp/batch-tx-requester/batch_tx_requester.ts +3 -0
  61. package/src/services/reqresp/rate-limiter/rate_limiter.ts +13 -9
  62. package/src/services/service.ts +7 -0
  63. package/src/services/tx_collection/file_store_tx_source.ts +43 -31
  64. package/src/services/tx_collection/tx_source.ts +8 -7
  65. package/src/test-helpers/mock-pubsub.ts +9 -0
  66. package/src/testbench/p2p_client_testbench_worker.ts +40 -9
  67. package/src/testbench/worker_client_manager.ts +0 -1
  68. package/src/util.ts +8 -12
  69. package/dest/mem_pools/tx_pool/aztec_kv_tx_pool.d.ts +0 -125
  70. package/dest/mem_pools/tx_pool/aztec_kv_tx_pool.d.ts.map +0 -1
  71. package/dest/mem_pools/tx_pool/aztec_kv_tx_pool.js +0 -596
  72. package/dest/mem_pools/tx_pool/eviction/eviction_manager.d.ts +0 -32
  73. package/dest/mem_pools/tx_pool/eviction/eviction_manager.d.ts.map +0 -1
  74. package/dest/mem_pools/tx_pool/eviction/eviction_manager.js +0 -112
  75. package/dest/mem_pools/tx_pool/eviction/eviction_strategy.d.ts +0 -157
  76. package/dest/mem_pools/tx_pool/eviction/eviction_strategy.d.ts.map +0 -1
  77. package/dest/mem_pools/tx_pool/eviction/eviction_strategy.js +0 -52
  78. package/dest/mem_pools/tx_pool/eviction/fee_payer_balance_eviction_rule.d.ts +0 -16
  79. package/dest/mem_pools/tx_pool/eviction/fee_payer_balance_eviction_rule.d.ts.map +0 -1
  80. package/dest/mem_pools/tx_pool/eviction/fee_payer_balance_eviction_rule.js +0 -123
  81. package/dest/mem_pools/tx_pool/eviction/invalid_txs_after_mining_rule.d.ts +0 -17
  82. package/dest/mem_pools/tx_pool/eviction/invalid_txs_after_mining_rule.d.ts.map +0 -1
  83. package/dest/mem_pools/tx_pool/eviction/invalid_txs_after_mining_rule.js +0 -84
  84. package/dest/mem_pools/tx_pool/eviction/invalid_txs_after_reorg_rule.d.ts +0 -19
  85. package/dest/mem_pools/tx_pool/eviction/invalid_txs_after_reorg_rule.d.ts.map +0 -1
  86. package/dest/mem_pools/tx_pool/eviction/invalid_txs_after_reorg_rule.js +0 -78
  87. package/dest/mem_pools/tx_pool/eviction/low_priority_eviction_rule.d.ts +0 -26
  88. package/dest/mem_pools/tx_pool/eviction/low_priority_eviction_rule.d.ts.map +0 -1
  89. package/dest/mem_pools/tx_pool/eviction/low_priority_eviction_rule.js +0 -84
  90. package/dest/mem_pools/tx_pool/eviction/nullifier_conflict_pre_add_rule.d.ts +0 -25
  91. package/dest/mem_pools/tx_pool/eviction/nullifier_conflict_pre_add_rule.d.ts.map +0 -1
  92. package/dest/mem_pools/tx_pool/eviction/nullifier_conflict_pre_add_rule.js +0 -57
  93. package/dest/mem_pools/tx_pool/index.d.ts +0 -3
  94. package/dest/mem_pools/tx_pool/index.d.ts.map +0 -1
  95. package/dest/mem_pools/tx_pool/index.js +0 -2
  96. package/dest/mem_pools/tx_pool/priority.d.ts +0 -12
  97. package/dest/mem_pools/tx_pool/priority.d.ts.map +0 -1
  98. package/dest/mem_pools/tx_pool/priority.js +0 -15
  99. package/dest/mem_pools/tx_pool/tx_pool.d.ts +0 -127
  100. package/dest/mem_pools/tx_pool/tx_pool.d.ts.map +0 -1
  101. package/dest/mem_pools/tx_pool/tx_pool.js +0 -3
  102. package/dest/mem_pools/tx_pool/tx_pool_test_suite.d.ts +0 -7
  103. package/dest/mem_pools/tx_pool/tx_pool_test_suite.d.ts.map +0 -1
  104. package/dest/mem_pools/tx_pool/tx_pool_test_suite.js +0 -402
  105. package/src/mem_pools/tx_pool/README.md +0 -270
  106. package/src/mem_pools/tx_pool/aztec_kv_tx_pool.ts +0 -746
  107. package/src/mem_pools/tx_pool/eviction/eviction_manager.ts +0 -132
  108. package/src/mem_pools/tx_pool/eviction/eviction_strategy.ts +0 -208
  109. package/src/mem_pools/tx_pool/eviction/fee_payer_balance_eviction_rule.ts +0 -163
  110. package/src/mem_pools/tx_pool/eviction/invalid_txs_after_mining_rule.ts +0 -104
  111. package/src/mem_pools/tx_pool/eviction/invalid_txs_after_reorg_rule.ts +0 -93
  112. package/src/mem_pools/tx_pool/eviction/low_priority_eviction_rule.ts +0 -106
  113. package/src/mem_pools/tx_pool/eviction/nullifier_conflict_pre_add_rule.ts +0 -75
  114. package/src/mem_pools/tx_pool/index.ts +0 -2
  115. package/src/mem_pools/tx_pool/priority.ts +0 -20
  116. package/src/mem_pools/tx_pool/tx_pool.ts +0 -141
  117. package/src/mem_pools/tx_pool/tx_pool_test_suite.ts +0 -321
@@ -1,402 +0,0 @@
1
- import { BlockNumber } from '@aztec/foundation/branded-types';
2
- import { unfreeze } from '@aztec/foundation/types';
3
- import { GasFees } from '@aztec/stdlib/gas';
4
- import { mockTx } from '@aztec/stdlib/testing';
5
- import { BlockHeader, GlobalVariables } from '@aztec/stdlib/tx';
6
- /**
7
- * Tests a TxPool implementation.
8
- * @param getTxPool - Gets a fresh TxPool
9
- */ export function describeTxPool(getTxPool) {
10
- let pool;
11
- const minedBlockHeader = BlockHeader.empty({
12
- globalVariables: GlobalVariables.empty({
13
- blockNumber: BlockNumber(1),
14
- timestamp: 0n
15
- })
16
- });
17
- beforeEach(()=>{
18
- pool = getTxPool();
19
- });
20
- afterEach(()=>{
21
- pool.removeAllListeners('txs-added');
22
- });
23
- it('adds txs to the pool as pending', async ()=>{
24
- const tx1 = await mockTx(1);
25
- await pool.addTxs([
26
- tx1
27
- ]);
28
- const poolTx = await pool.getTxByHash(tx1.getTxHash());
29
- expect(poolTx.getTxHash()).toEqual(tx1.getTxHash());
30
- await expect(pool.getTxStatus(tx1.getTxHash())).resolves.toEqual('pending');
31
- await expect(pool.getPendingTxHashes()).resolves.toEqual([
32
- tx1.getTxHash()
33
- ]);
34
- await expect(pool.getPendingTxCount()).resolves.toEqual(1);
35
- });
36
- it('emits txs-added event with new txs', async ()=>{
37
- const tx1 = await mockTx(1); // existing and pending
38
- const tx2 = await mockTx(2); // mined but not known
39
- const tx3 = await mockTx(3); // brand new
40
- await pool.addTxs([
41
- tx1
42
- ]);
43
- await pool.markAsMined([
44
- tx2.getTxHash()
45
- ], minedBlockHeader);
46
- let txsFromEvent = undefined;
47
- pool.once('txs-added', ({ txs })=>{
48
- txsFromEvent = txs;
49
- });
50
- await pool.addTxs([
51
- tx1,
52
- tx2,
53
- tx3
54
- ]);
55
- expect(txsFromEvent).toBeDefined();
56
- expect(txsFromEvent).toHaveLength(2);
57
- const eventHashes = txsFromEvent.map((tx)=>tx.getTxHash());
58
- expect(eventHashes).toEqual(expect.arrayContaining([
59
- tx2.getTxHash(),
60
- tx3.getTxHash()
61
- ]));
62
- });
63
- it('removes txs from the pool', async ()=>{
64
- const pendingTx = await mockTx(1);
65
- const minedTx = await mockTx(2);
66
- await pool.addTxs([
67
- pendingTx,
68
- minedTx
69
- ]);
70
- await pool.markAsMined([
71
- minedTx.getTxHash()
72
- ], minedBlockHeader);
73
- // Delete a pending tx - should be permanently deleted
74
- await pool.deleteTxs([
75
- pendingTx.getTxHash()
76
- ]);
77
- await expect(pool.getTxByHash(pendingTx.getTxHash())).resolves.toBeUndefined();
78
- await expect(pool.getTxStatus(pendingTx.getTxHash())).resolves.toBeUndefined();
79
- // Delete a mined tx - should be soft-deleted (still in storage)
80
- await pool.deleteTxs([
81
- minedTx.getTxHash()
82
- ]);
83
- await expect(pool.getTxByHash(minedTx.getTxHash())).resolves.toBeDefined();
84
- await expect(pool.getTxStatus(minedTx.getTxHash())).resolves.toEqual('deleted');
85
- await expect(pool.getMinedTxHashes()).resolves.toEqual([]);
86
- await expect(pool.getPendingTxCount()).resolves.toEqual(0);
87
- });
88
- it('marks txs as mined', async ()=>{
89
- const tx1 = await mockTx(1);
90
- const tx2 = await mockTx(2);
91
- await pool.addTxs([
92
- tx1,
93
- tx2
94
- ]);
95
- await pool.markAsMined([
96
- tx1.getTxHash()
97
- ], minedBlockHeader);
98
- const retrievedTx = await pool.getTxByHash(tx1.getTxHash());
99
- expect(retrievedTx?.getTxHash()).toEqual(tx1.getTxHash());
100
- await expect(pool.getTxStatus(tx1.getTxHash())).resolves.toEqual('mined');
101
- await expect(pool.getMinedTxHashes()).resolves.toEqual([
102
- [
103
- tx1.getTxHash(),
104
- 1
105
- ]
106
- ]);
107
- await expect(pool.getPendingTxHashes()).resolves.toEqual([
108
- tx2.getTxHash()
109
- ]);
110
- await expect(pool.getPendingTxCount()).resolves.toEqual(1);
111
- });
112
- it('marks txs as pending after being mined', async ()=>{
113
- const tx1 = await mockTx(1);
114
- const tx2 = await mockTx(2);
115
- await pool.addTxs([
116
- tx1,
117
- tx2
118
- ]);
119
- await pool.markAsMined([
120
- tx1.getTxHash()
121
- ], minedBlockHeader);
122
- await pool.markMinedAsPending([
123
- tx1.getTxHash()
124
- ], BlockNumber(1));
125
- await expect(pool.getMinedTxHashes()).resolves.toEqual([]);
126
- const pending = await pool.getPendingTxHashes();
127
- expect(pending).toHaveLength(2);
128
- expect(pending).toEqual(expect.arrayContaining([
129
- tx1.getTxHash(),
130
- tx2.getTxHash()
131
- ]));
132
- await expect(pool.getPendingTxCount()).resolves.toEqual(2);
133
- });
134
- it('only marks txs as pending if they are known', async ()=>{
135
- const tx1 = await mockTx(1);
136
- // simulate a situation where not all peers have all the txs
137
- const tx2 = await mockTx(2);
138
- const someTxHashThatThisPeerDidNotSee = tx2.getTxHash();
139
- await pool.addTxs([
140
- tx1
141
- ]);
142
- // this peer knows that tx2 was mined, but it does not have the tx object
143
- await pool.markAsMined([
144
- tx1.getTxHash(),
145
- someTxHashThatThisPeerDidNotSee
146
- ], minedBlockHeader);
147
- expect(await pool.getMinedTxHashes()).toEqual(expect.arrayContaining([
148
- [
149
- tx1.getTxHash(),
150
- 1
151
- ],
152
- [
153
- someTxHashThatThisPeerDidNotSee,
154
- 1
155
- ]
156
- ]));
157
- // reorg: both txs should now become available again
158
- await pool.markMinedAsPending([
159
- tx1.getTxHash(),
160
- someTxHashThatThisPeerDidNotSee
161
- ], BlockNumber(1));
162
- await expect(pool.getMinedTxHashes()).resolves.toEqual([]);
163
- await expect(pool.getPendingTxHashes()).resolves.toEqual([
164
- tx1.getTxHash()
165
- ]); // tx2 is not in the pool
166
- await expect(pool.getPendingTxCount()).resolves.toEqual(1);
167
- });
168
- it('returns all transactions in the pool', async ()=>{
169
- const tx1 = await mockTx(1);
170
- const tx2 = await mockTx(2);
171
- const tx3 = await mockTx(3);
172
- await pool.addTxs([
173
- tx1,
174
- tx2,
175
- tx3
176
- ]);
177
- const poolTxs = await pool.getAllTxs();
178
- expect(poolTxs).toHaveLength(3);
179
- const poolHashes = poolTxs.map((tx)=>tx.getTxHash());
180
- expect(poolHashes).toEqual(expect.arrayContaining([
181
- tx1.getTxHash(),
182
- tx2.getTxHash(),
183
- tx3.getTxHash()
184
- ]));
185
- await expect(pool.getPendingTxCount()).resolves.toEqual(3);
186
- });
187
- it('returns all txHashes in the pool', async ()=>{
188
- const tx1 = await mockTx(1);
189
- const tx2 = await mockTx(2);
190
- const tx3 = await mockTx(3);
191
- await pool.addTxs([
192
- tx1,
193
- tx2,
194
- tx3
195
- ]);
196
- const poolTxHashes = await pool.getAllTxHashes();
197
- const expectedHashes = [
198
- tx1,
199
- tx2,
200
- tx3
201
- ].map((tx)=>tx.getTxHash());
202
- expect(poolTxHashes).toHaveLength(3);
203
- expect(poolTxHashes).toEqual(expect.arrayContaining(expectedHashes));
204
- await expect(pool.getPendingTxCount()).resolves.toEqual(3);
205
- });
206
- it('returns txs by their hash', async ()=>{
207
- const tx1 = await mockTx(1);
208
- const tx2 = await mockTx(2);
209
- const tx3 = await mockTx(3);
210
- await pool.addTxs([
211
- tx1,
212
- tx2,
213
- tx3
214
- ]);
215
- const requestedTxs = await pool.getTxsByHash([
216
- tx1.getTxHash(),
217
- tx3.getTxHash()
218
- ]);
219
- expect(requestedTxs).toHaveLength(2);
220
- const requestedHashes = requestedTxs.map((tx)=>tx.getTxHash());
221
- expect(requestedHashes).toEqual(expect.arrayContaining([
222
- tx1.getTxHash(),
223
- tx3.getTxHash()
224
- ]));
225
- });
226
- it('returns a large number of transactions by their hash', async ()=>{
227
- const numTxs = 1_000;
228
- const txs = await Promise.all(Array.from({
229
- length: numTxs
230
- }, (_, i)=>mockTx(i)));
231
- const hashes = txs.map((tx)=>tx.getTxHash());
232
- await pool.addTxs(txs);
233
- const requestedTxs = await pool.getTxsByHash(hashes);
234
- expect(requestedTxs).toHaveLength(numTxs);
235
- const requestedHashes = requestedTxs.map((tx)=>tx.getTxHash());
236
- expect(requestedHashes).toEqual(expect.arrayContaining(hashes));
237
- });
238
- it('returns whether or not txs exist', async ()=>{
239
- const tx1 = await mockTx(1);
240
- const tx2 = await mockTx(2);
241
- const tx3 = await mockTx(3);
242
- await pool.addTxs([
243
- tx1,
244
- tx2,
245
- tx3
246
- ]);
247
- const tx4 = await mockTx(4);
248
- const tx5 = await mockTx(5);
249
- const availability = await pool.hasTxs([
250
- tx1.getTxHash(),
251
- tx2.getTxHash(),
252
- tx3.getTxHash(),
253
- tx4.getTxHash(),
254
- tx5.getTxHash()
255
- ]);
256
- expect(availability).toHaveLength(5);
257
- expect(availability).toEqual(expect.arrayContaining([
258
- true,
259
- true,
260
- true,
261
- false,
262
- false
263
- ]));
264
- });
265
- it('returns pending tx hashes sorted by priority', async ()=>{
266
- const withPriorityFee = (tx, fee)=>{
267
- const gs = unfreeze(tx.data.constants.txContext.gasSettings);
268
- gs.maxPriorityFeesPerGas = new GasFees(fee, fee);
269
- gs.maxFeesPerGas = new GasFees(fee, fee);
270
- return tx;
271
- };
272
- const tx1 = withPriorityFee(await mockTx(0), 1000);
273
- const tx2 = withPriorityFee(await mockTx(1), 100);
274
- const tx3 = withPriorityFee(await mockTx(2), 200);
275
- const tx4 = withPriorityFee(await mockTx(3), 3000);
276
- await pool.addTxs([
277
- tx1,
278
- tx2,
279
- tx3,
280
- tx4
281
- ]);
282
- const poolTxHashes = await pool.getPendingTxHashes();
283
- expect(poolTxHashes).toHaveLength(4);
284
- expect(poolTxHashes).toEqual([
285
- tx4,
286
- tx1,
287
- tx3,
288
- tx2
289
- ].map((tx)=>tx.getTxHash()));
290
- });
291
- describe('soft-delete', ()=>{
292
- it('soft-deletes mined txs and keeps them in storage', async ()=>{
293
- const txs = await Promise.all([
294
- mockTx(1),
295
- mockTx(2),
296
- mockTx(3)
297
- ]);
298
- await pool.addTxs(txs);
299
- // Mark first tx as mined
300
- await pool.markAsMined([
301
- txs[0].getTxHash()
302
- ], minedBlockHeader);
303
- // Verify initial state
304
- await expect(pool.getPendingTxCount()).resolves.toBe(2);
305
- await expect(pool.getTxByHash(txs[0].getTxHash())).resolves.toBeDefined();
306
- await expect(pool.getTxByHash(txs[1].getTxHash())).resolves.toBeDefined();
307
- // Delete mined tx - should be soft-deleted
308
- await pool.deleteTxs([
309
- txs[0].getTxHash()
310
- ]);
311
- // Delete pending tx - should be permanently deleted
312
- await pool.deleteTxs([
313
- txs[1].getTxHash()
314
- ]);
315
- // Verify mined tx still exists in storage but has 'deleted' status
316
- await expect(pool.getTxByHash(txs[0].getTxHash())).resolves.toBeDefined();
317
- await expect(pool.getTxStatus(txs[0].getTxHash())).resolves.toEqual('deleted');
318
- // Verify pending tx is permanently deleted
319
- await expect(pool.getTxByHash(txs[1].getTxHash())).resolves.toBeUndefined();
320
- await expect(pool.getTxStatus(txs[1].getTxHash())).resolves.toBeUndefined();
321
- // Verify remaining pending count
322
- await expect(pool.getPendingTxCount()).resolves.toBe(1);
323
- // Verify pending hashes don't include deleted txs
324
- const pendingHashes = await pool.getPendingTxHashes();
325
- expect(pendingHashes).toHaveLength(1);
326
- expect(pendingHashes.map((h)=>h.toString())).toContain(txs[2].getTxHash().toString());
327
- });
328
- it('cleans up old deleted mined transactions', async ()=>{
329
- const txs = await Promise.all([
330
- mockTx(1),
331
- mockTx(2),
332
- mockTx(3)
333
- ]);
334
- await pool.addTxs(txs);
335
- // Mark first two as mined in block 1
336
- await pool.markAsMined([
337
- txs[0].getTxHash(),
338
- txs[1].getTxHash()
339
- ], minedBlockHeader);
340
- // Soft-delete mined transactions
341
- await pool.deleteTxs([
342
- txs[0].getTxHash(),
343
- txs[1].getTxHash()
344
- ]);
345
- // Clean up deleted mined txs from block 1 and earlier
346
- const deletedCount = await pool.cleanupDeletedMinedTxs(BlockNumber(1));
347
- // Verify old transactions are permanently deleted
348
- expect(deletedCount).toBe(2);
349
- await expect(pool.getTxByHash(txs[0].getTxHash())).resolves.toBeUndefined();
350
- await expect(pool.getTxByHash(txs[1].getTxHash())).resolves.toBeUndefined();
351
- await expect(pool.getTxByHash(txs[2].getTxHash())).resolves.toBeDefined();
352
- });
353
- it('does not clean up recent deleted mined transactions', async ()=>{
354
- const txs = await Promise.all([
355
- mockTx(1),
356
- mockTx(2)
357
- ]);
358
- await pool.addTxs(txs);
359
- // Mark as mined in block 2
360
- const laterBlockHeader = BlockHeader.empty({
361
- globalVariables: GlobalVariables.empty({
362
- blockNumber: BlockNumber(2),
363
- timestamp: 0n
364
- })
365
- });
366
- await pool.markAsMined([
367
- txs[0].getTxHash()
368
- ], laterBlockHeader);
369
- // Soft-delete a mined transaction
370
- await pool.deleteTxs([
371
- txs[0].getTxHash()
372
- ]);
373
- // Try to clean up with block 1 (before the mined block)
374
- const deletedCount = await pool.cleanupDeletedMinedTxs(BlockNumber(1));
375
- // Verify no transactions were cleaned up
376
- expect(deletedCount).toBe(0);
377
- await expect(pool.getTxByHash(txs[0].getTxHash())).resolves.toBeDefined();
378
- });
379
- it('restores deleted mined tx when it is mined again', async ()=>{
380
- const tx = await mockTx(1);
381
- await pool.addTxs([
382
- tx
383
- ]);
384
- // Mark as mined
385
- await pool.markAsMined([
386
- tx.getTxHash()
387
- ], minedBlockHeader);
388
- // Soft-delete it
389
- await pool.deleteTxs([
390
- tx.getTxHash()
391
- ]);
392
- await expect(pool.getTxStatus(tx.getTxHash())).resolves.toEqual('deleted');
393
- // Mark as mined again (e.g., after a reorg)
394
- await pool.markAsMined([
395
- tx.getTxHash()
396
- ], minedBlockHeader);
397
- // Should be back to mined status
398
- await expect(pool.getTxStatus(tx.getTxHash())).resolves.toEqual('mined');
399
- await expect(pool.getTxByHash(tx.getTxHash())).resolves.toBeDefined();
400
- });
401
- });
402
- }
@@ -1,270 +0,0 @@
1
- # Transaction Pool (Mempool)
2
-
3
- This module implements the transaction pool (mempool) for the Aztec P2P network. The mempool holds unconfirmed transactions awaiting inclusion in a block.
4
-
5
- ## Overview
6
-
7
- The transaction pool serves as a staging area for transactions before they are included in blocks. It manages the lifecycle of transactions from initial submission through mining, handling duplicates, priority ordering, and eviction of invalid or low-priority transactions.
8
-
9
- ## Interface: `TxPool`
10
-
11
- The [`TxPool`](tx_pool.ts) interface defines the contract that all transaction pool implementations must fulfill:
12
-
13
- ### Transaction Lifecycle
14
-
15
- The lifecycle of transactions in the pool is summarised in the following table:
16
-
17
- | State | Meaning | Possible Future States |
18
- | --- | --- | --- |
19
- | Pending | Available to be added to a block, can be evicted | Protected, Mined, Soft Deleted |
20
- | Protected | Added to a proposal, must not be evicted | Mined, Pending |
21
- | Mined | Confirmed as added to a block | Soft Deleted, Pending |
22
- | Soft Deleted | Awaiting full deletion once state has been finalised on L1 | Pending, Deleted |
23
- | Deleted | Removed from the pool | N/A |
24
-
25
- **Note on why Soft Delete:**
26
- Mined transactions are soft-deleted rather than permanently removed to support:
27
- 1. Reorg handling — If a chain reorganization occurs, soft-deleted transactions are still available in the mempool
28
- 2. Slash condition detection — The epoch prune watcher needs access to transactions from pruned epochs to correctly identify data withholding slash conditions. Without soft-delete, transactions invalidated by reorgs (e.g., built on removed blocks) would be lost, causing false positives for data withholding violations.
29
-
30
- Mined transactions are permanently deleted via `cleanupDeletedMinedTxs()` once their original block is finalized on L1, ensuring theyremain available during the uncertainty window.
31
- Alternatively, mined transactions can be permanently deleted immediately by passing the `permanent: true` option to `deleteTxs()`.
32
-
33
- #### Transaction Lifecycle Methods
34
-
35
- | Method | Description |
36
- |--------|-------------|
37
- | `addTxs(txs, opts?)` | Adds transactions to the pool. Duplicates and nullifier conflicts are handled. Returns count of newly added txs. |
38
- | `deleteTxs(txHashes, opts?)` | Removes transactions from the pool. Supports soft-delete for mined txs. |
39
- | `markAsMined(txHashes, blockHeader)` | Marks transactions as included in a block. |
40
- | `markMinedAsPending(txHashes, blockNumber)` | Reverts mined transactions to pending (used during reorgs). |
41
- | `getArchivedTxByHash(txHash)` | Retrieves archived (historical) transactions. |
42
- | `getTxStatus(txHash)` | Returns status: `'pending'`, `'mined'`, `'deleted'`, or `undefined`. |
43
-
44
- ### Transaction Fetching
45
-
46
- | Method | Description |
47
- |--------|-------------|
48
- | `hasTx(txHash)` / `hasTxs(txHashes)` | Checks if transaction(s) exist in the pool. |
49
- | `getTxByHash(txHash)` | Retrieves a transaction by its hash. |
50
- | `getTxsByHash(txHashes)` | Batch retrieval of transactions by hash. |
51
- | `getAllTxs()` / `getAllTxHashes()` | Returns all transactions or their hashes. |
52
- | `getPendingTxHashes()` | Returns pending tx hashes **sorted by priority** (highest first). |
53
- | `getPendingTxCount()` | Returns count of pending transactions. |
54
- | `getMinedTxHashes()` | Returns mined tx hashes with their block numbers. |
55
-
56
- ### Pool Management
57
-
58
- | Method | Description |
59
- |--------|-------------|
60
- | `updateConfig(config)` | Updates pool configuration (max size, archive limit). |
61
- | `markTxsAsNonEvictable(txHashes)` | Protects transactions from eviction. |
62
- | `clearNonEvictableTxs()` | Clears non-evictable flag from all transactions. |
63
- | `cleanupDeletedMinedTxs(blockNumber)` | Permanently removes soft-deleted txs from blocks ≤ blockNumber. |
64
- | `isEmpty()` | Checks if the pool has no transactions. |
65
-
66
- ### Events
67
-
68
- The pool emits a `txs-added` event when new transactions are successfully added, allowing subscribers to react to pool changes.
69
-
70
- ## `AztecKVTxPool`
71
-
72
- The [`AztecKVTxPool`](aztec_kv_tx_pool.ts) is the production-grade implementation backed by a persistent key-value store. It provides:
73
-
74
- - **Persistent storage** via `AztecAsyncKVStore`
75
- - **Multiple indexes** for efficient queries
76
- - **Automatic eviction** of invalid and low-priority transactions
77
- - **Transaction archival** for historical lookups
78
- - **Soft-delete semantics** for mined transactions
79
-
80
- #### Storage Structure
81
-
82
- The pool maintains several KV maps and indexes:
83
-
84
- | Store | Purpose |
85
- |-------|---------|
86
- | `#txs` | Primary storage: tx hash → serialized tx buffer |
87
- | `#minedTxHashToBlock` | Index of mined txs: tx hash → block number |
88
- | `#pendingTxPriorityToHash` | Priority-ordered index of pending txs |
89
- | `#deletedMinedTxHashes` | Soft-deleted mined txs: tx hash → original block number |
90
- | `#blockToDeletedMinedTxHash` | Reverse index for cleanup: block → deleted tx hashes |
91
- | `#txHashToHistoricalBlockHeaderHash` | Anchor block reference for each tx |
92
- | `#historicalHeaderToTxHash` | Index from historical block → tx hashes |
93
- | `#feePayerToTxHash` | Index from fee payer address → tx hashes |
94
- | `#pendingNullifierToTxHash` | Index from nullifier → tx hash |
95
- | `#archivedTxs` | Archived transactions for historical lookup |
96
-
97
- #### In-Memory Caches
98
-
99
- | Cache | Purpose |
100
- |-------|---------|
101
- | `#pendingTxs` | Hydrated pending transactions for fast access |
102
- | `#nonEvictableTxs` | Set of tx hashes protected from eviction |
103
-
104
- ## Transaction Priority
105
-
106
- Transactions are prioritized based on their **total priority fees** (see [`priority.ts`](priority.ts)):
107
-
108
- ```typescript
109
- priorityFee = maxPriorityFeesPerGas.feePerDaGas + maxPriorityFeesPerGas.feePerL2Gas
110
- ```
111
-
112
- The priority is stored as a hex string derived from a 32-byte buffer representation of the fee amount, enabling lexicographic ordering in the KV store. Pending transactions are returned in **descending priority order** (highest fees first).
113
-
114
- ## Transaction Lifecycle in AztecKVTxPool
115
-
116
- ### 1. Adding Transactions
117
-
118
- When `addTxs()` is called:
119
-
120
- 1. Check for duplicates (skip if tx already exists)
121
- 2. Check for nullifier conflicts (see Nullifier Deduplication below)
122
- 3. Store the serialized tx in `#txs`
123
- 4. Index the tx by its anchor block hash
124
- 5. If not already mined, add to pending indexes:
125
- - Priority-to-hash index (for ordering)
126
- - Historical header index (for reorg handling)
127
- - Fee payer index (for balance validation)
128
- - Nullifier-to-tx-hash index (for conflict detection)
129
- 6. Record metrics
130
- 7. Trigger eviction rules for `TXS_ADDED` event
131
- 8. Emit `txs-added` event
132
-
133
- ### 2. Marking as Mined
134
-
135
- When a block is finalized, `markAsMined()`:
136
-
137
- 1. Move tx from pending to mined status
138
- 2. If previously soft-deleted, restore to mined status
139
- 3. Trigger eviction rules for `BLOCK_MINED` event
140
-
141
- ### 3. Handling Reorgs
142
-
143
- When blocks are pruned, `markMinedAsPending()`:
144
-
145
- 1. Remove tx from mined index
146
- 2. Rehydrate pending indexes
147
- 3. Trigger eviction rules for `CHAIN_PRUNED` event
148
-
149
- ### 4. Deleting Transactions
150
-
151
- The `deleteTxs()` method handles two cases:
152
-
153
- - **Pending transactions**: Permanently deleted (transactions and all indexes to the transaction)
154
- - **Mined transactions**: Soft-deleted by default (moved to `#deletedMinedTxHashes`), with option for permanent deletion
155
-
156
- Soft-deleted mined transactions are retained for potential future reference and can be permanently cleaned up later via `cleanupDeletedMinedTxs()`.
157
-
158
- ### Nullifier Deduplication
159
-
160
- The pool prevents nullifier spam attacks by ensuring only one pending transaction can reference each unique nullifier. When an incoming transaction shares nullifiers with existing pending transactions:
161
-
162
- - **Higher fee wins**: If the incoming tx has a higher priority fee than ALL conflicting txs, those conflicting txs are replaced
163
- - **Existing tx wins on tie**: If any conflicting tx has an equal or higher fee, the incoming tx is rejected
164
- - **Partial overlap counts**: Even a single shared nullifier triggers conflict resolution
165
-
166
- This is enforced at entry time via the `#pendingNullifierToTxHash` index, which maps each nullifier to the tx hash that references it. The index is maintained atomically with tx additions and removals.
167
-
168
- ## Eviction System
169
-
170
- The eviction system automatically removes invalid or low-priority transactions based on configurable rules. See the [`eviction/`](eviction/) subdirectory for implementation details.
171
-
172
- ### Architecture
173
-
174
- ```
175
- ┌─────────────────────────────────────────────────────────────────┐
176
- │ EvictionManager │
177
- │ Orchestrates eviction rules based on pool events │
178
- ├─────────────────────────────────────────────────────────────────┤
179
- │ │
180
- │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
181
- │ │EvictionRule #1 │ │EvictionRule #2 │ │EvictionRule #N │ │
182
- │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
183
- │ │
184
- └─────────────────────────────────────────────────────────────────┘
185
-
186
-
187
- ┌─────────────────┐
188
- │ TxPoolOperations│
189
- │ (interface) │
190
- └─────────────────┘
191
- ```
192
-
193
- The [`EvictionManager`](eviction/eviction_manager.ts) coordinates eviction by:
194
-
195
- 1. Registering multiple `EvictionRule` implementations
196
- 2. Calling each rule when tx pool events occur
197
- 3. Propagating configuration updates to all rules
198
-
199
- ### Eviction Events
200
-
201
- | Event | Trigger | Purpose |
202
- |-------|---------|---------|
203
- | `TXS_ADDED` | New transactions added | Enforce pool size limits |
204
- | `BLOCK_MINED` | Block finalized | Remove invalidated transactions |
205
- | `CHAIN_PRUNED` | Chain reorganization | Remove txs referencing pruned blocks and re-evaluate fee payer balances |
206
-
207
- ### Eviction Rules
208
-
209
- #### 1. `InvalidTxsAfterMiningRule`
210
-
211
- **Triggers on:** `BLOCK_MINED`
212
-
213
- Evicts transactions that become invalid after a block is mined:
214
-
215
- - Duplicate nullifiers: Txs with nullifiers already included in the mined block
216
- - Expired transactions: Txs with `expirationTimestamp` ≤ mined block timestamp
217
-
218
- #### 2. `InvalidTxsAfterReorgRule`
219
-
220
- **Triggers on:** `CHAIN_PRUNED`
221
-
222
- Evicts transactions that reference blocks no longer in the canonical chain:
223
-
224
- - Checks each pending tx's anchor block hash against the archive tree
225
- - Removes txs whose anchor blocks are not found (pruned)
226
-
227
- #### 3. `FeePayerBalanceEvictionRule`
228
-
229
- **Triggers on:** `TXS_ADDED`, `BLOCK_MINED`, `CHAIN_PRUNED`
230
-
231
- Evicts low-priority transactions when a fee payer's pending fee limits exceed their Fee Juice balance:
232
-
233
- - Evaluates transactions in priority order so higher-priority claims can fund lower-priority spends
234
- - Accounts for self-funding claims made during setup
235
- - Removes evictable txs that do not fit within the running balance
236
-
237
- #### 4. `LowPriorityEvictionRule`
238
-
239
- **Triggers on:** `TXS_ADDED`
240
-
241
- Enforces maximum pool size by evicting lowest-priority (by fee) transactions:
242
-
243
- - Configured via `maxPendingTxCount` option (0 = disabled)
244
- - Uses `getLowestPriorityEvictable()` to find txs to evict
245
-
246
- ### Non-Evictable Transactions
247
-
248
- Transactions can be marked as non-evictable via `markTxsAsNonEvictable()`. This protects them from all eviction rules, typically used during block building to ensure transactions being processed aren't evicted mid-operation. The flag is cleared after block processing via `clearNonEvictableTxs()`.
249
- The `clearNonEvictableTxs` is called upon getting new L2 block.
250
-
251
- ## Configuration
252
-
253
- The pool accepts configuration via `TxPoolOptions`:
254
-
255
- ```typescript
256
- type TxPoolOptions = {
257
- maxPendingTxCount?: number; // Max pending txs (0 = unlimited)
258
- archivedTxLimit?: number; // Number of archived txs to retain
259
- };
260
- ```
261
-
262
- Configuration can be updated at runtime via `updateConfig()`.
263
-
264
- ## Telemetry
265
-
266
- The pool integrates with the telemetry system to report:
267
-
268
- - Transaction counts (pending vs mined)
269
- - Transaction sizes
270
- - Store size estimates