@aztec/archiver 0.0.1-commit.e558bd1c → 0.0.1-commit.e5a3663dd

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (126) hide show
  1. package/README.md +12 -6
  2. package/dest/archiver.d.ts +26 -15
  3. package/dest/archiver.d.ts.map +1 -1
  4. package/dest/archiver.js +161 -153
  5. package/dest/config.d.ts +5 -3
  6. package/dest/config.d.ts.map +1 -1
  7. package/dest/config.js +16 -4
  8. package/dest/errors.d.ts +61 -10
  9. package/dest/errors.d.ts.map +1 -1
  10. package/dest/errors.js +88 -14
  11. package/dest/factory.d.ts +6 -7
  12. package/dest/factory.d.ts.map +1 -1
  13. package/dest/factory.js +40 -32
  14. package/dest/index.d.ts +11 -3
  15. package/dest/index.d.ts.map +1 -1
  16. package/dest/index.js +10 -2
  17. package/dest/l1/bin/retrieve-calldata.js +32 -28
  18. package/dest/l1/calldata_retriever.d.ts +74 -50
  19. package/dest/l1/calldata_retriever.d.ts.map +1 -1
  20. package/dest/l1/calldata_retriever.js +197 -260
  21. package/dest/l1/data_retrieval.d.ts +26 -17
  22. package/dest/l1/data_retrieval.d.ts.map +1 -1
  23. package/dest/l1/data_retrieval.js +42 -47
  24. package/dest/l1/spire_proposer.d.ts +5 -5
  25. package/dest/l1/spire_proposer.d.ts.map +1 -1
  26. package/dest/l1/spire_proposer.js +9 -17
  27. package/dest/l1/validate_historical_logs.d.ts +23 -0
  28. package/dest/l1/validate_historical_logs.d.ts.map +1 -0
  29. package/dest/l1/validate_historical_logs.js +108 -0
  30. package/dest/modules/contract_data_source_adapter.d.ts +25 -0
  31. package/dest/modules/contract_data_source_adapter.d.ts.map +1 -0
  32. package/dest/modules/contract_data_source_adapter.js +42 -0
  33. package/dest/modules/data_source_base.d.ts +27 -14
  34. package/dest/modules/data_source_base.d.ts.map +1 -1
  35. package/dest/modules/data_source_base.js +98 -125
  36. package/dest/modules/data_store_updater.d.ts +37 -17
  37. package/dest/modules/data_store_updater.d.ts.map +1 -1
  38. package/dest/modules/data_store_updater.js +155 -112
  39. package/dest/modules/instrumentation.d.ts +21 -3
  40. package/dest/modules/instrumentation.d.ts.map +1 -1
  41. package/dest/modules/instrumentation.js +41 -8
  42. package/dest/modules/l1_synchronizer.d.ts +13 -11
  43. package/dest/modules/l1_synchronizer.d.ts.map +1 -1
  44. package/dest/modules/l1_synchronizer.js +355 -182
  45. package/dest/modules/validation.d.ts +4 -3
  46. package/dest/modules/validation.d.ts.map +1 -1
  47. package/dest/modules/validation.js +6 -6
  48. package/dest/store/block_store.d.ts +107 -31
  49. package/dest/store/block_store.d.ts.map +1 -1
  50. package/dest/store/block_store.js +477 -141
  51. package/dest/store/contract_class_store.d.ts +17 -4
  52. package/dest/store/contract_class_store.d.ts.map +1 -1
  53. package/dest/store/contract_class_store.js +24 -68
  54. package/dest/store/contract_instance_store.d.ts +28 -1
  55. package/dest/store/contract_instance_store.d.ts.map +1 -1
  56. package/dest/store/contract_instance_store.js +37 -2
  57. package/dest/store/data_stores.d.ts +68 -0
  58. package/dest/store/data_stores.d.ts.map +1 -0
  59. package/dest/store/data_stores.js +50 -0
  60. package/dest/store/function_names_cache.d.ts +17 -0
  61. package/dest/store/function_names_cache.d.ts.map +1 -0
  62. package/dest/store/function_names_cache.js +30 -0
  63. package/dest/store/l2_tips_cache.d.ts +20 -0
  64. package/dest/store/l2_tips_cache.d.ts.map +1 -0
  65. package/dest/store/l2_tips_cache.js +109 -0
  66. package/dest/store/log_store.d.ts +6 -3
  67. package/dest/store/log_store.d.ts.map +1 -1
  68. package/dest/store/log_store.js +95 -20
  69. package/dest/store/message_store.d.ts +11 -1
  70. package/dest/store/message_store.d.ts.map +1 -1
  71. package/dest/store/message_store.js +51 -9
  72. package/dest/test/fake_l1_state.d.ts +25 -1
  73. package/dest/test/fake_l1_state.d.ts.map +1 -1
  74. package/dest/test/fake_l1_state.js +166 -32
  75. package/dest/test/mock_archiver.d.ts +1 -1
  76. package/dest/test/mock_archiver.d.ts.map +1 -1
  77. package/dest/test/mock_archiver.js +3 -2
  78. package/dest/test/mock_l1_to_l2_message_source.d.ts +1 -1
  79. package/dest/test/mock_l1_to_l2_message_source.d.ts.map +1 -1
  80. package/dest/test/mock_l1_to_l2_message_source.js +2 -1
  81. package/dest/test/mock_l2_block_source.d.ts +35 -5
  82. package/dest/test/mock_l2_block_source.d.ts.map +1 -1
  83. package/dest/test/mock_l2_block_source.js +182 -89
  84. package/dest/test/mock_structs.d.ts +4 -1
  85. package/dest/test/mock_structs.d.ts.map +1 -1
  86. package/dest/test/mock_structs.js +13 -1
  87. package/dest/test/noop_l1_archiver.d.ts +7 -4
  88. package/dest/test/noop_l1_archiver.d.ts.map +1 -1
  89. package/dest/test/noop_l1_archiver.js +14 -8
  90. package/package.json +13 -13
  91. package/src/archiver.ts +199 -174
  92. package/src/config.ts +23 -2
  93. package/src/errors.ts +133 -22
  94. package/src/factory.ts +53 -30
  95. package/src/index.ts +18 -2
  96. package/src/l1/README.md +25 -68
  97. package/src/l1/bin/retrieve-calldata.ts +40 -27
  98. package/src/l1/calldata_retriever.ts +261 -379
  99. package/src/l1/data_retrieval.ts +58 -69
  100. package/src/l1/spire_proposer.ts +7 -15
  101. package/src/l1/validate_historical_logs.ts +140 -0
  102. package/src/modules/contract_data_source_adapter.ts +59 -0
  103. package/src/modules/data_source_base.ts +142 -144
  104. package/src/modules/data_store_updater.ts +187 -141
  105. package/src/modules/instrumentation.ts +56 -9
  106. package/src/modules/l1_synchronizer.ts +463 -218
  107. package/src/modules/validation.ts +10 -9
  108. package/src/store/block_store.ts +587 -177
  109. package/src/store/contract_class_store.ts +31 -103
  110. package/src/store/contract_instance_store.ts +51 -5
  111. package/src/store/data_stores.ts +108 -0
  112. package/src/store/function_names_cache.ts +37 -0
  113. package/src/store/l2_tips_cache.ts +134 -0
  114. package/src/store/log_store.ts +128 -32
  115. package/src/store/message_store.ts +60 -10
  116. package/src/structs/inbox_message.ts +1 -1
  117. package/src/test/fake_l1_state.ts +213 -42
  118. package/src/test/mock_archiver.ts +3 -2
  119. package/src/test/mock_l1_to_l2_message_source.ts +1 -0
  120. package/src/test/mock_l2_block_source.ts +230 -82
  121. package/src/test/mock_structs.ts +20 -6
  122. package/src/test/noop_l1_archiver.ts +16 -8
  123. package/dest/store/kv_archiver_store.d.ts +0 -340
  124. package/dest/store/kv_archiver_store.d.ts.map +0 -1
  125. package/dest/store/kv_archiver_store.js +0 -446
  126. package/src/store/kv_archiver_store.ts +0 -639
package/src/errors.ts CHANGED
@@ -1,29 +1,17 @@
1
+ import type { BlockNumber, CheckpointNumber } from '@aztec/foundation/branded-types';
1
2
  import type { Fr } from '@aztec/foundation/schemas';
2
3
 
3
4
  export class NoBlobBodiesFoundError extends Error {
4
5
  constructor(l2BlockNum: number) {
5
6
  super(`No blob bodies found for block ${l2BlockNum}`);
6
- }
7
- }
8
-
9
- export class InitialBlockNumberNotSequentialError extends Error {
10
- constructor(
11
- public readonly newBlockNumber: number,
12
- public readonly previousBlockNumber: number | undefined,
13
- ) {
14
- super(
15
- `Cannot insert new block ${newBlockNumber} given previous block number in store is ${
16
- previousBlockNumber ?? 'undefined'
17
- }`,
18
- );
7
+ this.name = 'NoBlobBodiesFoundError';
19
8
  }
20
9
  }
21
10
 
22
11
  export class BlockNumberNotSequentialError extends Error {
23
12
  constructor(newBlockNumber: number, previous: number | undefined) {
24
- super(
25
- `Cannot insert new block ${newBlockNumber} given previous block number in batch is ${previous ?? 'undefined'}`,
26
- );
13
+ super(`Cannot insert new block ${newBlockNumber} given previous block number is ${previous ?? 'undefined'}`);
14
+ this.name = 'BlockNumberNotSequentialError';
27
15
  }
28
16
  }
29
17
 
@@ -37,22 +25,29 @@ export class InitialCheckpointNumberNotSequentialError extends Error {
37
25
  previousCheckpointNumber ?? 'undefined'
38
26
  }`,
39
27
  );
28
+ this.name = 'InitialCheckpointNumberNotSequentialError';
40
29
  }
41
30
  }
42
31
 
43
- export class CheckpointNumberNotSequentialError extends Error {
44
- constructor(newCheckpointNumber: number, previous: number | undefined) {
32
+ export class BlockCheckpointNumberNotSequentialError extends Error {
33
+ constructor(
34
+ blockNumber: BlockNumber,
35
+ blockCheckpointNumber: CheckpointNumber,
36
+ previous: CheckpointNumber | undefined,
37
+ ) {
45
38
  super(
46
- `Cannot insert new checkpoint ${newCheckpointNumber} given previous checkpoint number in batch is ${previous ?? 'undefined'}`,
39
+ `Cannot insert new block ${blockNumber} for checkpoint ${blockCheckpointNumber} given previous checkpoint number is ${previous ?? 'undefined'}`,
47
40
  );
41
+ this.name = 'BlockCheckpointNumberNotSequentialError';
48
42
  }
49
43
  }
50
44
 
51
- export class CheckpointNumberNotConsistentError extends Error {
52
- constructor(newCheckpointNumber: number, previous: number | undefined) {
45
+ export class CheckpointNumberNotSequentialError extends Error {
46
+ constructor(newCheckpointNumber: CheckpointNumber, previous: CheckpointNumber | undefined) {
53
47
  super(
54
- `Cannot insert block for new checkpoint ${newCheckpointNumber} given previous block was checkpoint ${previous ?? 'undefined'}`,
48
+ `Cannot insert new checkpoint ${newCheckpointNumber} given previous checkpoint number is ${previous ?? 'undefined'}`,
55
49
  );
50
+ this.name = 'CheckpointNumberNotSequentialError';
56
51
  }
57
52
  }
58
53
 
@@ -61,6 +56,7 @@ export class BlockIndexNotSequentialError extends Error {
61
56
  super(
62
57
  `Cannot insert new block at checkpoint index ${newBlockIndex} given previous block index is ${previousBlockIndex ?? 'undefined'}`,
63
58
  );
59
+ this.name = 'BlockIndexNotSequentialError';
64
60
  }
65
61
  }
66
62
 
@@ -74,21 +70,136 @@ export class BlockArchiveNotConsistentError extends Error {
74
70
  super(
75
71
  `Cannot insert new block number ${newBlockNumber} with archive ${newBlockArchive.toString()} previous block number is ${previousBlockNumber ?? 'undefined'}, previous archive is ${previousBlockArchive?.toString() ?? 'undefined'}`,
76
72
  );
73
+ this.name = 'BlockArchiveNotConsistentError';
77
74
  }
78
75
  }
79
76
 
80
77
  export class CheckpointNotFoundError extends Error {
81
78
  constructor(checkpointNumber: number) {
82
79
  super(`Failed to find expected checkpoint number ${checkpointNumber}`);
80
+ this.name = 'CheckpointNotFoundError';
83
81
  }
84
82
  }
85
83
 
86
84
  export class BlockNotFoundError extends Error {
87
85
  constructor(blockNumber: number) {
88
86
  super(`Failed to find expected block number ${blockNumber}`);
87
+ this.name = 'BlockNotFoundError';
88
+ }
89
+ }
90
+
91
+ /** Thrown when a proposed block matches a block that was already checkpointed. This is expected for late proposals. */
92
+ export class BlockAlreadyCheckpointedError extends Error {
93
+ constructor(public readonly blockNumber: number) {
94
+ super(`Block ${blockNumber} has already been checkpointed with the same content`);
95
+ this.name = 'BlockAlreadyCheckpointedError';
96
+ }
97
+ }
98
+
99
+ /** Thrown when logs are added for a tag whose last stored log has a higher block number than the new log. */
100
+ export class OutOfOrderLogInsertionError extends Error {
101
+ constructor(
102
+ public readonly logType: 'private' | 'public',
103
+ public readonly tag: string,
104
+ public readonly lastBlockNumber: number,
105
+ public readonly newBlockNumber: number,
106
+ ) {
107
+ super(
108
+ `Out-of-order ${logType} log insertion for tag ${tag}: ` +
109
+ `last existing log is from block ${lastBlockNumber} but new log is from block ${newBlockNumber}`,
110
+ );
111
+ this.name = 'OutOfOrderLogInsertionError';
112
+ }
113
+ }
114
+
115
+ /** Thrown when L1 to L2 messages are requested for a checkpoint whose message tree hasn't been sealed yet. */
116
+ export class L1ToL2MessagesNotReadyError extends Error {
117
+ constructor(
118
+ public readonly checkpointNumber: number,
119
+ public readonly inboxTreeInProgress: bigint,
120
+ ) {
121
+ super(
122
+ `Cannot get L1 to L2 messages for checkpoint ${checkpointNumber}: ` +
123
+ `inbox tree in progress is ${inboxTreeInProgress}, messages not yet sealed`,
124
+ );
125
+ this.name = 'L1ToL2MessagesNotReadyError';
126
+ }
127
+ }
128
+
129
+ /** Thrown when a proposed checkpoint number is stale (already processed). */
130
+ export class ProposedCheckpointStaleError extends Error {
131
+ constructor(
132
+ public readonly proposedCheckpointNumber: number,
133
+ public readonly currentProposedNumber: number,
134
+ ) {
135
+ super(`Stale proposed checkpoint ${proposedCheckpointNumber}: current proposed is ${currentProposedNumber}`);
136
+ this.name = 'ProposedCheckpointStaleError';
137
+ }
138
+ }
139
+
140
+ /** Thrown when a proposed checkpoint number is not the expected latestTip + 1. */
141
+ export class ProposedCheckpointNotSequentialError extends Error {
142
+ constructor(
143
+ public readonly proposedCheckpointNumber: number,
144
+ public readonly latestTipNumber: number,
145
+ ) {
146
+ super(
147
+ `Proposed checkpoint ${proposedCheckpointNumber} is not sequential: expected ${latestTipNumber + 1} (latest tip + 1, where tip is highest of confirmed or pending)`,
148
+ );
149
+ this.name = 'ProposedCheckpointNotSequentialError';
150
+ }
151
+ }
152
+
153
+ /** Thrown when a proposed checkpoint or block L2 slot has already expired on L1. */
154
+ export class BlockOrCheckpointSlotExpiredError extends Error {
155
+ constructor(
156
+ public readonly slot: number,
157
+ public readonly nextSlotStart: bigint,
158
+ public readonly l1TimestampSynced: bigint | undefined,
159
+ ) {
160
+ super(
161
+ `Checkpoint or block for slot ${slot} is expired: L1 synced to ${l1TimestampSynced} which is past the next slot start ${nextSlotStart}. ` +
162
+ `If the checkpoint still lands via a late L1 tx, the archiver will pick it up via normal L1-sync (not the pending-queue shortcut).`,
163
+ );
164
+ this.name = 'BlockOrCheckpointSlotExpiredError';
165
+ }
166
+ }
167
+
168
+ /** Thrown when attempting to promote a proposed checkpoint but no proposed checkpoint exists in the store. */
169
+ export class NoProposedCheckpointToPromoteError extends Error {
170
+ constructor() {
171
+ super('Cannot promote proposed checkpoint: no proposed checkpoint exists');
172
+ this.name = 'NoProposedCheckpointToPromoteError';
173
+ }
174
+ }
175
+
176
+ /** Thrown when the archive root of the proposed checkpoint does not match the expected one. */
177
+ export class ProposedCheckpointArchiveRootMismatchError extends Error {
178
+ constructor(
179
+ public readonly expectedArchiveRoot: Fr,
180
+ public readonly actualArchiveRoot: Fr,
181
+ ) {
182
+ super(
183
+ `Cannot promote proposed checkpoint: archive root mismatch (expected ${expectedArchiveRoot}, got ${actualArchiveRoot})`,
184
+ );
185
+ this.name = 'ProposedCheckpointArchiveRootMismatchError';
186
+ }
187
+ }
188
+
189
+ /** Thrown when the proposed checkpoint does not directly follow the latest confirmed checkpoint. */
190
+ export class ProposedCheckpointPromotionNotSequentialError extends Error {
191
+ constructor(
192
+ public readonly proposedCheckpointNumber: number,
193
+ public readonly latestCheckpointNumber: number,
194
+ ) {
195
+ super(
196
+ `Cannot promote proposed checkpoint: not sequential (latest ${latestCheckpointNumber}, proposed ${proposedCheckpointNumber})`,
197
+ );
198
+ this.name = 'ProposedCheckpointPromotionNotSequentialError';
89
199
  }
90
200
  }
91
201
 
202
+ /** Thrown when a proposed block conflicts with an already checkpointed block (different content). */
92
203
  export class CannotOverwriteCheckpointedBlockError extends Error {
93
204
  constructor(
94
205
  public readonly blockNumber: number,
package/src/factory.ts CHANGED
@@ -1,5 +1,6 @@
1
1
  import { EpochCache } from '@aztec/epoch-cache';
2
2
  import { createEthereumChain } from '@aztec/ethereum/chain';
3
+ import { makeL1HttpTransport } from '@aztec/ethereum/client';
3
4
  import { InboxContract, RollupContract } from '@aztec/ethereum/contracts';
4
5
  import type { ViemPublicDebugClient } from '@aztec/ethereum/types';
5
6
  import { BlockNumber } from '@aztec/foundation/branded-types';
@@ -7,38 +8,37 @@ import { Buffer32 } from '@aztec/foundation/buffer';
7
8
  import { merge } from '@aztec/foundation/collection';
8
9
  import { Fr } from '@aztec/foundation/curves/bn254';
9
10
  import { DateProvider } from '@aztec/foundation/timer';
10
- import type { DataStoreConfig } from '@aztec/kv-store/config';
11
11
  import { createStore } from '@aztec/kv-store/lmdb-v2';
12
12
  import { protocolContractNames } from '@aztec/protocol-contracts';
13
13
  import { BundledProtocolContractsProvider } from '@aztec/protocol-contracts/providers/bundle';
14
14
  import { FunctionType, decodeFunctionSignature } from '@aztec/stdlib/abi';
15
15
  import type { ArchiverEmitter } from '@aztec/stdlib/block';
16
- import { type ContractClassPublic, computePublicBytecodeCommitment } from '@aztec/stdlib/contract';
17
- import type { L1RollupConstants } from '@aztec/stdlib/epoch-helpers';
16
+ import { type ContractClassPublicWithCommitment, computePublicBytecodeCommitment } from '@aztec/stdlib/contract';
17
+ import type { DataStoreConfig } from '@aztec/stdlib/kv-store';
18
18
  import { getTelemetryClient } from '@aztec/telemetry-client';
19
19
 
20
20
  import { EventEmitter } from 'events';
21
- import { createPublicClient, fallback, http } from 'viem';
21
+ import { createPublicClient } from 'viem';
22
22
 
23
23
  import { Archiver, type ArchiverDeps } from './archiver.js';
24
24
  import { type ArchiverConfig, mapArchiverConfig } from './config.js';
25
25
  import { ArchiverInstrumentation } from './modules/instrumentation.js';
26
26
  import { ArchiverL1Synchronizer } from './modules/l1_synchronizer.js';
27
- import { ARCHIVER_DB_VERSION, KVArchiverDataStore } from './store/kv_archiver_store.js';
27
+ import { ARCHIVER_DB_VERSION, type ArchiverDataStores, createArchiverDataStores } from './store/data_stores.js';
28
+ import { L2TipsCache } from './store/l2_tips_cache.js';
28
29
 
29
30
  export const ARCHIVER_STORE_NAME = 'archiver';
30
31
 
31
32
  /** Creates an archiver store. */
32
33
  export async function createArchiverStore(
33
34
  userConfig: Pick<ArchiverConfig, 'archiverStoreMapSizeKb' | 'maxLogs'> & DataStoreConfig,
34
- l1Constants: Pick<L1RollupConstants, 'epochDuration'>,
35
- ) {
35
+ ): Promise<ArchiverDataStores> {
36
36
  const config = {
37
37
  ...userConfig,
38
38
  dataStoreMapSizeKb: userConfig.archiverStoreMapSizeKb ?? userConfig.dataStoreMapSizeKb,
39
39
  };
40
40
  const store = await createStore(ARCHIVER_STORE_NAME, ARCHIVER_DB_VERSION, config);
41
- return new KVArchiverDataStore(store, config.maxLogs, l1Constants);
41
+ return createArchiverDataStores(store, { logsMaxPageSize: config.maxLogs });
42
42
  }
43
43
 
44
44
  /**
@@ -53,14 +53,15 @@ export async function createArchiver(
53
53
  deps: ArchiverDeps,
54
54
  opts: { blockUntilSync: boolean } = { blockUntilSync: true },
55
55
  ): Promise<Archiver> {
56
- const archiverStore = await createArchiverStore(config, { epochDuration: config.aztecEpochDuration });
56
+ const archiverStore = await createArchiverStore(config);
57
57
  await registerProtocolContracts(archiverStore);
58
58
 
59
59
  // Create Ethereum clients
60
60
  const chain = createEthereumChain(config.l1RpcUrls, config.l1ChainId);
61
+ const httpTimeout = config.l1HttpTimeoutMS;
61
62
  const publicClient = createPublicClient({
62
63
  chain: chain.chainInfo,
63
- transport: fallback(config.l1RpcUrls.map(url => http(url, { batch: false }))),
64
+ transport: makeL1HttpTransport(config.l1RpcUrls, { timeout: httpTimeout }),
64
65
  pollingInterval: config.viemPollingIntervalMS,
65
66
  });
66
67
 
@@ -68,7 +69,7 @@ export async function createArchiver(
68
69
  const debugRpcUrls = config.l1DebugRpcUrls.length > 0 ? config.l1DebugRpcUrls : config.l1RpcUrls;
69
70
  const debugClient = createPublicClient({
70
71
  chain: chain.chainInfo,
71
- transport: fallback(debugRpcUrls.map(url => http(url, { batch: false }))),
72
+ transport: makeL1HttpTransport(debugRpcUrls, { timeout: httpTimeout }),
72
73
  pollingInterval: config.viemPollingIntervalMS,
73
74
  }) as ViemPublicDebugClient;
74
75
 
@@ -77,14 +78,23 @@ export async function createArchiver(
77
78
  const inbox = new InboxContract(publicClient, config.l1Contracts.inboxAddress);
78
79
 
79
80
  // Fetch L1 constants from rollup contract
80
- const [l1StartBlock, l1GenesisTime, proofSubmissionEpochs, genesisArchiveRoot, slashingProposerAddress] =
81
- await Promise.all([
82
- rollup.getL1StartBlock(),
83
- rollup.getL1GenesisTime(),
84
- rollup.getProofSubmissionEpochs(),
85
- rollup.getGenesisArchiveTreeRoot(),
86
- rollup.getSlashingProposerAddress(),
87
- ] as const);
81
+ const [
82
+ l1StartBlock,
83
+ l1GenesisTime,
84
+ proofSubmissionEpochs,
85
+ genesisArchiveRoot,
86
+ slashingProposerAddress,
87
+ targetCommitteeSize,
88
+ rollupManaLimit,
89
+ ] = await Promise.all([
90
+ rollup.getL1StartBlock(),
91
+ rollup.getL1GenesisTime(),
92
+ rollup.getProofSubmissionEpochs(),
93
+ rollup.getGenesisArchiveTreeRoot(),
94
+ rollup.getSlashingProposerAddress(),
95
+ rollup.getTargetCommitteeSize(),
96
+ rollup.getManaLimit(),
97
+ ] as const);
88
98
 
89
99
  const l1StartBlockHash = await publicClient
90
100
  .getBlock({ blockNumber: l1StartBlock, includeTransactions: false })
@@ -100,7 +110,9 @@ export async function createArchiver(
100
110
  slotDuration,
101
111
  ethereumSlotDuration,
102
112
  proofSubmissionEpochs: Number(proofSubmissionEpochs),
113
+ targetCommitteeSize,
103
114
  genesisArchiveRoot: Fr.fromString(genesisArchiveRoot.toString()),
115
+ rollupManaLimit: Number(rollupManaLimit),
104
116
  };
105
117
 
106
118
  const archiverConfig = merge(
@@ -109,24 +121,27 @@ export async function createArchiver(
109
121
  batchSize: 100,
110
122
  maxAllowedEthClientDriftSeconds: 300,
111
123
  ethereumAllowNoDebugHosts: false,
124
+ skipHistoricalLogsCheck: false,
112
125
  },
113
126
  mapArchiverConfig(config),
114
127
  );
115
128
 
116
129
  const epochCache = deps.epochCache ?? (await EpochCache.create(config.l1Contracts.rollupAddress, config, deps));
117
130
  const telemetry = deps.telemetry ?? getTelemetryClient();
118
- const instrumentation = await ArchiverInstrumentation.new(telemetry, () => archiverStore.estimateSize());
131
+ const instrumentation = await ArchiverInstrumentation.new(telemetry, () => archiverStore.db.estimateSize());
119
132
 
120
133
  // Create the event emitter that will be shared by archiver and synchronizer
121
134
  const events = new EventEmitter() as ArchiverEmitter;
122
135
 
136
+ // Create L2 tips cache shared by archiver and synchronizer
137
+ const l2TipsCache = new L2TipsCache(archiverStore.blocks);
138
+
123
139
  // Create the L1 synchronizer
124
140
  const synchronizer = new ArchiverL1Synchronizer(
125
141
  publicClient,
126
142
  debugClient,
127
143
  rollup,
128
144
  inbox,
129
- { ...config.l1Contracts, slashingProposerAddress },
130
145
  archiverStore,
131
146
  archiverConfig,
132
147
  deps.blobClient,
@@ -136,6 +151,8 @@ export async function createArchiver(
136
151
  l1Constants,
137
152
  events,
138
153
  instrumentation.tracer,
154
+ l2TipsCache,
155
+ undefined, // log (use default)
139
156
  );
140
157
 
141
158
  const archiver = new Archiver(
@@ -150,31 +167,37 @@ export async function createArchiver(
150
167
  l1Constants,
151
168
  synchronizer,
152
169
  events,
170
+ l2TipsCache,
153
171
  );
154
172
 
155
173
  await archiver.start(opts.blockUntilSync);
156
174
  return archiver;
157
175
  }
158
176
 
159
- /** Registers protocol contracts in the archiver store. */
160
- export async function registerProtocolContracts(store: KVArchiverDataStore) {
177
+ /** Registers protocol contracts in the archiver store. Idempotent — skips contracts that already exist (e.g. on node restart). */
178
+ export async function registerProtocolContracts(stores: ArchiverDataStores) {
161
179
  const blockNumber = 0;
162
180
  for (const name of protocolContractNames) {
163
181
  const provider = new BundledProtocolContractsProvider();
164
182
  const contract = await provider.getProtocolContractArtifact(name);
165
- const contractClassPublic: ContractClassPublic = {
183
+
184
+ // Skip if already registered (happens on node restart with a persisted store).
185
+ if (await stores.contractClasses.getContractClass(contract.contractClass.id)) {
186
+ continue;
187
+ }
188
+
189
+ const publicBytecodeCommitment = await computePublicBytecodeCommitment(contract.contractClass.packedBytecode);
190
+ const contractClassPublic: ContractClassPublicWithCommitment = {
166
191
  ...contract.contractClass,
167
- privateFunctions: [],
168
- utilityFunctions: [],
192
+ publicBytecodeCommitment,
169
193
  };
170
194
 
171
195
  const publicFunctionSignatures = contract.artifact.functions
172
196
  .filter(fn => fn.functionType === FunctionType.PUBLIC)
173
197
  .map(fn => decodeFunctionSignature(fn.name, fn.parameters));
174
198
 
175
- await store.registerContractFunctionSignatures(publicFunctionSignatures);
176
- const bytecodeCommitment = await computePublicBytecodeCommitment(contractClassPublic.packedBytecode);
177
- await store.addContractClasses([contractClassPublic], [bytecodeCommitment], BlockNumber(blockNumber));
178
- await store.addContractInstances([contract.instance], BlockNumber(blockNumber));
199
+ await stores.functionNames.register(publicFunctionSignatures);
200
+ await stores.contractClasses.addContractClasses([contractClassPublic], BlockNumber(blockNumber));
201
+ await stores.contractInstances.addContractInstances([contract.instance], BlockNumber(blockNumber));
179
202
  }
180
203
  }
package/src/index.ts CHANGED
@@ -6,7 +6,23 @@ export * from './modules/data_store_updater.js';
6
6
  export * from './config.js';
7
7
 
8
8
  export { type L1PublishedData } from './structs/published.js';
9
- export { KVArchiverDataStore, ARCHIVER_DB_VERSION } from './store/kv_archiver_store.js';
9
+ export {
10
+ ARCHIVER_DB_VERSION,
11
+ type ArchiverDataStores,
12
+ type ArchiverL1SynchPoint,
13
+ backupArchiverDataStores,
14
+ createArchiverDataStores,
15
+ createContractDataSource,
16
+ getArchiverSynchPoint,
17
+ } from './store/data_stores.js';
18
+ export { FunctionNamesCache } from './store/function_names_cache.js';
19
+ export { ArchiverContractDataSourceAdapter } from './modules/contract_data_source_adapter.js';
20
+ export { BlockStore } from './store/block_store.js';
21
+ export { LogStore } from './store/log_store.js';
22
+ export { MessageStore } from './store/message_store.js';
23
+ export { ContractClassStore } from './store/contract_class_store.js';
10
24
  export { ContractInstanceStore } from './store/contract_instance_store.js';
25
+ export { L2TipsCache } from './store/l2_tips_cache.js';
11
26
 
12
- export { retrieveCheckpointsFromRollup, retrieveL2ProofVerifiedEvents } from './l1/data_retrieval.js';
27
+ export { retrieveL2ProofVerifiedEvents } from './l1/data_retrieval.js';
28
+ export { CalldataRetriever } from './l1/calldata_retriever.js';
package/src/l1/README.md CHANGED
@@ -5,29 +5,27 @@ Modules and classes to handle data retrieval from L1 for the archiver.
5
5
  ## Calldata Retriever
6
6
 
7
7
  The sequencer publisher bundles multiple operations into a single multicall3 transaction for gas
8
- efficiency. A typical transaction includes:
8
+ efficiency. The archiver needs to extract the `propose` calldata from these bundled transactions
9
+ to reconstruct L2 blocks.
9
10
 
10
- 1. Attestation invalidations (if needed): `invalidateBadAttestation`, `invalidateInsufficientAttestations`
11
- 2. Block proposal: `propose` (exactly one per transaction to the rollup contract)
12
- 3. Governance and slashing (if needed): votes, payload creation/execution
11
+ The retriever uses hash matching against `attestationsHash` and `payloadDigest` from the
12
+ `CheckpointProposed` L1 event to verify it has found the correct propose calldata. These hashes
13
+ are always required.
13
14
 
14
- The archiver needs to extract the `propose` calldata from these bundled transactions to reconstruct
15
- L2 blocks. This class needs to handle scenarios where the transaction was submitted via multicall3,
16
- as well as alternative ways for submitting the `propose` call that other clients might use.
15
+ ### Multicall3 Decoding with Hash Matching
17
16
 
18
- ### Multicall3 Validation and Decoding
19
-
20
- First attempt to decode the transaction as a multicall3 `aggregate3` call with validation:
17
+ First attempt to decode the transaction as a multicall3 `aggregate3` call:
21
18
 
22
19
  - Check if transaction is to multicall3 address (`0xcA11bde05977b3631167028862bE2a173976CA11`)
23
20
  - Decode as `aggregate3(Call3[] calldata calls)`
24
- - Allow calls to known addresses and methods (rollup, governance, slashing contracts, etc.)
25
- - Find the single `propose` call to the rollup contract
26
- - Verify exactly one `propose` call exists
27
- - Extract and return the propose calldata
21
+ - Find all calls matching the rollup contract address and the `propose` function selector
22
+ - Verify each candidate by computing `attestationsHash` (keccak256 of ABI-encoded attestations)
23
+ and `payloadDigest` (keccak256 of the consensus payload signing hash) and comparing against
24
+ expected values from the `CheckpointProposed` event
25
+ - Return the verified candidate (if multiple verify, return the first with a warning)
28
26
 
29
- This step handles the common case efficiently without requiring expensive trace or debug RPC calls.
30
- Any validation failure triggers fallback to the next step.
27
+ This approach works regardless of what other calls are in the multicall3 bundle, because hash
28
+ matching identifies the correct propose call without needing an allowlist.
31
29
 
32
30
  ### Direct Propose Call
33
31
 
@@ -35,64 +33,23 @@ Second attempt to decode the transaction as a direct `propose` call to the rollu
35
33
 
36
34
  - Check if transaction is to the rollup address
37
35
  - Decode as `propose` function call
38
- - Verify the function is indeed `propose`
36
+ - Verify against expected hashes
39
37
  - Return the transaction input as the propose calldata
40
38
 
41
- This handles scenarios where clients submit transactions directly to the rollup contract without
42
- using multicall3 for bundling. Any validation failure triggers fallback to the next step.
43
-
44
39
  ### Spire Proposer Call
45
40
 
46
- Given existing attempts to route the call via the Spire proposer, we also check if the tx is `to` the
47
- proposer known address, and if so, we try decoding it as either a multicall3 or a direct call to the
48
- rollup contract.
49
-
50
- Similar as with the multicall3 check, we check that there are no other calls in the Spire proposer, so
51
- we are absolutely sure that the only call is the successful one to the rollup. Any extraneous call would
52
- imply an unexpected path to calling `propose` in the rollup contract, and since we cannot verify if the
53
- calldata arguments we extracted are the correct ones (see the section below), we cannot know for sure which
54
- one is the call that succeeded, so we don't know which calldata to process.
55
-
56
- Furthermore, since the Spire proposer is upgradeable, we check if the implementation has not changed in
57
- order to decode. As usual, any validation failure triggers fallback to the next step.
58
-
59
- ### Verifying Multicall3 Arguments
60
-
61
- **This is NOT implemented for simplicity's sake**
62
-
63
- If the checks above don't hold, such as when there are multiple calls to `propose`, then we cannot
64
- reliably extract the `propose` calldata from the multicall3 arguments alone. We can try a best-effort
65
- where we try all `propose` calls we see and validate them against on-chain data. Note that we can use these
66
- same strategies if we were to obtain the calldata from another source.
67
-
68
- #### TempBlockLog Verification
69
-
70
- Read the stored `TempBlockLog` for the L2 block number from L1 and verify it matches our decoded header hash,
71
- since the `TempBlockLog` stores the hash of the proposed block header, the payload commitment, and the attestations.
72
-
73
- However, `TempBlockLog` is only stored temporarily and deleted after proven, so this method only works for recent
74
- blocks, not for historical data syncing.
75
-
76
- #### Archive Verification
77
-
78
- Verify that the archive root in the decoded propose is correct with regard to the block header. This requires
79
- hashing the block header we have retrieved, inserting it into the archive tree, and checking the resulting root
80
- against the one we got from L1.
81
-
82
- However, this requires that the archive keeps a reference to world-state, which is not the case in the current
83
- system.
84
-
85
- #### Emit Commitments in Rollup Contract
86
-
87
- Modify rollup contract to emit commitments to the block header in the `L2BlockProposed` event, allowing us to easily
88
- verify the calldata we obtained vs the emitted event.
41
+ Given existing attempts to route the call via the Spire proposer, we also check if the tx is
42
+ `to` the proposer known address. If so, we extract all wrapped calls and try each as either
43
+ a multicall3 or direct propose call, using hash matching to find and verify the correct one.
89
44
 
90
- However, modifying the rollup contract is out of scope for this change. But we can implement this approach in `v2`.
45
+ Since the Spire proposer is upgradeable, we check that the implementation has not changed in
46
+ order to decode. Any validation failure triggers fallback to the next step.
91
47
 
92
48
  ### Debug and Trace Transaction Fallback
93
49
 
94
- Last, we use L1 node's trace/debug RPC methods to definitively identify the one successful `propose` call within the tx.
95
- We can then extract the exact calldata that hit the `propose` function in the rollup contract.
50
+ Last, we use L1 node's trace/debug RPC methods to definitively identify the one successful
51
+ `propose` call within the tx. We can then extract the exact calldata that hit the `propose`
52
+ function in the rollup contract.
96
53
 
97
- This approach requires access to a debug-enabled L1 node, which may be more resource-intensive, so we only
98
- use it as a fallback when the first step fails, which should be rare in practice.
54
+ This approach requires access to a debug-enabled L1 node, which may be more resource-intensive,
55
+ so we only use it as a fallback when earlier steps fail, which should be rare in practice.