eventsourcingdb-merkle 2.1.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +76 -26
- package/dist/commands/generate-salt-key.js +27 -0
- package/dist/commands/get-proof.js +22 -3
- package/dist/commands/merkle-root.js +13 -3
- package/dist/commands/verify-proof.js +14 -13
- package/dist/lib/hash.js +4 -0
- package/dist/lib/merkleTree.js +170 -67
- package/dist/lib/merkleTreeVersion1.js +86 -0
- package/dist/lib/merkleTreeVersion2.js +51 -0
- package/dist/lib/merkleTreeVersion3.js +65 -0
- package/dist/lib/saltKey.js +65 -0
- package/dist/lib/tree.js +63 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -79,11 +79,13 @@ eventsourcingdb-merkle merkle-root --backup=backup.json
|
|
|
79
79
|
The command outputs the Merkle root, the total number of events, and the ID of the last event it covers:
|
|
80
80
|
|
|
81
81
|
```
|
|
82
|
-
Merkle Root:
|
|
82
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
83
83
|
Total events: 4
|
|
84
84
|
Last event ID: 3
|
|
85
85
|
```
|
|
86
86
|
|
|
87
|
+
The root covers the number of events as well, so that a proof against it can name the position of its event, as described under [Verifying Merkle Proofs](#verifying-merkle-proofs).
|
|
88
|
+
|
|
87
89
|
#### Calculating the Merkle Root of an Earlier State
|
|
88
90
|
|
|
89
91
|
A root that you published once has to be calculated again later, for instance to generate proofs against it. Since events are never changed, a newer backup, or the database itself, holds the events of an older backup, followed by those written since. Pass the last event ID that the root covered with `--last-event-id`, and the command calculates the root over the events up to it, which is the root of the older backup:
|
|
@@ -94,6 +96,27 @@ eventsourcingdb-merkle merkle-root --backup=backup-2025.json --last-event-id=3
|
|
|
94
96
|
|
|
95
97
|
So you do not have to keep the older backup to prove against its root. If the backup or the database holds no event with that ID, the command refuses rather than calculating a root over other events.
|
|
96
98
|
|
|
99
|
+
#### Salting the Leaves
|
|
100
|
+
|
|
101
|
+
A proof names the hashes next to the path from its event to the root, and the first of them belongs to a neighboring event. That hash does not reveal the event, but whoever holds the proof and has a guess about the neighbor, such as its type, its subject, its time, and its data, can calculate its hash and so check the guess. Where your events can be guessed like that, salt the leaves of the tree: a random value in front of every leaf, which only a proof of that leaf discloses, makes the other hashes in a proof tell nothing.
|
|
102
|
+
|
|
103
|
+
The tool derives the salt of every leaf from a key, so that no salt has to be kept. Generate the key once, and pass it with `--salt-key-file` whenever you calculate the root, or a proof against it:
|
|
104
|
+
|
|
105
|
+
```shell
|
|
106
|
+
eventsourcingdb-merkle generate-salt-key --output=merkle.key
|
|
107
|
+
eventsourcingdb-merkle merkle-root --backup=backup.json --salt-key-file=merkle.key
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
The key file holds 64 hexadecimal digits. `generate-salt-key` never overwrites a file, and it lets nobody write the key, and on Linux and macOS, only the owner of the file read it. Windows knows only whether a file is read-only, so there, who can read the key follows from the permissions of its folder. Keep the key as long as the root, and as safely as a password: a root calculated with it is calculated again with that key only, so without it, no further proof can be generated for the root, and whoever has it can calculate the salt of every leaf. A proof carries the salt of its own leaf, so verifying it needs no key.
|
|
111
|
+
|
|
112
|
+
#### Calculating a Root of an Earlier Version
|
|
113
|
+
|
|
114
|
+
Releases before 4.0 built the tree in other ways. Release 3.0 built version 2 of the tree, which has the leaves and inner nodes of version 3, but a root that does not cover the number of events, and proofs that follow the sides of their siblings, so that they do not bind the ID of their event. Releases before 3.0 built version 1, which hashes leaves and inner nodes alike, and pairs the last node of a level with itself where it has no partner, so that a list of hashes has the same root as the same list with its last hash twice. A root of version 3 starts with `esdb:merkle:v3:`, a root of version 2 with `esdb:merkle:v2:`, and a root of version 1 has no prefix. Version 1 knows no salts. To calculate a root of an earlier version again, or to prove an event against one, pass `--tree-version=1` or `--tree-version=2`:
|
|
115
|
+
|
|
116
|
+
```shell
|
|
117
|
+
eventsourcingdb-merkle merkle-root --backup=backup.json --tree-version=2
|
|
118
|
+
```
|
|
119
|
+
|
|
97
120
|
### Validating Individual Events
|
|
98
121
|
|
|
99
122
|
Sometimes you want to verify the integrity of a single event without processing the entire backup. The `validate-event-hash` command provides two ways to do this.
|
|
@@ -159,21 +182,39 @@ The command outputs the proof in a human-readable format:
|
|
|
159
182
|
```
|
|
160
183
|
Merkle Proof for Event 42
|
|
161
184
|
|
|
162
|
-
Event Hash: abc123...
|
|
163
|
-
Merkle Root:
|
|
185
|
+
Event Hash: esdb:hash:v2:abc123...
|
|
186
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
187
|
+
Last Event ID: 99
|
|
164
188
|
|
|
165
189
|
Sibling Hashes (3):
|
|
166
|
-
1. def456...
|
|
167
|
-
2. ghi789...
|
|
168
|
-
3. jkl012...
|
|
190
|
+
1. def456...
|
|
191
|
+
2. ghi789...
|
|
192
|
+
3. jkl012...
|
|
169
193
|
```
|
|
170
194
|
|
|
195
|
+
A proof of version 3 names the ID of the last event that its root covers, which fixes the position of its event in the tree. A proof of version 1 or 2 names no last event ID, but the side of each sibling instead, such as `def456... (left)`.
|
|
196
|
+
|
|
171
197
|
To prove an event against the root of an earlier state, pass its last event ID with `--last-event-id`, as for `merkle-root`. An event after it is not part of that tree, so the command refuses to prove it:
|
|
172
198
|
|
|
173
199
|
```shell
|
|
174
200
|
eventsourcingdb-merkle get-proof --backup=backup.json --event-id=42 --last-event-id=1234566
|
|
175
201
|
```
|
|
176
202
|
|
|
203
|
+
To prove an event against a root whose leaves are salted, pass the same key with `--salt-key-file`, and to prove it against a root of an earlier version, pass `--tree-version`, as for `merkle-root`. A proof against a salted root carries the salt of the leaf of its event, right after the hash of the event:
|
|
204
|
+
|
|
205
|
+
```shell
|
|
206
|
+
eventsourcingdb-merkle get-proof --backup=backup.json --event-id=42 --salt-key-file=merkle.key
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
Merkle Proof for Event 42
|
|
211
|
+
|
|
212
|
+
Event Hash: esdb:hash:v2:abc123...
|
|
213
|
+
Salt: 9c1f0b...
|
|
214
|
+
Merkle Root: esdb:merkle:v3:5f0c3e...
|
|
215
|
+
Last Event ID: 99
|
|
216
|
+
```
|
|
217
|
+
|
|
177
218
|
#### Outputting JSON
|
|
178
219
|
|
|
179
220
|
To get the proof as JSON for programmatic use, add the `--json` flag:
|
|
@@ -187,25 +228,19 @@ The output can then be saved to a file and shared with others:
|
|
|
187
228
|
```json
|
|
188
229
|
{
|
|
189
230
|
"eventId": "42",
|
|
190
|
-
"eventHash": "abc123...",
|
|
231
|
+
"eventHash": "esdb:hash:v2:abc123...",
|
|
191
232
|
"siblingHashes": [
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
},
|
|
196
|
-
{
|
|
197
|
-
"hash": "ghi789...",
|
|
198
|
-
"position": "right"
|
|
199
|
-
},
|
|
200
|
-
{
|
|
201
|
-
"hash": "jkl012...",
|
|
202
|
-
"position": "left"
|
|
203
|
-
}
|
|
233
|
+
"def456...",
|
|
234
|
+
"ghi789...",
|
|
235
|
+
"jkl012..."
|
|
204
236
|
],
|
|
205
|
-
"merkleRoot": "
|
|
237
|
+
"merkleRoot": "esdb:merkle:v3:6a0e4b2f9d1c...",
|
|
238
|
+
"lastEventId": "99"
|
|
206
239
|
}
|
|
207
240
|
```
|
|
208
241
|
|
|
242
|
+
A proof against a salted root holds its salt in `salt`, right after `eventHash`. A proof of version 1 or 2 holds no `lastEventId`, and every sibling in it is an object with its `hash` and its `position`, `left` or `right`.
|
|
243
|
+
|
|
209
244
|
### Verifying Merkle Proofs
|
|
210
245
|
|
|
211
246
|
Once you have a Merkle proof, you can verify it without needing the original backup file. This is particularly useful when you receive a proof from someone else and want to verify that the event is indeed part of the backup with the claimed Merkle root.
|
|
@@ -219,16 +254,19 @@ eventsourcingdb-merkle verify-proof proof.json
|
|
|
219
254
|
Or with a JSON string:
|
|
220
255
|
|
|
221
256
|
```shell
|
|
222
|
-
eventsourcingdb-merkle verify-proof '{"eventId":"42","eventHash":"...","siblingHashes":[],"merkleRoot":"
|
|
257
|
+
eventsourcingdb-merkle verify-proof '{"eventId":"42","eventHash":"...","siblingHashes":["..."],"merkleRoot":"esdb:merkle:v3:...","lastEventId":"99"}'
|
|
223
258
|
```
|
|
224
259
|
|
|
260
|
+
The command verifies proofs of every version of the tree, which it tells apart by their root, and takes the salt of a salted proof from the proof itself, so it needs neither the backup nor the key.
|
|
261
|
+
|
|
225
262
|
If the proof is valid, the command confirms this and displays the event details:
|
|
226
263
|
|
|
227
264
|
```
|
|
228
265
|
✓ Merkle proof is valid
|
|
229
266
|
Event ID: 42
|
|
230
|
-
Event Hash: abc123...
|
|
231
|
-
Merkle Root:
|
|
267
|
+
Event Hash: esdb:hash:v2:abc123...
|
|
268
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
269
|
+
Last Event ID: 99
|
|
232
270
|
```
|
|
233
271
|
|
|
234
272
|
If the proof is invalid, the command indicates this with a non-zero exit code:
|
|
@@ -236,10 +274,13 @@ If the proof is invalid, the command indicates this with a non-zero exit code:
|
|
|
236
274
|
```
|
|
237
275
|
✗ Merkle proof is invalid
|
|
238
276
|
Event ID: 42
|
|
239
|
-
Event Hash: abc123...
|
|
240
|
-
Merkle Root:
|
|
277
|
+
Event Hash: esdb:hash:v2:abc123...
|
|
278
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
279
|
+
Last Event ID: 99
|
|
241
280
|
```
|
|
242
281
|
|
|
282
|
+
A valid proof shows that the event is part of the tree with the root it names, so compare that root with the one that was published. In version 3, it also shows that the event is the one with its ID, in a tree that ends at the last event ID: its path leads to the root from the leaf at that position only, and the root covers the number of events, so neither ID can be changed without the proof becoming invalid. A proof of version 1 or 2 does not bind the ID of its event, so whoever hands it on could change it, and its ID is only as trustworthy as whoever gave you the proof.
|
|
283
|
+
|
|
243
284
|
## Understanding the Technical Details
|
|
244
285
|
|
|
245
286
|
The tool implements Merkle trees using SHA-256 hashing, following the EventSourcingDB specification for event hash calculation, which knows two versions. An event keeps the version of the hash it was written with, so the tool verifies every hash in the version it names:
|
|
@@ -249,7 +290,16 @@ The tool implements Merkle trees using SHA-256 hashing, following the EventSourc
|
|
|
249
290
|
|
|
250
291
|
In both versions, the data is hashed as it appears in the backup or in the event given with `--event`, rather than parsed and serialized again, since that would change escape sequences such as `\u003c` for `<` and the notation of numbers. Only the whitespace between its tokens is left out, which EventSourcingDB never writes, so that an event formatted afterwards, as by a viewer or by `jq` without `-c`, still verifies.
|
|
251
292
|
|
|
252
|
-
The Merkle
|
|
293
|
+
The Merkle trees of versions 2 and 3 follow [RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), as the tree of [eventsourcingdb-auditing](https://github.com/thenativeweb/eventsourcingdb-auditing) does:
|
|
294
|
+
|
|
295
|
+
- The data of a leaf is the hash of its event as it is stored, prefix included, as text in UTF-8. Where the leaves are salted, its salt comes first: the HMAC-SHA256 of that text under the salt key. Since nothing marks where the salt ends, the tree takes only hashes in the form of version 1 or 2, which give the data of a leaf other lengths with a salt than without one, and refuses an event with any other hash, as `verify-proof` refuses a proof of one.
|
|
296
|
+
- The hash of a leaf is the SHA-256 of the byte `0x00` followed by its data, and the hash of an inner node is the SHA-256 of the byte `0x01` followed by the hashes of its two children, so that neither can pass for the other.
|
|
297
|
+
- The tree splits its leaves at the largest power of two smaller than their number, and never duplicates a node.
|
|
298
|
+
- The root of version 2 is the hash of the top node in hexadecimal, after the prefix `esdb:merkle:v2:`. The root of version 3 is the SHA-256 of the byte `0x02`, the number of leaves as 8 bytes, big-endian, and the hash of the top node, in hexadecimal, after the prefix `esdb:merkle:v3:`.
|
|
299
|
+
|
|
300
|
+
A proof of version 3 is verified as [RFC 9162](https://www.rfc-editor.org/rfc/rfc9162), section 2.1.3.2, verifies an inclusion proof, with the event ID as the index of its leaf and one more than the last event ID as the size of the tree, from which the side of every sibling follows. EventSourcingDB numbers its events from 0 without gaps, so the leaf at an index is the event with that ID, and the tree of version 3 refuses a source whose IDs do not run like that. Certificate Transparency, for which RFC 9162 was written, signs the size of a tree together with its root. The root of version 3 hashes it in instead, since without it, a proof could bring a size of its own, under which its path would lead to the root from the leaf at another index.
|
|
301
|
+
|
|
302
|
+
The tree of version 1 takes the stored hashes as its leaves, and hashes the concatenation of the hashes of its two children, as text, for an inner node. Where a level has an odd number of nodes, it pairs the last one with itself.
|
|
253
303
|
|
|
254
304
|
Backup files are expected to be in one of the two NDJSON formats that EventSourcingDB writes, and the tool reads every backup that a restore of EventSourcingDB takes. It checks the structure of a backup, the hashes, and the chain, but neither the values of the events nor the schemas, which a restore checks with the rules of the server:
|
|
255
305
|
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
import { Flags } from '@oclif/core';
|
|
2
|
+
import BaseCommand from '../lib/BaseCommand.js';
|
|
3
|
+
import { describeError } from '../lib/describeError.js';
|
|
4
|
+
import { writeSaltKey } from '../lib/saltKey.js';
|
|
5
|
+
export default class GenerateSaltKey extends BaseCommand {
|
|
6
|
+
static description = 'Generate a key to salt the leaves of Merkle trees with';
|
|
7
|
+
static examples = ['<%= config.bin %> <%= command.id %> --output=merkle.key'];
|
|
8
|
+
static flags = {
|
|
9
|
+
output: Flags.string({
|
|
10
|
+
description: 'File to write the salt key to; an existing file is never overwritten',
|
|
11
|
+
required: true,
|
|
12
|
+
}),
|
|
13
|
+
};
|
|
14
|
+
static args = {};
|
|
15
|
+
async run() {
|
|
16
|
+
const { flags } = await this.parse(GenerateSaltKey);
|
|
17
|
+
try {
|
|
18
|
+
await writeSaltKey(flags.output);
|
|
19
|
+
this.log(`✓ Salt key written to ${flags.output}`);
|
|
20
|
+
this.log('Keep it safe: without it, no further proof can be generated for a root calculated with it.');
|
|
21
|
+
}
|
|
22
|
+
catch (error) {
|
|
23
|
+
this.log(`✗ Failed to generate salt key: ${describeError(error)}`);
|
|
24
|
+
process.exitCode = 1;
|
|
25
|
+
}
|
|
26
|
+
}
|
|
27
|
+
}
|
|
@@ -5,12 +5,14 @@ import { describeError } from '../lib/describeError.js';
|
|
|
5
5
|
import { parseEventId } from '../lib/eventId.js';
|
|
6
6
|
import { generateMerkleProof } from '../lib/merkleTree.js';
|
|
7
7
|
import { getSource, readEvents, sourceFlags } from '../lib/source.js';
|
|
8
|
+
import { getTree, getTreeFlags, getTreeHashes, readTreeOptions } from '../lib/tree.js';
|
|
8
9
|
export default class GetProof extends BaseCommand {
|
|
9
10
|
static description = 'Generate a Merkle proof for a specific event of an EventSourcingDB backup file or a running EventSourcingDB';
|
|
10
11
|
static examples = [
|
|
11
12
|
'<%= config.bin %> <%= command.id %> --backup=backup.json --event-id=42',
|
|
12
13
|
'<%= config.bin %> <%= command.id %> --backup=/path/to/backup.json --event-id=0 --json',
|
|
13
14
|
'<%= config.bin %> <%= command.id %> --backup=backup.json --event-id=42 --last-event-id=1234566',
|
|
15
|
+
'<%= config.bin %> <%= command.id %> --backup=backup.json --event-id=42 --salt-key-file=merkle.key',
|
|
14
16
|
'<%= config.bin %> <%= command.id %> --esdb-url=http://localhost:3000 --esdb-api-token="$ESDB_API_TOKEN" --event-id=42',
|
|
15
17
|
];
|
|
16
18
|
static flags = {
|
|
@@ -20,6 +22,7 @@ export default class GetProof extends BaseCommand {
|
|
|
20
22
|
description: 'ID of the last event the root covers, to prove against the root of an earlier state',
|
|
21
23
|
}),
|
|
22
24
|
json: Flags.boolean({ description: 'Output proof as JSON' }),
|
|
25
|
+
...getTreeFlags('Version of the Merkle tree: 1 to prove against roots that releases before 3.0 calculated, 2 to prove against roots that release 3.0 calculated'),
|
|
23
26
|
};
|
|
24
27
|
static args = {};
|
|
25
28
|
async run() {
|
|
@@ -39,7 +42,13 @@ export default class GetProof extends BaseCommand {
|
|
|
39
42
|
this.refuseCall(`--last-event-id has to be an event ID, but is '${lastEventId}'`);
|
|
40
43
|
return;
|
|
41
44
|
}
|
|
45
|
+
const tree = getTree(flags);
|
|
46
|
+
if ('refusal' in tree) {
|
|
47
|
+
this.refuseCall(tree.refusal);
|
|
48
|
+
return;
|
|
49
|
+
}
|
|
42
50
|
try {
|
|
51
|
+
const treeOptions = await readTreeOptions(tree);
|
|
43
52
|
const events = await readEvents(source, { lastEventId });
|
|
44
53
|
// An event after the last one is not part of the tree, so it has no
|
|
45
54
|
// proof against its root, although the source holds it.
|
|
@@ -58,8 +67,8 @@ export default class GetProof extends BaseCommand {
|
|
|
58
67
|
process.exitCode = 1;
|
|
59
68
|
return;
|
|
60
69
|
}
|
|
61
|
-
const hashes = events
|
|
62
|
-
const proofData = generateMerkleProof(hashes, event
|
|
70
|
+
const hashes = getTreeHashes(events, treeOptions);
|
|
71
|
+
const proofData = generateMerkleProof(hashes, events.indexOf(event), treeOptions);
|
|
63
72
|
const proof = {
|
|
64
73
|
eventId: event.id,
|
|
65
74
|
...proofData,
|
|
@@ -71,11 +80,21 @@ export default class GetProof extends BaseCommand {
|
|
|
71
80
|
this.log(`Merkle Proof for Event ${proof.eventId}`);
|
|
72
81
|
this.log('');
|
|
73
82
|
this.log(`Event Hash: ${proof.eventHash}`);
|
|
83
|
+
if (proof.salt !== undefined) {
|
|
84
|
+
this.log(`Salt: ${proof.salt}`);
|
|
85
|
+
}
|
|
74
86
|
this.log(`Merkle Root: ${proof.merkleRoot}`);
|
|
87
|
+
if ('lastEventId' in proof) {
|
|
88
|
+
this.log(`Last Event ID: ${proof.lastEventId}`);
|
|
89
|
+
}
|
|
75
90
|
this.log('');
|
|
76
91
|
this.log(`Sibling Hashes (${proof.siblingHashes.length}):`);
|
|
92
|
+
// The siblings of a proof of version 3 have no sides, since they follow
|
|
93
|
+
// from the position of the event and the size of the tree.
|
|
77
94
|
for (const [index, sibling] of proof.siblingHashes.entries()) {
|
|
78
|
-
this.log(
|
|
95
|
+
this.log(typeof sibling === 'string'
|
|
96
|
+
? ` ${index + 1}. ${sibling}`
|
|
97
|
+
: ` ${index + 1}. ${sibling.hash} (${sibling.position})`);
|
|
79
98
|
}
|
|
80
99
|
}
|
|
81
100
|
catch (error) {
|
|
@@ -2,13 +2,16 @@ import { Flags } from '@oclif/core';
|
|
|
2
2
|
import BaseCommand from '../lib/BaseCommand.js';
|
|
3
3
|
import { describeError } from '../lib/describeError.js';
|
|
4
4
|
import { parseEventId } from '../lib/eventId.js';
|
|
5
|
-
import {
|
|
5
|
+
import { calculateMerkleRoot } from '../lib/merkleTree.js';
|
|
6
6
|
import { getSource, readEvents, sourceFlags } from '../lib/source.js';
|
|
7
|
+
import { getTree, getTreeFlags, getTreeHashes, readTreeOptions } from '../lib/tree.js';
|
|
7
8
|
export default class MerkleRoot extends BaseCommand {
|
|
8
9
|
static description = 'Calculate the Merkle root for an EventSourcingDB backup file or a running EventSourcingDB';
|
|
9
10
|
static examples = [
|
|
10
11
|
'<%= config.bin %> <%= command.id %> --backup=backup.json',
|
|
11
12
|
'<%= config.bin %> <%= command.id %> --backup=/path/to/backup.json --last-event-id=41',
|
|
13
|
+
'<%= config.bin %> <%= command.id %> --backup=backup.json --salt-key-file=merkle.key',
|
|
14
|
+
'<%= config.bin %> <%= command.id %> --backup=backup.json --tree-version=1',
|
|
12
15
|
'<%= config.bin %> <%= command.id %> --esdb-url=http://localhost:3000 --esdb-api-token="$ESDB_API_TOKEN"',
|
|
13
16
|
];
|
|
14
17
|
static flags = {
|
|
@@ -16,6 +19,7 @@ export default class MerkleRoot extends BaseCommand {
|
|
|
16
19
|
'last-event-id': Flags.string({
|
|
17
20
|
description: 'ID of the last event to cover, to calculate the root of an earlier state again',
|
|
18
21
|
}),
|
|
22
|
+
...getTreeFlags('Version of the Merkle tree: 1 for roots that releases before 3.0 calculated, 2 for roots that release 3.0 calculated'),
|
|
19
23
|
};
|
|
20
24
|
static args = {};
|
|
21
25
|
async run() {
|
|
@@ -30,10 +34,16 @@ export default class MerkleRoot extends BaseCommand {
|
|
|
30
34
|
this.refuseCall(`--last-event-id has to be an event ID, but is '${lastEventId}'`);
|
|
31
35
|
return;
|
|
32
36
|
}
|
|
37
|
+
const tree = getTree(flags);
|
|
38
|
+
if ('refusal' in tree) {
|
|
39
|
+
this.refuseCall(tree.refusal);
|
|
40
|
+
return;
|
|
41
|
+
}
|
|
33
42
|
try {
|
|
43
|
+
const treeOptions = await readTreeOptions(tree);
|
|
34
44
|
const events = await readEvents(source, { lastEventId });
|
|
35
|
-
const hashes = events
|
|
36
|
-
const merkleRoot =
|
|
45
|
+
const hashes = getTreeHashes(events, treeOptions);
|
|
46
|
+
const merkleRoot = calculateMerkleRoot(hashes, treeOptions);
|
|
37
47
|
// The ID of the last event is what calculates this root again later,
|
|
38
48
|
// from a source that has grown since.
|
|
39
49
|
this.log(`Merkle Root: ${merkleRoot}`);
|
|
@@ -1,9 +1,10 @@
|
|
|
1
1
|
import { Args } from '@oclif/core';
|
|
2
2
|
import BaseCommand from '../lib/BaseCommand.js';
|
|
3
|
-
import { verifyMerkleProof } from '../lib/merkleTree.js';
|
|
3
|
+
import { findProofStructureProblem, verifyMerkleProof } from '../lib/merkleTree.js';
|
|
4
4
|
export default class VerifyProof extends BaseCommand {
|
|
5
5
|
static description = 'Verify a Merkle proof';
|
|
6
6
|
static examples = [
|
|
7
|
+
'<%= config.bin %> <%= command.id %> \'{"eventId":"42","eventHash":"...","siblingHashes":["..."],"merkleRoot":"esdb:merkle:v3:...","lastEventId":"99"}\'',
|
|
7
8
|
'<%= config.bin %> <%= command.id %> \'{"eventId":"42","eventHash":"...","siblingHashes":[],"merkleRoot":"..."}\'',
|
|
8
9
|
'<%= config.bin %> <%= command.id %> proof.json',
|
|
9
10
|
];
|
|
@@ -27,26 +28,26 @@ export default class VerifyProof extends BaseCommand {
|
|
|
27
28
|
const fileContent = readFileSync(args.proof, 'utf-8');
|
|
28
29
|
proof = JSON.parse(fileContent);
|
|
29
30
|
}
|
|
30
|
-
|
|
31
|
-
if (
|
|
31
|
+
const structureProblem = findProofStructureProblem(proof);
|
|
32
|
+
if (structureProblem !== undefined) {
|
|
32
33
|
this.log('✗ Invalid proof structure');
|
|
33
|
-
this.log(
|
|
34
|
+
this.log(structureProblem);
|
|
34
35
|
process.exitCode = 1;
|
|
35
36
|
return;
|
|
36
37
|
}
|
|
37
38
|
const isValid = verifyMerkleProof(proof);
|
|
38
|
-
|
|
39
|
-
this.log(`✗ Merkle proof is invalid`);
|
|
40
|
-
this.log(`Event ID: ${proof.eventId}`);
|
|
41
|
-
this.log(`Event Hash: ${proof.eventHash}`);
|
|
42
|
-
this.log(`Merkle Root: ${proof.merkleRoot}`);
|
|
43
|
-
process.exitCode = 1;
|
|
44
|
-
return;
|
|
45
|
-
}
|
|
46
|
-
this.log(`✓ Merkle proof is valid`);
|
|
39
|
+
this.log(isValid ? '✓ Merkle proof is valid' : '✗ Merkle proof is invalid');
|
|
47
40
|
this.log(`Event ID: ${proof.eventId}`);
|
|
48
41
|
this.log(`Event Hash: ${proof.eventHash}`);
|
|
49
42
|
this.log(`Merkle Root: ${proof.merkleRoot}`);
|
|
43
|
+
// The root of version 3 covers the size of its tree, so the ID of the last
|
|
44
|
+
// event tells which state of the store it is the root of.
|
|
45
|
+
if ('lastEventId' in proof) {
|
|
46
|
+
this.log(`Last Event ID: ${proof.lastEventId}`);
|
|
47
|
+
}
|
|
48
|
+
if (!isValid) {
|
|
49
|
+
process.exitCode = 1;
|
|
50
|
+
}
|
|
50
51
|
}
|
|
51
52
|
catch (error) {
|
|
52
53
|
this.log(`✗ Failed to verify Merkle proof: ${error instanceof Error ? error.message : String(error)}`);
|
package/dist/lib/hash.js
CHANGED
|
@@ -50,6 +50,10 @@ export const assertKnownHashVersion = (storedHash) => {
|
|
|
50
50
|
const [version = ''] = storedHash.slice(hashPrefix.length).split(':');
|
|
51
51
|
throw new UnsupportedHashVersionError(version);
|
|
52
52
|
};
|
|
53
|
+
// A stored hash of version 1 is a SHA-256 in hexadecimal, and one of version 2
|
|
54
|
+
// the same after its prefix, as EventSourcingDB writes them.
|
|
55
|
+
const knownStoredHashPattern = new RegExp(`^(${hashVersion2Prefix})?[0-9a-f]{64}$`, 'u');
|
|
56
|
+
export const isKnownStoredHash = (text) => knownStoredHashPattern.test(text);
|
|
53
57
|
// calculateEventHash computes the hash of an event in the version that its
|
|
54
58
|
// stored hash names, since an event keeps the version of the hash it was
|
|
55
59
|
// written with. An event without a stored hash names no version, which is what
|
package/dist/lib/merkleTree.js
CHANGED
|
@@ -1,81 +1,184 @@
|
|
|
1
|
-
import {
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
1
|
+
import { isKnownStoredHash } from './hash.js';
|
|
2
|
+
import { isObject } from './isObject.js';
|
|
3
|
+
import { buildVersion1Root, generateVersion1Proof, verifyVersion1Proof, } from './merkleTreeVersion1.js';
|
|
4
|
+
import { buildTree, calculateRootFromAuditPath, hashLeaf, } from './merkleTreeVersion2.js';
|
|
5
|
+
import { buildVersion3Tree, calculateVersion3Root } from './merkleTreeVersion3.js';
|
|
6
|
+
import { deriveSalt } from './saltKey.js';
|
|
7
|
+
// A root that names the version of its tree starts with this prefix. A root
|
|
8
|
+
// without it is of version 1, which predates the prefix.
|
|
9
|
+
const merkleRootPrefix = 'esdb:merkle:';
|
|
10
|
+
const merkleRootVersion2Prefix = `${merkleRootPrefix}v2:`;
|
|
11
|
+
const merkleRootVersion3Prefix = `${merkleRootPrefix}v3:`;
|
|
12
|
+
// A salt and the hash of a node are SHA-256 digests, which a proof carries in
|
|
13
|
+
// hexadecimal. Decoding takes text that is not exactly that only as far as it
|
|
14
|
+
// can, so a proof that holds more than the digest would still verify.
|
|
15
|
+
const digestPattern = /^[0-9a-f]{64}$/iu;
|
|
16
|
+
export class UnsupportedMerkleTreeVersionError extends Error {
|
|
17
|
+
constructor(version) {
|
|
18
|
+
super(`Unsupported Merkle tree version: ${version}`);
|
|
19
|
+
this.name = 'UnsupportedMerkleTreeVersionError';
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
// The data of a leaf of versions 2 and 3 is the stored hash of its event, as
|
|
23
|
+
// text, which covers both versions of the hash alike, preceded by its salt where
|
|
24
|
+
// the leaves are salted.
|
|
25
|
+
const getLeafHash = (storedHash, salt) => hashLeaf(Buffer.concat([salt ?? Buffer.alloc(0), Buffer.from(storedHash, 'utf-8')]));
|
|
26
|
+
const getSalt = (saltKey, storedHash) => saltKey === undefined ? undefined : deriveSalt(saltKey, storedHash);
|
|
27
|
+
const getLeafHashes = (hashes, saltKey) => hashes.map(hash => getLeafHash(hash, getSalt(saltKey, hash)));
|
|
28
|
+
const formatVersion2Root = (root) => `${merkleRootVersion2Prefix}${root.toString('hex')}`;
|
|
29
|
+
const formatVersion3Root = (root) => `${merkleRootVersion3Prefix}${root.toString('hex')}`;
|
|
30
|
+
export const calculateMerkleRoot = (hashes, options) => {
|
|
31
|
+
if (options.treeVersion === 1) {
|
|
32
|
+
return buildVersion1Root(hashes);
|
|
18
33
|
}
|
|
19
|
-
const
|
|
20
|
-
if (
|
|
21
|
-
|
|
34
|
+
const leafHashes = getLeafHashes(hashes, options.saltKey);
|
|
35
|
+
if (options.treeVersion === 2) {
|
|
36
|
+
return formatVersion2Root(buildTree(leafHashes).root);
|
|
22
37
|
}
|
|
23
|
-
return root;
|
|
38
|
+
return formatVersion3Root(buildVersion3Tree(leafHashes).root);
|
|
24
39
|
};
|
|
25
|
-
export const generateMerkleProof = (hashes,
|
|
26
|
-
const
|
|
27
|
-
if (
|
|
40
|
+
export const generateMerkleProof = (hashes, index, options) => {
|
|
41
|
+
const eventHash = hashes[index];
|
|
42
|
+
if (eventHash === undefined) {
|
|
28
43
|
throw new Error('Target hash not found in the list');
|
|
29
44
|
}
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
let currentIndex = targetIndex;
|
|
33
|
-
while (currentLevel.length > 1) {
|
|
34
|
-
const nextLevel = [];
|
|
35
|
-
let nextIndex = 0;
|
|
36
|
-
for (let i = 0; i < currentLevel.length; i += 2) {
|
|
37
|
-
const left = currentLevel[i];
|
|
38
|
-
const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
|
|
39
|
-
if (left === undefined || right === undefined) {
|
|
40
|
-
throw new Error('Unexpected undefined hash while generating Merkle proof');
|
|
41
|
-
}
|
|
42
|
-
if (i === currentIndex || i + 1 === currentIndex) {
|
|
43
|
-
if (currentIndex % 2 === 0) {
|
|
44
|
-
// Target is on the left
|
|
45
|
-
if (i + 1 < currentLevel.length) {
|
|
46
|
-
siblingHashes.push({ hash: right, position: 'right' });
|
|
47
|
-
}
|
|
48
|
-
}
|
|
49
|
-
else {
|
|
50
|
-
// Target is on the right
|
|
51
|
-
siblingHashes.push({ hash: left, position: 'left' });
|
|
52
|
-
}
|
|
53
|
-
nextIndex = Math.floor(currentIndex / 2);
|
|
54
|
-
}
|
|
55
|
-
nextLevel.push(hashPair(left, right));
|
|
56
|
-
}
|
|
57
|
-
currentLevel = nextLevel;
|
|
58
|
-
currentIndex = nextIndex;
|
|
45
|
+
if (options.treeVersion === 1) {
|
|
46
|
+
return generateVersion1Proof(hashes, eventHash);
|
|
59
47
|
}
|
|
60
|
-
const
|
|
61
|
-
|
|
62
|
-
|
|
48
|
+
const salt = getSalt(options.saltKey, eventHash);
|
|
49
|
+
const leaf = {
|
|
50
|
+
eventHash,
|
|
51
|
+
...(salt === undefined ? {} : { salt: salt.toString('hex') }),
|
|
52
|
+
};
|
|
53
|
+
const leafHashes = getLeafHashes(hashes, options.saltKey);
|
|
54
|
+
if (options.treeVersion === 2) {
|
|
55
|
+
const { root, auditPath } = buildTree(leafHashes, index);
|
|
56
|
+
return {
|
|
57
|
+
...leaf,
|
|
58
|
+
siblingHashes: auditPath.map(({ hash, position }) => ({
|
|
59
|
+
hash: hash.toString('hex'),
|
|
60
|
+
position,
|
|
61
|
+
})),
|
|
62
|
+
merkleRoot: formatVersion2Root(root),
|
|
63
|
+
};
|
|
63
64
|
}
|
|
65
|
+
// The leaf at an index of the tree of version 3 is the event with that ID, so
|
|
66
|
+
// the last event of the tree has the ID one below the number of its leaves.
|
|
67
|
+
const { root, auditPath } = buildVersion3Tree(leafHashes, index);
|
|
64
68
|
return {
|
|
65
|
-
|
|
66
|
-
siblingHashes,
|
|
67
|
-
merkleRoot,
|
|
69
|
+
...leaf,
|
|
70
|
+
siblingHashes: auditPath.map(hash => hash.toString('hex')),
|
|
71
|
+
merkleRoot: formatVersion3Root(root),
|
|
72
|
+
lastEventId: String(hashes.length - 1),
|
|
68
73
|
};
|
|
69
74
|
};
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
+
// getMerkleTreeVersion tells the version of the tree that a root names, the way
|
|
76
|
+
// the version of a hash is told: a root without the prefix is of version 1, and
|
|
77
|
+
// every other root with it of a version that the tool does not know.
|
|
78
|
+
export const getMerkleTreeVersion = (merkleRoot) => {
|
|
79
|
+
if (!merkleRoot.startsWith(merkleRootPrefix)) {
|
|
80
|
+
return 1;
|
|
81
|
+
}
|
|
82
|
+
if (merkleRoot.startsWith(merkleRootVersion2Prefix)) {
|
|
83
|
+
return 2;
|
|
84
|
+
}
|
|
85
|
+
if (merkleRoot.startsWith(merkleRootVersion3Prefix)) {
|
|
86
|
+
return 3;
|
|
87
|
+
}
|
|
88
|
+
const [version = ''] = merkleRoot.slice(merkleRootPrefix.length).split(':');
|
|
89
|
+
throw new UnsupportedMerkleTreeVersionError(version);
|
|
90
|
+
};
|
|
91
|
+
const isDigest = (value) => typeof value === 'string' && digestPattern.test(value);
|
|
92
|
+
const isSiblingWithSide = (sibling) => isObject(sibling) &&
|
|
93
|
+
isDigest(sibling.hash) &&
|
|
94
|
+
(sibling.position === 'left' || sibling.position === 'right');
|
|
95
|
+
// An ID in a proof of version 3 is written as EventSourcingDB writes IDs, since
|
|
96
|
+
// get-proof writes it rather than anyone typing it, so that the same proof has
|
|
97
|
+
// one notation only.
|
|
98
|
+
const eventIdPattern = /^(0|[1-9][0-9]*)$/u;
|
|
99
|
+
const isEventId = (value) => typeof value === 'string' && eventIdPattern.test(value);
|
|
100
|
+
// findProofStructureProblem describes what keeps a proof from being one that
|
|
101
|
+
// can be verified, or returns undefined if nothing does. Which fields a proof may
|
|
102
|
+
// or has to have depends on the version of its tree alone, so they are part of
|
|
103
|
+
// the structure, as is the form of every field. The form of the event hash in
|
|
104
|
+
// versions 2 and 3 matters for another reason: the data of a leaf is its salt
|
|
105
|
+
// followed by the stored hash, with nothing between them that tells where the
|
|
106
|
+
// salt ends, so without that form, a proof whose salt took the first bytes of
|
|
107
|
+
// the hash would pass for a proof of the rest of it. With it, the data of a leaf
|
|
108
|
+
// has other lengths with a salt than without one. Whether the fields fit one
|
|
109
|
+
// another, such as an event ID beyond the last one, is left to the verification,
|
|
110
|
+
// as RFC 9162 leaves it.
|
|
111
|
+
export const findProofStructureProblem = (proof) => {
|
|
112
|
+
if (!isObject(proof) ||
|
|
113
|
+
typeof proof.eventHash !== 'string' ||
|
|
114
|
+
proof.eventHash === '' ||
|
|
115
|
+
typeof proof.merkleRoot !== 'string' ||
|
|
116
|
+
proof.merkleRoot === '' ||
|
|
117
|
+
!Array.isArray(proof.siblingHashes)) {
|
|
118
|
+
return 'Must contain eventHash, merkleRoot, and siblingHashes';
|
|
119
|
+
}
|
|
120
|
+
const treeVersion = getMerkleTreeVersion(proof.merkleRoot);
|
|
121
|
+
const hasSalt = proof.salt !== undefined;
|
|
122
|
+
const hasLastEventId = proof.lastEventId !== undefined;
|
|
123
|
+
if (treeVersion === 1) {
|
|
124
|
+
if (hasSalt) {
|
|
125
|
+
return 'A proof of Merkle tree version 1 has no salt';
|
|
126
|
+
}
|
|
127
|
+
if (hasLastEventId) {
|
|
128
|
+
return 'A proof of Merkle tree version 1 has no last event ID';
|
|
129
|
+
}
|
|
130
|
+
return undefined;
|
|
131
|
+
}
|
|
132
|
+
if (treeVersion === 2 && hasLastEventId) {
|
|
133
|
+
return 'A proof of Merkle tree version 2 has no last event ID';
|
|
134
|
+
}
|
|
135
|
+
if (treeVersion === 3) {
|
|
136
|
+
if (proof.eventId === undefined || !hasLastEventId) {
|
|
137
|
+
return 'A proof of Merkle tree version 3 has to contain eventId and lastEventId';
|
|
75
138
|
}
|
|
76
|
-
|
|
77
|
-
|
|
139
|
+
if (!isEventId(proof.eventId)) {
|
|
140
|
+
return 'The event ID has to be digits alone, without a sign or leading zeros';
|
|
78
141
|
}
|
|
79
142
|
}
|
|
80
|
-
|
|
143
|
+
if (!isKnownStoredHash(proof.eventHash)) {
|
|
144
|
+
return 'The event hash has to be a hash of version 1 or 2';
|
|
145
|
+
}
|
|
146
|
+
if (hasSalt && !isDigest(proof.salt)) {
|
|
147
|
+
return 'The salt has to be 64 hexadecimal digits';
|
|
148
|
+
}
|
|
149
|
+
if (treeVersion === 2) {
|
|
150
|
+
return proof.siblingHashes.every(isSiblingWithSide)
|
|
151
|
+
? undefined
|
|
152
|
+
: 'Every sibling has to have a hash of 64 hexadecimal digits, and the position left or right';
|
|
153
|
+
}
|
|
154
|
+
if (!proof.siblingHashes.every(isDigest)) {
|
|
155
|
+
return 'Every sibling has to be a hash of 64 hexadecimal digits';
|
|
156
|
+
}
|
|
157
|
+
if (!isEventId(proof.lastEventId)) {
|
|
158
|
+
return 'The last event ID has to be digits alone, without a sign or leading zeros';
|
|
159
|
+
}
|
|
160
|
+
return undefined;
|
|
161
|
+
};
|
|
162
|
+
// verifyMerkleProof tells whether a proof leads from the hash of its event to
|
|
163
|
+
// its root, in the version of the tree that the root names, and in version 3,
|
|
164
|
+
// whether it does so from the leaf of its event in a tree that ends at its last
|
|
165
|
+
// event. A proof in which findProofStructureProblem finds a problem proves
|
|
166
|
+
// nothing, and one that passes it has the fields of its version.
|
|
167
|
+
export const verifyMerkleProof = (proof) => {
|
|
168
|
+
if (findProofStructureProblem(proof) !== undefined) {
|
|
169
|
+
return false;
|
|
170
|
+
}
|
|
171
|
+
const treeVersion = getMerkleTreeVersion(proof.merkleRoot);
|
|
172
|
+
if (treeVersion === 1) {
|
|
173
|
+
return verifyVersion1Proof(proof);
|
|
174
|
+
}
|
|
175
|
+
const salt = proof.salt === undefined ? undefined : Buffer.from(proof.salt, 'hex');
|
|
176
|
+
const leafHash = getLeafHash(proof.eventHash, salt);
|
|
177
|
+
if (treeVersion === 2) {
|
|
178
|
+
const auditPath = proof.siblingHashes.map(({ hash, position }) => ({ hash: Buffer.from(hash, 'hex'), position }));
|
|
179
|
+
return formatVersion2Root(calculateRootFromAuditPath(leafHash, auditPath)) === proof.merkleRoot;
|
|
180
|
+
}
|
|
181
|
+
const { eventId, lastEventId, siblingHashes } = proof;
|
|
182
|
+
const root = calculateVersion3Root(leafHash, BigInt(eventId), BigInt(lastEventId) + 1n, siblingHashes.map(hash => Buffer.from(hash, 'hex')));
|
|
183
|
+
return root !== undefined && formatVersion3Root(root) === proof.merkleRoot;
|
|
81
184
|
};
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
import { hashPair } from './hash.js';
|
|
2
|
+
// The Merkle tree of version 1, which releases before 3.0 calculated. It hashes
|
|
3
|
+
// leaves and inner nodes alike, and pairs a node that is left without a partner
|
|
4
|
+
// with itself, so that a list of hashes has the same root as the same list with
|
|
5
|
+
// its last hash twice. It stays only to calculate the roots that were published
|
|
6
|
+
// with it again, and to verify their proofs.
|
|
7
|
+
export const buildVersion1Root = (hashes) => {
|
|
8
|
+
if (hashes.length === 0) {
|
|
9
|
+
throw new Error('Cannot build Merkle root from empty list');
|
|
10
|
+
}
|
|
11
|
+
let currentLevel = [...hashes];
|
|
12
|
+
while (currentLevel.length > 1) {
|
|
13
|
+
const nextLevel = [];
|
|
14
|
+
for (let i = 0; i < currentLevel.length; i += 2) {
|
|
15
|
+
const left = currentLevel[i];
|
|
16
|
+
const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
|
|
17
|
+
if (left === undefined || right === undefined) {
|
|
18
|
+
throw new Error('Unexpected undefined hash while building Merkle tree');
|
|
19
|
+
}
|
|
20
|
+
nextLevel.push(hashPair(left, right));
|
|
21
|
+
}
|
|
22
|
+
currentLevel = nextLevel;
|
|
23
|
+
}
|
|
24
|
+
const [root] = currentLevel;
|
|
25
|
+
if (root === undefined) {
|
|
26
|
+
throw new Error('Failed to compute Merkle root');
|
|
27
|
+
}
|
|
28
|
+
return root;
|
|
29
|
+
};
|
|
30
|
+
export const generateVersion1Proof = (hashes, targetHash) => {
|
|
31
|
+
const targetIndex = hashes.indexOf(targetHash);
|
|
32
|
+
if (targetIndex === -1) {
|
|
33
|
+
throw new Error('Target hash not found in the list');
|
|
34
|
+
}
|
|
35
|
+
const siblingHashes = [];
|
|
36
|
+
let currentLevel = [...hashes];
|
|
37
|
+
let currentIndex = targetIndex;
|
|
38
|
+
while (currentLevel.length > 1) {
|
|
39
|
+
const nextLevel = [];
|
|
40
|
+
let nextIndex = 0;
|
|
41
|
+
for (let i = 0; i < currentLevel.length; i += 2) {
|
|
42
|
+
const left = currentLevel[i];
|
|
43
|
+
const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
|
|
44
|
+
if (left === undefined || right === undefined) {
|
|
45
|
+
throw new Error('Unexpected undefined hash while generating Merkle proof');
|
|
46
|
+
}
|
|
47
|
+
if (i === currentIndex || i + 1 === currentIndex) {
|
|
48
|
+
if (currentIndex % 2 === 0) {
|
|
49
|
+
// Target is on the left
|
|
50
|
+
if (i + 1 < currentLevel.length) {
|
|
51
|
+
siblingHashes.push({ hash: right, position: 'right' });
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
else {
|
|
55
|
+
// Target is on the right
|
|
56
|
+
siblingHashes.push({ hash: left, position: 'left' });
|
|
57
|
+
}
|
|
58
|
+
nextIndex = Math.floor(currentIndex / 2);
|
|
59
|
+
}
|
|
60
|
+
nextLevel.push(hashPair(left, right));
|
|
61
|
+
}
|
|
62
|
+
currentLevel = nextLevel;
|
|
63
|
+
currentIndex = nextIndex;
|
|
64
|
+
}
|
|
65
|
+
const [merkleRoot] = currentLevel;
|
|
66
|
+
if (merkleRoot === undefined) {
|
|
67
|
+
throw new Error('Failed to compute Merkle root');
|
|
68
|
+
}
|
|
69
|
+
return {
|
|
70
|
+
eventHash: targetHash,
|
|
71
|
+
siblingHashes,
|
|
72
|
+
merkleRoot,
|
|
73
|
+
};
|
|
74
|
+
};
|
|
75
|
+
export const verifyVersion1Proof = (proof) => {
|
|
76
|
+
let currentHash = proof.eventHash;
|
|
77
|
+
for (const { hash, position } of proof.siblingHashes) {
|
|
78
|
+
if (position === 'left') {
|
|
79
|
+
currentHash = hashPair(hash, currentHash);
|
|
80
|
+
}
|
|
81
|
+
else {
|
|
82
|
+
currentHash = hashPair(currentHash, hash);
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
return currentHash === proof.merkleRoot;
|
|
86
|
+
};
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
// RFC 6962 puts a byte of its own in front of what a leaf and an inner node
|
|
3
|
+
// hash, so that neither can pass for the other: without them, the hash of an
|
|
4
|
+
// inner node would also be the hash of a leaf whose data are the hashes of its
|
|
5
|
+
// children.
|
|
6
|
+
const leafPrefix = Buffer.from([0x00]);
|
|
7
|
+
const nodePrefix = Buffer.from([0x01]);
|
|
8
|
+
export const hashLeaf = (data) => createHash('sha256').update(leafPrefix).update(data).digest();
|
|
9
|
+
export const hashNode = (left, right) => createHash('sha256').update(nodePrefix).update(left).update(right).digest();
|
|
10
|
+
// buildTree calculates the root of the tree over the hashes of its leaves, as
|
|
11
|
+
// RFC 6962 defines it, and the audit path of the leaf at the given index. RFC
|
|
12
|
+
// 6962 defines the tree by splitting the leaves at the largest power of two
|
|
13
|
+
// smaller than their number. Pairing the nodes level by level and moving a node
|
|
14
|
+
// that is left without a partner up unchanged builds the same tree, without
|
|
15
|
+
// copying the leaves for every subtree. The node is moved rather than paired
|
|
16
|
+
// with itself, since that would give a list of leaves the same root as the same
|
|
17
|
+
// list with its last leaf twice.
|
|
18
|
+
export const buildTree = (leafHashes, index) => {
|
|
19
|
+
if (leafHashes.length === 0) {
|
|
20
|
+
throw new Error('Cannot build Merkle root from empty list');
|
|
21
|
+
}
|
|
22
|
+
const auditPath = [];
|
|
23
|
+
let level = leafHashes;
|
|
24
|
+
let currentIndex = index;
|
|
25
|
+
while (level.length > 1) {
|
|
26
|
+
if (currentIndex !== undefined) {
|
|
27
|
+
const isLeftChild = currentIndex % 2 === 0;
|
|
28
|
+
const sibling = level[isLeftChild ? currentIndex + 1 : currentIndex - 1];
|
|
29
|
+
if (sibling !== undefined) {
|
|
30
|
+
auditPath.push({ hash: sibling, position: isLeftChild ? 'right' : 'left' });
|
|
31
|
+
}
|
|
32
|
+
currentIndex = Math.floor(currentIndex / 2);
|
|
33
|
+
}
|
|
34
|
+
const nextLevel = [];
|
|
35
|
+
for (let position = 0; position < level.length; position += 2) {
|
|
36
|
+
const left = level[position];
|
|
37
|
+
const right = level[position + 1];
|
|
38
|
+
if (left === undefined) {
|
|
39
|
+
throw new Error('Unexpected undefined hash while building Merkle tree');
|
|
40
|
+
}
|
|
41
|
+
nextLevel.push(right === undefined ? left : hashNode(left, right));
|
|
42
|
+
}
|
|
43
|
+
level = nextLevel;
|
|
44
|
+
}
|
|
45
|
+
const [root] = level;
|
|
46
|
+
if (root === undefined) {
|
|
47
|
+
throw new Error('Failed to compute Merkle root');
|
|
48
|
+
}
|
|
49
|
+
return { root, auditPath };
|
|
50
|
+
};
|
|
51
|
+
export const calculateRootFromAuditPath = (leafHash, auditPath) => auditPath.reduce((hash, sibling) => sibling.position === 'left' ? hashNode(sibling.hash, hash) : hashNode(hash, sibling.hash), leafHash);
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
import { buildTree, hashNode } from './merkleTreeVersion2.js';
|
|
3
|
+
// The tree of version 3 is the tree of version 2, but its root covers the size
|
|
4
|
+
// of the tree as well. A proof of version 3 locates its leaf by the index and
|
|
5
|
+
// the size of the tree, as RFC 9162 does, and the index of an event is its ID.
|
|
6
|
+
// RFC 9162 binds the index only where the size is known, though: a path that
|
|
7
|
+
// leads to the root from one leaf of a tree of one size can lead to it from a
|
|
8
|
+
// leaf at another index of a tree of another size. Certificate Transparency
|
|
9
|
+
// signs the size along with the root, but a root that the tool calculates is
|
|
10
|
+
// published as it is, so the root hashes the size in, behind a byte of its own,
|
|
11
|
+
// as RFC 6962 puts one in front of a leaf and of an inner node.
|
|
12
|
+
const rootPrefix = Buffer.from([0x02]);
|
|
13
|
+
// RFC 6962 encodes the size of a tree in 8 bytes, as it does in a tree head, so
|
|
14
|
+
// a tree has at most 2^64 - 1 leaves.
|
|
15
|
+
const treeSizeLength = 8;
|
|
16
|
+
const maximumTreeSize = 2n ** 64n - 1n;
|
|
17
|
+
const hashRoot = (treeSize, root) => {
|
|
18
|
+
const size = Buffer.alloc(treeSizeLength);
|
|
19
|
+
size.writeBigUInt64BE(treeSize);
|
|
20
|
+
return createHash('sha256').update(rootPrefix).update(size).update(root).digest();
|
|
21
|
+
};
|
|
22
|
+
export const buildVersion3Tree = (leafHashes, index) => {
|
|
23
|
+
const { root, auditPath } = buildTree(leafHashes, index);
|
|
24
|
+
return {
|
|
25
|
+
root: hashRoot(BigInt(leafHashes.length), root),
|
|
26
|
+
auditPath: auditPath.map(({ hash }) => hash),
|
|
27
|
+
};
|
|
28
|
+
};
|
|
29
|
+
// calculateVersion3Root calculates the root from the audit path of the leaf at
|
|
30
|
+
// the given index of a tree of the given size, as section 2.1.3.2 of RFC 9162
|
|
31
|
+
// verifies an inclusion proof, or returns undefined where the path does not fit
|
|
32
|
+
// that leaf in that tree. The side of each sibling follows from the index of the
|
|
33
|
+
// node on the path and the last index of its level: a node with an odd index is
|
|
34
|
+
// a right child, and so is the last node of a level, which has no partner on its
|
|
35
|
+
// right and is moved up unchanged until it is one. Both indexes halve from one
|
|
36
|
+
// level to the next, and the path fits where it ends at the level of the root.
|
|
37
|
+
export const calculateVersion3Root = (leafHash, leafIndex, treeSize, auditPath) => {
|
|
38
|
+
if (leafIndex >= treeSize || treeSize > maximumTreeSize) {
|
|
39
|
+
return undefined;
|
|
40
|
+
}
|
|
41
|
+
let index = leafIndex;
|
|
42
|
+
let lastIndex = treeSize - 1n;
|
|
43
|
+
let hash = leafHash;
|
|
44
|
+
for (const sibling of auditPath) {
|
|
45
|
+
if (lastIndex === 0n) {
|
|
46
|
+
return undefined;
|
|
47
|
+
}
|
|
48
|
+
if (index % 2n === 1n || index === lastIndex) {
|
|
49
|
+
hash = hashNode(sibling, hash);
|
|
50
|
+
while (index % 2n === 0n && index !== 0n) {
|
|
51
|
+
index /= 2n;
|
|
52
|
+
lastIndex /= 2n;
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
else {
|
|
56
|
+
hash = hashNode(hash, sibling);
|
|
57
|
+
}
|
|
58
|
+
index /= 2n;
|
|
59
|
+
lastIndex /= 2n;
|
|
60
|
+
}
|
|
61
|
+
if (lastIndex !== 0n) {
|
|
62
|
+
return undefined;
|
|
63
|
+
}
|
|
64
|
+
return hashRoot(treeSize, hash);
|
|
65
|
+
};
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
import { createHmac, randomBytes } from 'node:crypto';
|
|
2
|
+
import { access, open, readFile, unlink } from 'node:fs/promises';
|
|
3
|
+
// A salt key is 32 random bytes: RFC 2104 discourages keys for HMAC that are
|
|
4
|
+
// shorter than the output of the hash, and longer ones add nothing. It is kept
|
|
5
|
+
// as text of 64 hexadecimal digits, so that it can be stored wherever a password
|
|
6
|
+
// can, and whitespace around them is left out, so that a final newline, which
|
|
7
|
+
// one editor adds and another does not, does not make the same key another one.
|
|
8
|
+
const saltKeyLength = 32;
|
|
9
|
+
const saltKeyPattern = /^[0-9a-f]{64}$/iu;
|
|
10
|
+
export const readSaltKey = async (filePath) => {
|
|
11
|
+
try {
|
|
12
|
+
await access(filePath);
|
|
13
|
+
}
|
|
14
|
+
catch {
|
|
15
|
+
throw new Error(`File not found: ${filePath}`);
|
|
16
|
+
}
|
|
17
|
+
const digits = (await readFile(filePath, 'utf-8')).trim();
|
|
18
|
+
// The message does not show what the file holds, since that may be a key
|
|
19
|
+
// that was only written down wrong.
|
|
20
|
+
if (!saltKeyPattern.test(digits)) {
|
|
21
|
+
throw new Error(`the salt key in ${filePath} has to be 64 hexadecimal digits`);
|
|
22
|
+
}
|
|
23
|
+
return Buffer.from(digits, 'hex');
|
|
24
|
+
};
|
|
25
|
+
// writeSaltKey never overwrites a file, since a root whose leaves were salted
|
|
26
|
+
// with a key gets no further proof once the key is gone. Nobody may write the
|
|
27
|
+
// file, so that it is not replaced by accident either, as by a command that
|
|
28
|
+
// redirects its output into it, and on Linux and macOS, only its owner may read
|
|
29
|
+
// it. Windows knows only whether a file is read-only, so there, who may read it
|
|
30
|
+
// follows from the permissions of its folder.
|
|
31
|
+
export const writeSaltKey = async (filePath) => {
|
|
32
|
+
const digits = randomBytes(saltKeyLength).toString('hex');
|
|
33
|
+
let file;
|
|
34
|
+
try {
|
|
35
|
+
file = await open(filePath, 'wx', 0o400);
|
|
36
|
+
}
|
|
37
|
+
catch (error) {
|
|
38
|
+
if (error.code === 'EEXIST') {
|
|
39
|
+
throw new Error(`${filePath} already exists`);
|
|
40
|
+
}
|
|
41
|
+
throw error;
|
|
42
|
+
}
|
|
43
|
+
// A root may be calculated and published with the key right after it was
|
|
44
|
+
// written, so the content of the file reaches the disk before the key counts as
|
|
45
|
+
// written. Its entry in the folder is not flushed on its own, so whether a key
|
|
46
|
+
// written right before a power failure is found afterwards is left to the file
|
|
47
|
+
// system. A file that could not be written completely is removed again, so that
|
|
48
|
+
// it neither passes for a key nor keeps the next attempt from writing one. It
|
|
49
|
+
// is closed first, which Windows needs to remove it, but a failure to close it
|
|
50
|
+
// keeps it neither from being removed nor from reporting why writing it failed.
|
|
51
|
+
try {
|
|
52
|
+
await file.writeFile(`${digits}\n`);
|
|
53
|
+
await file.sync();
|
|
54
|
+
}
|
|
55
|
+
catch (error) {
|
|
56
|
+
await file.close().catch(() => undefined);
|
|
57
|
+
await unlink(filePath);
|
|
58
|
+
throw error;
|
|
59
|
+
}
|
|
60
|
+
await file.close();
|
|
61
|
+
};
|
|
62
|
+
// deriveSalt derives the salt of a leaf from the key and the stored hash of its
|
|
63
|
+
// event, so that no salt has to be kept: whoever holds the key gets the same
|
|
64
|
+
// salts again, and whoever does not cannot tell them from random ones.
|
|
65
|
+
export const deriveSalt = (saltKey, storedHash) => createHmac('sha256', saltKey).update(storedHash, 'utf-8').digest();
|
package/dist/lib/tree.js
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
import { Flags } from '@oclif/core';
|
|
2
|
+
import { isKnownStoredHash } from './hash.js';
|
|
3
|
+
import { readSaltKey } from './saltKey.js';
|
|
4
|
+
// The flags that name the tree a command calculates: its version, and the file
|
|
5
|
+
// with the key that salts its leaves. Version 3 is the default, and versions 1
|
|
6
|
+
// and 2 stay for the roots that the releases before 4.0 calculated. Only the
|
|
7
|
+
// commands that calculate a tree take them, since a proof names its version and
|
|
8
|
+
// carries its salt.
|
|
9
|
+
export const getTreeFlags = (treeVersionDescription) => ({
|
|
10
|
+
'salt-key-file': Flags.string({
|
|
11
|
+
description: 'File with the salt key to salt the leaves of the tree with',
|
|
12
|
+
}),
|
|
13
|
+
'tree-version': Flags.string({
|
|
14
|
+
description: treeVersionDescription,
|
|
15
|
+
options: ['1', '2', '3'],
|
|
16
|
+
default: '3',
|
|
17
|
+
}),
|
|
18
|
+
});
|
|
19
|
+
// getTree returns the tree that the flags name, or why they name none: the tree
|
|
20
|
+
// of version 1 has no salts, so a salt key file for it would be ignored, which
|
|
21
|
+
// would leave a root unsalted that was meant to be salted.
|
|
22
|
+
export const getTree = (flags) => {
|
|
23
|
+
const saltKeyFile = flags['salt-key-file'];
|
|
24
|
+
if (flags['tree-version'] === '1') {
|
|
25
|
+
if (saltKeyFile !== undefined) {
|
|
26
|
+
return { refusal: '--salt-key-file and --tree-version=1 cannot be used together' };
|
|
27
|
+
}
|
|
28
|
+
return { treeVersion: 1 };
|
|
29
|
+
}
|
|
30
|
+
return { treeVersion: flags['tree-version'] === '2' ? 2 : 3, saltKeyFile };
|
|
31
|
+
};
|
|
32
|
+
// readTreeOptions reads the salt key of a tree, if it has one. A command reads it
|
|
33
|
+
// before its source, so that a key file that cannot be read stops it before it
|
|
34
|
+
// reads a large backup or database for nothing.
|
|
35
|
+
export const readTreeOptions = async (tree) => {
|
|
36
|
+
if (tree.treeVersion === 1 || tree.saltKeyFile === undefined) {
|
|
37
|
+
return { treeVersion: tree.treeVersion };
|
|
38
|
+
}
|
|
39
|
+
return { treeVersion: tree.treeVersion, saltKey: await readSaltKey(tree.saltKeyFile) };
|
|
40
|
+
};
|
|
41
|
+
// getTreeHashes returns the stored hashes of the events, as the leaves of the
|
|
42
|
+
// tree, or throws where the tree could not prove each of them. The trees of
|
|
43
|
+
// versions 2 and 3 take only hashes of a form they know, since only they keep
|
|
44
|
+
// the data of a leaf unambiguous, which is why verify-proof refuses any other.
|
|
45
|
+
// The tree of version 3 also takes the events only where their IDs are their
|
|
46
|
+
// positions, as EventSourcingDB numbers them, in the notation in which it
|
|
47
|
+
// writes them, since a proof of version 3 names the position of its leaf by the
|
|
48
|
+
// ID of its event. Otherwise, the root would cover leaves that no proof could
|
|
49
|
+
// prove. The tree of version 1 takes any events, as it always did, and so does
|
|
50
|
+
// the tree of version 2 as far as their IDs are concerned, as release 3.0 did.
|
|
51
|
+
export const getTreeHashes = (events, treeOptions) => {
|
|
52
|
+
if (treeOptions.treeVersion !== 1) {
|
|
53
|
+
for (const [position, event] of events.entries()) {
|
|
54
|
+
if (treeOptions.treeVersion === 3 && event.id !== String(position)) {
|
|
55
|
+
throw new Error(`event ${event.id} is at position ${position} of the source, but the IDs of the events have to run from 0 without gaps`);
|
|
56
|
+
}
|
|
57
|
+
if (!isKnownStoredHash(event.hash)) {
|
|
58
|
+
throw new Error(`event ${event.id} has no hash of version 1 or 2`);
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
return events.map(({ hash }) => hash);
|
|
63
|
+
};
|