eventsourcingdb-merkle 3.0.0 → 4.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -30
- package/dist/commands/get-proof.js +9 -2
- package/dist/commands/merkle-root.js +1 -1
- package/dist/commands/verify-proof.js +10 -9
- package/dist/lib/merkleTree.js +108 -35
- package/dist/lib/merkleTreeVersion3.js +65 -0
- package/dist/lib/tree.js +25 -16
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -79,11 +79,13 @@ eventsourcingdb-merkle merkle-root --backup=backup.json
|
|
|
79
79
|
The command outputs the Merkle root, the total number of events, and the ID of the last event it covers:
|
|
80
80
|
|
|
81
81
|
```
|
|
82
|
-
Merkle Root: esdb:merkle:
|
|
82
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
83
83
|
Total events: 4
|
|
84
84
|
Last event ID: 3
|
|
85
85
|
```
|
|
86
86
|
|
|
87
|
+
The root covers the number of events as well, so that a proof against it can name the position of its event, as described under [Verifying Merkle Proofs](#verifying-merkle-proofs).
|
|
88
|
+
|
|
87
89
|
#### Calculating the Merkle Root of an Earlier State
|
|
88
90
|
|
|
89
91
|
A root that you published once has to be calculated again later, for instance to generate proofs against it. Since events are never changed, a newer backup, or the database itself, holds the events of an older backup, followed by those written since. Pass the last event ID that the root covered with `--last-event-id`, and the command calculates the root over the events up to it, which is the root of the older backup:
|
|
@@ -107,12 +109,12 @@ eventsourcingdb-merkle merkle-root --backup=backup.json --salt-key-file=merkle.k
|
|
|
107
109
|
|
|
108
110
|
The key file holds 64 hexadecimal digits. `generate-salt-key` never overwrites a file, and it lets nobody write the key, and on Linux and macOS, only the owner of the file read it. Windows knows only whether a file is read-only, so there, who can read the key follows from the permissions of its folder. Keep the key as long as the root, and as safely as a password: a root calculated with it is calculated again with that key only, so without it, no further proof can be generated for the root, and whoever has it can calculate the salt of every leaf. A proof carries the salt of its own leaf, so verifying it needs no key.
|
|
109
111
|
|
|
110
|
-
#### Calculating a Root of Version
|
|
112
|
+
#### Calculating a Root of an Earlier Version
|
|
111
113
|
|
|
112
|
-
Releases before
|
|
114
|
+
Releases before 4.0 built the tree in other ways. Release 3.0 built version 2 of the tree, which has the leaves and inner nodes of version 3, but a root that does not cover the number of events, and proofs that follow the sides of their siblings, so that they do not bind the ID of their event. Releases before 3.0 built version 1, which hashes leaves and inner nodes alike, and pairs the last node of a level with itself where it has no partner, so that a list of hashes has the same root as the same list with its last hash twice. A root of version 3 starts with `esdb:merkle:v3:`, a root of version 2 with `esdb:merkle:v2:`, and a root of version 1 has no prefix. Version 1 knows no salts. To calculate a root of an earlier version again, or to prove an event against one, pass `--tree-version=1` or `--tree-version=2`:
|
|
113
115
|
|
|
114
116
|
```shell
|
|
115
|
-
eventsourcingdb-merkle merkle-root --backup=backup.json --tree-version=
|
|
117
|
+
eventsourcingdb-merkle merkle-root --backup=backup.json --tree-version=2
|
|
116
118
|
```
|
|
117
119
|
|
|
118
120
|
### Validating Individual Events
|
|
@@ -181,21 +183,24 @@ The command outputs the proof in a human-readable format:
|
|
|
181
183
|
Merkle Proof for Event 42
|
|
182
184
|
|
|
183
185
|
Event Hash: esdb:hash:v2:abc123...
|
|
184
|
-
Merkle Root: esdb:merkle:
|
|
186
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
187
|
+
Last Event ID: 99
|
|
185
188
|
|
|
186
189
|
Sibling Hashes (3):
|
|
187
|
-
1. def456...
|
|
188
|
-
2. ghi789...
|
|
189
|
-
3. jkl012...
|
|
190
|
+
1. def456...
|
|
191
|
+
2. ghi789...
|
|
192
|
+
3. jkl012...
|
|
190
193
|
```
|
|
191
194
|
|
|
195
|
+
A proof of version 3 names the ID of the last event that its root covers, which fixes the position of its event in the tree. A proof of version 1 or 2 names no last event ID, but the side of each sibling instead, such as `def456... (left)`.
|
|
196
|
+
|
|
192
197
|
To prove an event against the root of an earlier state, pass its last event ID with `--last-event-id`, as for `merkle-root`. An event after it is not part of that tree, so the command refuses to prove it:
|
|
193
198
|
|
|
194
199
|
```shell
|
|
195
200
|
eventsourcingdb-merkle get-proof --backup=backup.json --event-id=42 --last-event-id=1234566
|
|
196
201
|
```
|
|
197
202
|
|
|
198
|
-
To prove an event against a root whose leaves are salted, pass the same key with `--salt-key-file`, and to prove it against a root of version
|
|
203
|
+
To prove an event against a root whose leaves are salted, pass the same key with `--salt-key-file`, and to prove it against a root of an earlier version, pass `--tree-version`, as for `merkle-root`. A proof against a salted root carries the salt of the leaf of its event, right after the hash of the event:
|
|
199
204
|
|
|
200
205
|
```shell
|
|
201
206
|
eventsourcingdb-merkle get-proof --backup=backup.json --event-id=42 --salt-key-file=merkle.key
|
|
@@ -206,7 +211,8 @@ Merkle Proof for Event 42
|
|
|
206
211
|
|
|
207
212
|
Event Hash: esdb:hash:v2:abc123...
|
|
208
213
|
Salt: 9c1f0b...
|
|
209
|
-
Merkle Root: esdb:merkle:
|
|
214
|
+
Merkle Root: esdb:merkle:v3:5f0c3e...
|
|
215
|
+
Last Event ID: 99
|
|
210
216
|
```
|
|
211
217
|
|
|
212
218
|
#### Outputting JSON
|
|
@@ -224,24 +230,16 @@ The output can then be saved to a file and shared with others:
|
|
|
224
230
|
"eventId": "42",
|
|
225
231
|
"eventHash": "esdb:hash:v2:abc123...",
|
|
226
232
|
"siblingHashes": [
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
},
|
|
231
|
-
{
|
|
232
|
-
"hash": "ghi789...",
|
|
233
|
-
"position": "right"
|
|
234
|
-
},
|
|
235
|
-
{
|
|
236
|
-
"hash": "jkl012...",
|
|
237
|
-
"position": "left"
|
|
238
|
-
}
|
|
233
|
+
"def456...",
|
|
234
|
+
"ghi789...",
|
|
235
|
+
"jkl012..."
|
|
239
236
|
],
|
|
240
|
-
"merkleRoot": "esdb:merkle:
|
|
237
|
+
"merkleRoot": "esdb:merkle:v3:6a0e4b2f9d1c...",
|
|
238
|
+
"lastEventId": "99"
|
|
241
239
|
}
|
|
242
240
|
```
|
|
243
241
|
|
|
244
|
-
A proof against a salted root holds its salt in `salt`, right after `eventHash`.
|
|
242
|
+
A proof against a salted root holds its salt in `salt`, right after `eventHash`. A proof of version 1 or 2 holds no `lastEventId`, and every sibling in it is an object with its `hash` and its `position`, `left` or `right`.
|
|
245
243
|
|
|
246
244
|
### Verifying Merkle Proofs
|
|
247
245
|
|
|
@@ -256,10 +254,10 @@ eventsourcingdb-merkle verify-proof proof.json
|
|
|
256
254
|
Or with a JSON string:
|
|
257
255
|
|
|
258
256
|
```shell
|
|
259
|
-
eventsourcingdb-merkle verify-proof '{"eventId":"42","eventHash":"...","siblingHashes":[],"merkleRoot":"
|
|
257
|
+
eventsourcingdb-merkle verify-proof '{"eventId":"42","eventHash":"...","siblingHashes":["..."],"merkleRoot":"esdb:merkle:v3:...","lastEventId":"99"}'
|
|
260
258
|
```
|
|
261
259
|
|
|
262
|
-
The command verifies proofs of
|
|
260
|
+
The command verifies proofs of every version of the tree, which it tells apart by their root, and takes the salt of a salted proof from the proof itself, so it needs neither the backup nor the key.
|
|
263
261
|
|
|
264
262
|
If the proof is valid, the command confirms this and displays the event details:
|
|
265
263
|
|
|
@@ -267,7 +265,8 @@ If the proof is valid, the command confirms this and displays the event details:
|
|
|
267
265
|
✓ Merkle proof is valid
|
|
268
266
|
Event ID: 42
|
|
269
267
|
Event Hash: esdb:hash:v2:abc123...
|
|
270
|
-
Merkle Root: esdb:merkle:
|
|
268
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
269
|
+
Last Event ID: 99
|
|
271
270
|
```
|
|
272
271
|
|
|
273
272
|
If the proof is invalid, the command indicates this with a non-zero exit code:
|
|
@@ -276,9 +275,12 @@ If the proof is invalid, the command indicates this with a non-zero exit code:
|
|
|
276
275
|
✗ Merkle proof is invalid
|
|
277
276
|
Event ID: 42
|
|
278
277
|
Event Hash: esdb:hash:v2:abc123...
|
|
279
|
-
Merkle Root: esdb:merkle:
|
|
278
|
+
Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
|
|
279
|
+
Last Event ID: 99
|
|
280
280
|
```
|
|
281
281
|
|
|
282
|
+
A valid proof shows that the event is part of the tree with the root it names, so compare that root with the one that was published. In version 3, it also shows that the event is the one with its ID, in a tree that ends at the last event ID: its path leads to the root from the leaf at that position only, and the root covers the number of events, so neither ID can be changed without the proof becoming invalid. A proof of version 1 or 2 does not bind the ID of its event, so whoever hands it on could change it, and its ID is only as trustworthy as whoever gave you the proof.
|
|
283
|
+
|
|
282
284
|
## Understanding the Technical Details
|
|
283
285
|
|
|
284
286
|
The tool implements Merkle trees using SHA-256 hashing, following the EventSourcingDB specification for event hash calculation, which knows two versions. An event keeps the version of the hash it was written with, so the tool verifies every hash in the version it names:
|
|
@@ -288,11 +290,14 @@ The tool implements Merkle trees using SHA-256 hashing, following the EventSourc
|
|
|
288
290
|
|
|
289
291
|
In both versions, the data is hashed as it appears in the backup or in the event given with `--event`, rather than parsed and serialized again, since that would change escape sequences such as `\u003c` for `<` and the notation of numbers. Only the whitespace between its tokens is left out, which EventSourcingDB never writes, so that an event formatted afterwards, as by a viewer or by `jq` without `-c`, still verifies.
|
|
290
292
|
|
|
291
|
-
The Merkle
|
|
293
|
+
The Merkle trees of versions 2 and 3 follow [RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), as the tree of [eventsourcingdb-auditing](https://github.com/thenativeweb/eventsourcingdb-auditing) does:
|
|
292
294
|
|
|
293
295
|
- The data of a leaf is the hash of its event as it is stored, prefix included, as text in UTF-8. Where the leaves are salted, its salt comes first: the HMAC-SHA256 of that text under the salt key. Since nothing marks where the salt ends, the tree takes only hashes in the form of version 1 or 2, which give the data of a leaf other lengths with a salt than without one, and refuses an event with any other hash, as `verify-proof` refuses a proof of one.
|
|
294
296
|
- The hash of a leaf is the SHA-256 of the byte `0x00` followed by its data, and the hash of an inner node is the SHA-256 of the byte `0x01` followed by the hashes of its two children, so that neither can pass for the other.
|
|
295
|
-
- The tree splits its leaves at the largest power of two smaller than their number, and never duplicates a node.
|
|
297
|
+
- The tree splits its leaves at the largest power of two smaller than their number, and never duplicates a node.
|
|
298
|
+
- The root of version 2 is the hash of the top node in hexadecimal, after the prefix `esdb:merkle:v2:`. The root of version 3 is the SHA-256 of the byte `0x02`, the number of leaves as 8 bytes, big-endian, and the hash of the top node, in hexadecimal, after the prefix `esdb:merkle:v3:`.
|
|
299
|
+
|
|
300
|
+
A proof of version 3 is verified as [RFC 9162](https://www.rfc-editor.org/rfc/rfc9162), section 2.1.3.2, verifies an inclusion proof, with the event ID as the index of its leaf and one more than the last event ID as the size of the tree, from which the side of every sibling follows. EventSourcingDB numbers its events from 0 without gaps, so the leaf at an index is the event with that ID, and the tree of version 3 refuses a source whose IDs do not run like that. Certificate Transparency, for which RFC 9162 was written, signs the size of a tree together with its root. The root of version 3 hashes it in instead, since without it, a proof could bring a size of its own, under which its path would lead to the root from the leaf at another index.
|
|
296
301
|
|
|
297
302
|
The tree of version 1 takes the stored hashes as its leaves, and hashes the concatenation of the hashes of its two children, as text, for an inner node. Where a level has an odd number of nodes, it pairs the last one with itself.
|
|
298
303
|
|
|
@@ -22,7 +22,7 @@ export default class GetProof extends BaseCommand {
|
|
|
22
22
|
description: 'ID of the last event the root covers, to prove against the root of an earlier state',
|
|
23
23
|
}),
|
|
24
24
|
json: Flags.boolean({ description: 'Output proof as JSON' }),
|
|
25
|
-
...getTreeFlags('Version of the Merkle tree: 1 to prove against roots that releases before 3.0 calculated'),
|
|
25
|
+
...getTreeFlags('Version of the Merkle tree: 1 to prove against roots that releases before 3.0 calculated, 2 to prove against roots that release 3.0 calculated'),
|
|
26
26
|
};
|
|
27
27
|
static args = {};
|
|
28
28
|
async run() {
|
|
@@ -84,10 +84,17 @@ export default class GetProof extends BaseCommand {
|
|
|
84
84
|
this.log(`Salt: ${proof.salt}`);
|
|
85
85
|
}
|
|
86
86
|
this.log(`Merkle Root: ${proof.merkleRoot}`);
|
|
87
|
+
if ('lastEventId' in proof) {
|
|
88
|
+
this.log(`Last Event ID: ${proof.lastEventId}`);
|
|
89
|
+
}
|
|
87
90
|
this.log('');
|
|
88
91
|
this.log(`Sibling Hashes (${proof.siblingHashes.length}):`);
|
|
92
|
+
// The siblings of a proof of version 3 have no sides, since they follow
|
|
93
|
+
// from the position of the event and the size of the tree.
|
|
89
94
|
for (const [index, sibling] of proof.siblingHashes.entries()) {
|
|
90
|
-
this.log(
|
|
95
|
+
this.log(typeof sibling === 'string'
|
|
96
|
+
? ` ${index + 1}. ${sibling}`
|
|
97
|
+
: ` ${index + 1}. ${sibling.hash} (${sibling.position})`);
|
|
91
98
|
}
|
|
92
99
|
}
|
|
93
100
|
catch (error) {
|
|
@@ -19,7 +19,7 @@ export default class MerkleRoot extends BaseCommand {
|
|
|
19
19
|
'last-event-id': Flags.string({
|
|
20
20
|
description: 'ID of the last event to cover, to calculate the root of an earlier state again',
|
|
21
21
|
}),
|
|
22
|
-
...getTreeFlags('Version of the Merkle tree: 1 for roots that releases before 3.0 calculated'),
|
|
22
|
+
...getTreeFlags('Version of the Merkle tree: 1 for roots that releases before 3.0 calculated, 2 for roots that release 3.0 calculated'),
|
|
23
23
|
};
|
|
24
24
|
static args = {};
|
|
25
25
|
async run() {
|
|
@@ -4,6 +4,7 @@ import { findProofStructureProblem, verifyMerkleProof } from '../lib/merkleTree.
|
|
|
4
4
|
export default class VerifyProof extends BaseCommand {
|
|
5
5
|
static description = 'Verify a Merkle proof';
|
|
6
6
|
static examples = [
|
|
7
|
+
'<%= config.bin %> <%= command.id %> \'{"eventId":"42","eventHash":"...","siblingHashes":["..."],"merkleRoot":"esdb:merkle:v3:...","lastEventId":"99"}\'',
|
|
7
8
|
'<%= config.bin %> <%= command.id %> \'{"eventId":"42","eventHash":"...","siblingHashes":[],"merkleRoot":"..."}\'',
|
|
8
9
|
'<%= config.bin %> <%= command.id %> proof.json',
|
|
9
10
|
];
|
|
@@ -35,18 +36,18 @@ export default class VerifyProof extends BaseCommand {
|
|
|
35
36
|
return;
|
|
36
37
|
}
|
|
37
38
|
const isValid = verifyMerkleProof(proof);
|
|
38
|
-
|
|
39
|
-
this.log(`✗ Merkle proof is invalid`);
|
|
40
|
-
this.log(`Event ID: ${proof.eventId}`);
|
|
41
|
-
this.log(`Event Hash: ${proof.eventHash}`);
|
|
42
|
-
this.log(`Merkle Root: ${proof.merkleRoot}`);
|
|
43
|
-
process.exitCode = 1;
|
|
44
|
-
return;
|
|
45
|
-
}
|
|
46
|
-
this.log(`✓ Merkle proof is valid`);
|
|
39
|
+
this.log(isValid ? '✓ Merkle proof is valid' : '✗ Merkle proof is invalid');
|
|
47
40
|
this.log(`Event ID: ${proof.eventId}`);
|
|
48
41
|
this.log(`Event Hash: ${proof.eventHash}`);
|
|
49
42
|
this.log(`Merkle Root: ${proof.merkleRoot}`);
|
|
43
|
+
// The root of version 3 covers the size of its tree, so the ID of the last
|
|
44
|
+
// event tells which state of the store it is the root of.
|
|
45
|
+
if ('lastEventId' in proof) {
|
|
46
|
+
this.log(`Last Event ID: ${proof.lastEventId}`);
|
|
47
|
+
}
|
|
48
|
+
if (!isValid) {
|
|
49
|
+
process.exitCode = 1;
|
|
50
|
+
}
|
|
50
51
|
}
|
|
51
52
|
catch (error) {
|
|
52
53
|
this.log(`✗ Failed to verify Merkle proof: ${error instanceof Error ? error.message : String(error)}`);
|
package/dist/lib/merkleTree.js
CHANGED
|
@@ -2,11 +2,13 @@ import { isKnownStoredHash } from './hash.js';
|
|
|
2
2
|
import { isObject } from './isObject.js';
|
|
3
3
|
import { buildVersion1Root, generateVersion1Proof, verifyVersion1Proof, } from './merkleTreeVersion1.js';
|
|
4
4
|
import { buildTree, calculateRootFromAuditPath, hashLeaf, } from './merkleTreeVersion2.js';
|
|
5
|
+
import { buildVersion3Tree, calculateVersion3Root } from './merkleTreeVersion3.js';
|
|
5
6
|
import { deriveSalt } from './saltKey.js';
|
|
6
7
|
// A root that names the version of its tree starts with this prefix. A root
|
|
7
8
|
// without it is of version 1, which predates the prefix.
|
|
8
9
|
const merkleRootPrefix = 'esdb:merkle:';
|
|
9
10
|
const merkleRootVersion2Prefix = `${merkleRootPrefix}v2:`;
|
|
11
|
+
const merkleRootVersion3Prefix = `${merkleRootPrefix}v3:`;
|
|
10
12
|
// A salt and the hash of a node are SHA-256 digests, which a proof carries in
|
|
11
13
|
// hexadecimal. Decoding takes text that is not exactly that only as far as it
|
|
12
14
|
// can, so a proof that holds more than the digest would still verify.
|
|
@@ -17,18 +19,23 @@ export class UnsupportedMerkleTreeVersionError extends Error {
|
|
|
17
19
|
this.name = 'UnsupportedMerkleTreeVersionError';
|
|
18
20
|
}
|
|
19
21
|
}
|
|
20
|
-
// The data of a leaf of
|
|
21
|
-
// covers both versions of the hash alike, preceded by its salt where
|
|
22
|
-
// are salted.
|
|
22
|
+
// The data of a leaf of versions 2 and 3 is the stored hash of its event, as
|
|
23
|
+
// text, which covers both versions of the hash alike, preceded by its salt where
|
|
24
|
+
// the leaves are salted.
|
|
23
25
|
const getLeafHash = (storedHash, salt) => hashLeaf(Buffer.concat([salt ?? Buffer.alloc(0), Buffer.from(storedHash, 'utf-8')]));
|
|
24
26
|
const getSalt = (saltKey, storedHash) => saltKey === undefined ? undefined : deriveSalt(saltKey, storedHash);
|
|
25
27
|
const getLeafHashes = (hashes, saltKey) => hashes.map(hash => getLeafHash(hash, getSalt(saltKey, hash)));
|
|
26
28
|
const formatVersion2Root = (root) => `${merkleRootVersion2Prefix}${root.toString('hex')}`;
|
|
29
|
+
const formatVersion3Root = (root) => `${merkleRootVersion3Prefix}${root.toString('hex')}`;
|
|
27
30
|
export const calculateMerkleRoot = (hashes, options) => {
|
|
28
31
|
if (options.treeVersion === 1) {
|
|
29
32
|
return buildVersion1Root(hashes);
|
|
30
33
|
}
|
|
31
|
-
|
|
34
|
+
const leafHashes = getLeafHashes(hashes, options.saltKey);
|
|
35
|
+
if (options.treeVersion === 2) {
|
|
36
|
+
return formatVersion2Root(buildTree(leafHashes).root);
|
|
37
|
+
}
|
|
38
|
+
return formatVersion3Root(buildVersion3Tree(leafHashes).root);
|
|
32
39
|
};
|
|
33
40
|
export const generateMerkleProof = (hashes, index, options) => {
|
|
34
41
|
const eventHash = hashes[index];
|
|
@@ -39,15 +46,30 @@ export const generateMerkleProof = (hashes, index, options) => {
|
|
|
39
46
|
return generateVersion1Proof(hashes, eventHash);
|
|
40
47
|
}
|
|
41
48
|
const salt = getSalt(options.saltKey, eventHash);
|
|
42
|
-
const
|
|
43
|
-
return {
|
|
49
|
+
const leaf = {
|
|
44
50
|
eventHash,
|
|
45
51
|
...(salt === undefined ? {} : { salt: salt.toString('hex') }),
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
})
|
|
50
|
-
|
|
52
|
+
};
|
|
53
|
+
const leafHashes = getLeafHashes(hashes, options.saltKey);
|
|
54
|
+
if (options.treeVersion === 2) {
|
|
55
|
+
const { root, auditPath } = buildTree(leafHashes, index);
|
|
56
|
+
return {
|
|
57
|
+
...leaf,
|
|
58
|
+
siblingHashes: auditPath.map(({ hash, position }) => ({
|
|
59
|
+
hash: hash.toString('hex'),
|
|
60
|
+
position,
|
|
61
|
+
})),
|
|
62
|
+
merkleRoot: formatVersion2Root(root),
|
|
63
|
+
};
|
|
64
|
+
}
|
|
65
|
+
// The leaf at an index of the tree of version 3 is the event with that ID, so
|
|
66
|
+
// the last event of the tree has the ID one below the number of its leaves.
|
|
67
|
+
const { root, auditPath } = buildVersion3Tree(leafHashes, index);
|
|
68
|
+
return {
|
|
69
|
+
...leaf,
|
|
70
|
+
siblingHashes: auditPath.map(hash => hash.toString('hex')),
|
|
71
|
+
merkleRoot: formatVersion3Root(root),
|
|
72
|
+
lastEventId: String(hashes.length - 1),
|
|
51
73
|
};
|
|
52
74
|
};
|
|
53
75
|
// getMerkleTreeVersion tells the version of the tree that a root names, the way
|
|
@@ -60,21 +82,37 @@ export const getMerkleTreeVersion = (merkleRoot) => {
|
|
|
60
82
|
if (merkleRoot.startsWith(merkleRootVersion2Prefix)) {
|
|
61
83
|
return 2;
|
|
62
84
|
}
|
|
85
|
+
if (merkleRoot.startsWith(merkleRootVersion3Prefix)) {
|
|
86
|
+
return 3;
|
|
87
|
+
}
|
|
63
88
|
const [version = ''] = merkleRoot.slice(merkleRootPrefix.length).split(':');
|
|
64
89
|
throw new UnsupportedMerkleTreeVersionError(version);
|
|
65
90
|
};
|
|
66
|
-
const
|
|
67
|
-
|
|
68
|
-
|
|
91
|
+
const isDigest = (value) => typeof value === 'string' && digestPattern.test(value);
|
|
92
|
+
const isText = (value) => typeof value === 'string';
|
|
93
|
+
// A sibling of version 1 or 2 is an object with its hash and its side. The hash
|
|
94
|
+
// of a sibling of version 2 is a digest, but that of version 1 has no fixed
|
|
95
|
+
// form, since the tree of version 1 takes any stored hash as a leaf, so the
|
|
96
|
+
// first sibling of a proof may be the stored hash of a neighbor.
|
|
97
|
+
const isSiblingWithSide = (isHash) => (sibling) => isObject(sibling) &&
|
|
98
|
+
isHash(sibling.hash) &&
|
|
69
99
|
(sibling.position === 'left' || sibling.position === 'right');
|
|
100
|
+
// An ID in a proof of version 3 is written as EventSourcingDB writes IDs, since
|
|
101
|
+
// get-proof writes it rather than anyone typing it, so that the same proof has
|
|
102
|
+
// one notation only.
|
|
103
|
+
const eventIdPattern = /^(0|[1-9][0-9]*)$/u;
|
|
104
|
+
const isEventId = (value) => typeof value === 'string' && eventIdPattern.test(value);
|
|
70
105
|
// findProofStructureProblem describes what keeps a proof from being one that
|
|
71
|
-
// can be verified, or returns undefined if nothing does.
|
|
72
|
-
//
|
|
73
|
-
//
|
|
74
|
-
//
|
|
75
|
-
//
|
|
76
|
-
//
|
|
77
|
-
//
|
|
106
|
+
// can be verified, or returns undefined if nothing does. Which fields a proof may
|
|
107
|
+
// or has to have depends on the version of its tree alone, so they are part of
|
|
108
|
+
// the structure, as is the form of every field. The form of the event hash in
|
|
109
|
+
// versions 2 and 3 matters for another reason: the data of a leaf is its salt
|
|
110
|
+
// followed by the stored hash, with nothing between them that tells where the
|
|
111
|
+
// salt ends, so without that form, a proof whose salt took the first bytes of
|
|
112
|
+
// the hash would pass for a proof of the rest of it. With it, the data of a leaf
|
|
113
|
+
// has other lengths with a salt than without one. Whether the fields fit one
|
|
114
|
+
// another, such as an event ID beyond the last one, is left to the verification,
|
|
115
|
+
// as RFC 9162 leaves it.
|
|
78
116
|
export const findProofStructureProblem = (proof) => {
|
|
79
117
|
if (!isObject(proof) ||
|
|
80
118
|
typeof proof.eventHash !== 'string' ||
|
|
@@ -84,36 +122,71 @@ export const findProofStructureProblem = (proof) => {
|
|
|
84
122
|
!Array.isArray(proof.siblingHashes)) {
|
|
85
123
|
return 'Must contain eventHash, merkleRoot, and siblingHashes';
|
|
86
124
|
}
|
|
125
|
+
const treeVersion = getMerkleTreeVersion(proof.merkleRoot);
|
|
87
126
|
const hasSalt = proof.salt !== undefined;
|
|
88
|
-
|
|
89
|
-
|
|
127
|
+
const hasLastEventId = proof.lastEventId !== undefined;
|
|
128
|
+
if (treeVersion === 1) {
|
|
129
|
+
if (hasSalt) {
|
|
130
|
+
return 'A proof of Merkle tree version 1 has no salt';
|
|
131
|
+
}
|
|
132
|
+
if (hasLastEventId) {
|
|
133
|
+
return 'A proof of Merkle tree version 1 has no last event ID';
|
|
134
|
+
}
|
|
135
|
+
if (!proof.siblingHashes.every(isSiblingWithSide(isText))) {
|
|
136
|
+
return 'Every sibling has to have a hash, and the position left or right';
|
|
137
|
+
}
|
|
138
|
+
return undefined;
|
|
139
|
+
}
|
|
140
|
+
if (treeVersion === 2 && hasLastEventId) {
|
|
141
|
+
return 'A proof of Merkle tree version 2 has no last event ID';
|
|
142
|
+
}
|
|
143
|
+
if (treeVersion === 3) {
|
|
144
|
+
if (proof.eventId === undefined || !hasLastEventId) {
|
|
145
|
+
return 'A proof of Merkle tree version 3 has to contain eventId and lastEventId';
|
|
146
|
+
}
|
|
147
|
+
if (!isEventId(proof.eventId)) {
|
|
148
|
+
return 'The event ID has to be digits alone, without a sign or leading zeros';
|
|
149
|
+
}
|
|
90
150
|
}
|
|
91
151
|
if (!isKnownStoredHash(proof.eventHash)) {
|
|
92
152
|
return 'The event hash has to be a hash of version 1 or 2';
|
|
93
153
|
}
|
|
94
|
-
if (hasSalt &&
|
|
154
|
+
if (hasSalt && !isDigest(proof.salt)) {
|
|
95
155
|
return 'The salt has to be 64 hexadecimal digits';
|
|
96
156
|
}
|
|
97
|
-
if (
|
|
98
|
-
return
|
|
157
|
+
if (treeVersion === 2) {
|
|
158
|
+
return proof.siblingHashes.every(isSiblingWithSide(isDigest))
|
|
159
|
+
? undefined
|
|
160
|
+
: 'Every sibling has to have a hash of 64 hexadecimal digits, and the position left or right';
|
|
161
|
+
}
|
|
162
|
+
if (!proof.siblingHashes.every(isDigest)) {
|
|
163
|
+
return 'Every sibling has to be a hash of 64 hexadecimal digits';
|
|
164
|
+
}
|
|
165
|
+
if (!isEventId(proof.lastEventId)) {
|
|
166
|
+
return 'The last event ID has to be digits alone, without a sign or leading zeros';
|
|
99
167
|
}
|
|
100
168
|
return undefined;
|
|
101
169
|
};
|
|
102
170
|
// verifyMerkleProof tells whether a proof leads from the hash of its event to
|
|
103
|
-
// its root, in the version of the tree that the root names
|
|
104
|
-
//
|
|
171
|
+
// its root, in the version of the tree that the root names, and in version 3,
|
|
172
|
+
// whether it does so from the leaf of its event in a tree that ends at its last
|
|
173
|
+
// event. A proof in which findProofStructureProblem finds a problem proves
|
|
174
|
+
// nothing, and one that passes it has the fields of its version.
|
|
105
175
|
export const verifyMerkleProof = (proof) => {
|
|
106
176
|
if (findProofStructureProblem(proof) !== undefined) {
|
|
107
177
|
return false;
|
|
108
178
|
}
|
|
109
|
-
|
|
179
|
+
const treeVersion = getMerkleTreeVersion(proof.merkleRoot);
|
|
180
|
+
if (treeVersion === 1) {
|
|
110
181
|
return verifyVersion1Proof(proof);
|
|
111
182
|
}
|
|
112
|
-
const auditPath = proof.siblingHashes.map(({ hash, position }) => ({
|
|
113
|
-
hash: Buffer.from(hash, 'hex'),
|
|
114
|
-
position,
|
|
115
|
-
}));
|
|
116
183
|
const salt = proof.salt === undefined ? undefined : Buffer.from(proof.salt, 'hex');
|
|
117
|
-
const
|
|
118
|
-
|
|
184
|
+
const leafHash = getLeafHash(proof.eventHash, salt);
|
|
185
|
+
if (treeVersion === 2) {
|
|
186
|
+
const auditPath = proof.siblingHashes.map(({ hash, position }) => ({ hash: Buffer.from(hash, 'hex'), position }));
|
|
187
|
+
return formatVersion2Root(calculateRootFromAuditPath(leafHash, auditPath)) === proof.merkleRoot;
|
|
188
|
+
}
|
|
189
|
+
const { eventId, lastEventId, siblingHashes } = proof;
|
|
190
|
+
const root = calculateVersion3Root(leafHash, BigInt(eventId), BigInt(lastEventId) + 1n, siblingHashes.map(hash => Buffer.from(hash, 'hex')));
|
|
191
|
+
return root !== undefined && formatVersion3Root(root) === proof.merkleRoot;
|
|
119
192
|
};
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
import { buildTree, hashNode } from './merkleTreeVersion2.js';
|
|
3
|
+
// The tree of version 3 is the tree of version 2, but its root covers the size
|
|
4
|
+
// of the tree as well. A proof of version 3 locates its leaf by the index and
|
|
5
|
+
// the size of the tree, as RFC 9162 does, and the index of an event is its ID.
|
|
6
|
+
// RFC 9162 binds the index only where the size is known, though: a path that
|
|
7
|
+
// leads to the root from one leaf of a tree of one size can lead to it from a
|
|
8
|
+
// leaf at another index of a tree of another size. Certificate Transparency
|
|
9
|
+
// signs the size along with the root, but a root that the tool calculates is
|
|
10
|
+
// published as it is, so the root hashes the size in, behind a byte of its own,
|
|
11
|
+
// as RFC 6962 puts one in front of a leaf and of an inner node.
|
|
12
|
+
const rootPrefix = Buffer.from([0x02]);
|
|
13
|
+
// RFC 6962 encodes the size of a tree in 8 bytes, as it does in a tree head, so
|
|
14
|
+
// a tree has at most 2^64 - 1 leaves.
|
|
15
|
+
const treeSizeLength = 8;
|
|
16
|
+
const maximumTreeSize = 2n ** 64n - 1n;
|
|
17
|
+
const hashRoot = (treeSize, root) => {
|
|
18
|
+
const size = Buffer.alloc(treeSizeLength);
|
|
19
|
+
size.writeBigUInt64BE(treeSize);
|
|
20
|
+
return createHash('sha256').update(rootPrefix).update(size).update(root).digest();
|
|
21
|
+
};
|
|
22
|
+
export const buildVersion3Tree = (leafHashes, index) => {
|
|
23
|
+
const { root, auditPath } = buildTree(leafHashes, index);
|
|
24
|
+
return {
|
|
25
|
+
root: hashRoot(BigInt(leafHashes.length), root),
|
|
26
|
+
auditPath: auditPath.map(({ hash }) => hash),
|
|
27
|
+
};
|
|
28
|
+
};
|
|
29
|
+
// calculateVersion3Root calculates the root from the audit path of the leaf at
|
|
30
|
+
// the given index of a tree of the given size, as section 2.1.3.2 of RFC 9162
|
|
31
|
+
// verifies an inclusion proof, or returns undefined where the path does not fit
|
|
32
|
+
// that leaf in that tree. The side of each sibling follows from the index of the
|
|
33
|
+
// node on the path and the last index of its level: a node with an odd index is
|
|
34
|
+
// a right child, and so is the last node of a level, which has no partner on its
|
|
35
|
+
// right and is moved up unchanged until it is one. Both indexes halve from one
|
|
36
|
+
// level to the next, and the path fits where it ends at the level of the root.
|
|
37
|
+
export const calculateVersion3Root = (leafHash, leafIndex, treeSize, auditPath) => {
|
|
38
|
+
if (leafIndex >= treeSize || treeSize > maximumTreeSize) {
|
|
39
|
+
return undefined;
|
|
40
|
+
}
|
|
41
|
+
let index = leafIndex;
|
|
42
|
+
let lastIndex = treeSize - 1n;
|
|
43
|
+
let hash = leafHash;
|
|
44
|
+
for (const sibling of auditPath) {
|
|
45
|
+
if (lastIndex === 0n) {
|
|
46
|
+
return undefined;
|
|
47
|
+
}
|
|
48
|
+
if (index % 2n === 1n || index === lastIndex) {
|
|
49
|
+
hash = hashNode(sibling, hash);
|
|
50
|
+
while (index % 2n === 0n && index !== 0n) {
|
|
51
|
+
index /= 2n;
|
|
52
|
+
lastIndex /= 2n;
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
else {
|
|
56
|
+
hash = hashNode(hash, sibling);
|
|
57
|
+
}
|
|
58
|
+
index /= 2n;
|
|
59
|
+
lastIndex /= 2n;
|
|
60
|
+
}
|
|
61
|
+
if (lastIndex !== 0n) {
|
|
62
|
+
return undefined;
|
|
63
|
+
}
|
|
64
|
+
return hashRoot(treeSize, hash);
|
|
65
|
+
};
|
package/dist/lib/tree.js
CHANGED
|
@@ -2,18 +2,18 @@ import { Flags } from '@oclif/core';
|
|
|
2
2
|
import { isKnownStoredHash } from './hash.js';
|
|
3
3
|
import { readSaltKey } from './saltKey.js';
|
|
4
4
|
// The flags that name the tree a command calculates: its version, and the file
|
|
5
|
-
// with the key that salts its leaves. Version
|
|
6
|
-
//
|
|
7
|
-
// that calculate a tree take them, since a proof names its version and
|
|
8
|
-
// its salt.
|
|
5
|
+
// with the key that salts its leaves. Version 3 is the default, and versions 1
|
|
6
|
+
// and 2 stay for the roots that the releases before 4.0 calculated. Only the
|
|
7
|
+
// commands that calculate a tree take them, since a proof names its version and
|
|
8
|
+
// carries its salt.
|
|
9
9
|
export const getTreeFlags = (treeVersionDescription) => ({
|
|
10
10
|
'salt-key-file': Flags.string({
|
|
11
11
|
description: 'File with the salt key to salt the leaves of the tree with',
|
|
12
12
|
}),
|
|
13
13
|
'tree-version': Flags.string({
|
|
14
14
|
description: treeVersionDescription,
|
|
15
|
-
options: ['1', '2'],
|
|
16
|
-
default: '
|
|
15
|
+
options: ['1', '2', '3'],
|
|
16
|
+
default: '3',
|
|
17
17
|
}),
|
|
18
18
|
});
|
|
19
19
|
// getTree returns the tree that the flags name, or why they name none: the tree
|
|
@@ -27,7 +27,7 @@ export const getTree = (flags) => {
|
|
|
27
27
|
}
|
|
28
28
|
return { treeVersion: 1 };
|
|
29
29
|
}
|
|
30
|
-
return { treeVersion: 2, saltKeyFile };
|
|
30
|
+
return { treeVersion: flags['tree-version'] === '2' ? 2 : 3, saltKeyFile };
|
|
31
31
|
};
|
|
32
32
|
// readTreeOptions reads the salt key of a tree, if it has one. A command reads it
|
|
33
33
|
// before its source, so that a key file that cannot be read stops it before it
|
|
@@ -36,18 +36,27 @@ export const readTreeOptions = async (tree) => {
|
|
|
36
36
|
if (tree.treeVersion === 1 || tree.saltKeyFile === undefined) {
|
|
37
37
|
return { treeVersion: tree.treeVersion };
|
|
38
38
|
}
|
|
39
|
-
return { treeVersion:
|
|
39
|
+
return { treeVersion: tree.treeVersion, saltKey: await readSaltKey(tree.saltKeyFile) };
|
|
40
40
|
};
|
|
41
41
|
// getTreeHashes returns the stored hashes of the events, as the leaves of the
|
|
42
|
-
// tree
|
|
43
|
-
//
|
|
44
|
-
//
|
|
45
|
-
//
|
|
42
|
+
// tree, or throws where the tree could not prove each of them. The trees of
|
|
43
|
+
// versions 2 and 3 take only hashes of a form they know, since only they keep
|
|
44
|
+
// the data of a leaf unambiguous, which is why verify-proof refuses any other.
|
|
45
|
+
// The tree of version 3 also takes the events only where their IDs are their
|
|
46
|
+
// positions, as EventSourcingDB numbers them, in the notation in which it
|
|
47
|
+
// writes them, since a proof of version 3 names the position of its leaf by the
|
|
48
|
+
// ID of its event. Otherwise, the root would cover leaves that no proof could
|
|
49
|
+
// prove. The tree of version 1 takes any events, as it always did, and so does
|
|
50
|
+
// the tree of version 2 as far as their IDs are concerned, as release 3.0 did.
|
|
46
51
|
export const getTreeHashes = (events, treeOptions) => {
|
|
47
|
-
if (treeOptions.treeVersion
|
|
48
|
-
const event
|
|
49
|
-
|
|
50
|
-
|
|
52
|
+
if (treeOptions.treeVersion !== 1) {
|
|
53
|
+
for (const [position, event] of events.entries()) {
|
|
54
|
+
if (treeOptions.treeVersion === 3 && event.id !== String(position)) {
|
|
55
|
+
throw new Error(`event ${event.id} is at position ${position} of the source, but the IDs of the events have to run from 0 without gaps`);
|
|
56
|
+
}
|
|
57
|
+
if (!isKnownStoredHash(event.hash)) {
|
|
58
|
+
throw new Error(`event ${event.id} has no hash of version 1 or 2`);
|
|
59
|
+
}
|
|
51
60
|
}
|
|
52
61
|
}
|
|
53
62
|
return events.map(({ hash }) => hash);
|