eventsourcingdb-merkle 2.1.0 → 4.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -79,11 +79,13 @@ eventsourcingdb-merkle merkle-root --backup=backup.json
79
79
  The command outputs the Merkle root, the total number of events, and the ID of the last event it covers:
80
80
 
81
81
  ```
82
- Merkle Root: d1f6713af7ba3896ff238bb51f3986b2bad53b09bda7f23370e0a8b6eb97eeab
82
+ Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
83
83
  Total events: 4
84
84
  Last event ID: 3
85
85
  ```
86
86
 
87
+ The root covers the number of events as well, so that a proof against it can name the position of its event, as described under [Verifying Merkle Proofs](#verifying-merkle-proofs).
88
+
87
89
  #### Calculating the Merkle Root of an Earlier State
88
90
 
89
91
  A root that you published once has to be calculated again later, for instance to generate proofs against it. Since events are never changed, a newer backup, or the database itself, holds the events of an older backup, followed by those written since. Pass the last event ID that the root covered with `--last-event-id`, and the command calculates the root over the events up to it, which is the root of the older backup:
@@ -94,6 +96,27 @@ eventsourcingdb-merkle merkle-root --backup=backup-2025.json --last-event-id=3
94
96
 
95
97
  So you do not have to keep the older backup to prove against its root. If the backup or the database holds no event with that ID, the command refuses rather than calculating a root over other events.
96
98
 
99
+ #### Salting the Leaves
100
+
101
+ A proof names the hashes next to the path from its event to the root, and the first of them belongs to a neighboring event. That hash does not reveal the event, but whoever holds the proof and has a guess about the neighbor, such as its type, its subject, its time, and its data, can calculate its hash and so check the guess. Where your events can be guessed like that, salt the leaves of the tree: a random value in front of every leaf, which only a proof of that leaf discloses, makes the other hashes in a proof tell nothing.
102
+
103
+ The tool derives the salt of every leaf from a key, so that no salt has to be kept. Generate the key once, and pass it with `--salt-key-file` whenever you calculate the root, or a proof against it:
104
+
105
+ ```shell
106
+ eventsourcingdb-merkle generate-salt-key --output=merkle.key
107
+ eventsourcingdb-merkle merkle-root --backup=backup.json --salt-key-file=merkle.key
108
+ ```
109
+
110
+ The key file holds 64 hexadecimal digits. `generate-salt-key` never overwrites a file, and it lets nobody write the key, and on Linux and macOS, only the owner of the file read it. Windows knows only whether a file is read-only, so there, who can read the key follows from the permissions of its folder. Keep the key as long as the root, and as safely as a password: a root calculated with it is calculated again with that key only, so without it, no further proof can be generated for the root, and whoever has it can calculate the salt of every leaf. A proof carries the salt of its own leaf, so verifying it needs no key.
111
+
112
+ #### Calculating a Root of an Earlier Version
113
+
114
+ Releases before 4.0 built the tree in other ways. Release 3.0 built version 2 of the tree, which has the leaves and inner nodes of version 3, but a root that does not cover the number of events, and proofs that follow the sides of their siblings, so that they do not bind the ID of their event. Releases before 3.0 built version 1, which hashes leaves and inner nodes alike, and pairs the last node of a level with itself where it has no partner, so that a list of hashes has the same root as the same list with its last hash twice. A root of version 3 starts with `esdb:merkle:v3:`, a root of version 2 with `esdb:merkle:v2:`, and a root of version 1 has no prefix. Version 1 knows no salts. To calculate a root of an earlier version again, or to prove an event against one, pass `--tree-version=1` or `--tree-version=2`:
115
+
116
+ ```shell
117
+ eventsourcingdb-merkle merkle-root --backup=backup.json --tree-version=2
118
+ ```
119
+
97
120
  ### Validating Individual Events
98
121
 
99
122
  Sometimes you want to verify the integrity of a single event without processing the entire backup. The `validate-event-hash` command provides two ways to do this.
@@ -159,21 +182,39 @@ The command outputs the proof in a human-readable format:
159
182
  ```
160
183
  Merkle Proof for Event 42
161
184
 
162
- Event Hash: abc123...
163
- Merkle Root: d1f6713af7ba3896ff238bb51f3986b2bad53b09bda7f23370e0a8b6eb97eeab
185
+ Event Hash: esdb:hash:v2:abc123...
186
+ Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
187
+ Last Event ID: 99
164
188
 
165
189
  Sibling Hashes (3):
166
- 1. def456... (left)
167
- 2. ghi789... (right)
168
- 3. jkl012... (left)
190
+ 1. def456...
191
+ 2. ghi789...
192
+ 3. jkl012...
169
193
  ```
170
194
 
195
+ A proof of version 3 names the ID of the last event that its root covers, which fixes the position of its event in the tree. A proof of version 1 or 2 names no last event ID, but the side of each sibling instead, such as `def456... (left)`.
196
+
171
197
  To prove an event against the root of an earlier state, pass its last event ID with `--last-event-id`, as for `merkle-root`. An event after it is not part of that tree, so the command refuses to prove it:
172
198
 
173
199
  ```shell
174
200
  eventsourcingdb-merkle get-proof --backup=backup.json --event-id=42 --last-event-id=1234566
175
201
  ```
176
202
 
203
+ To prove an event against a root whose leaves are salted, pass the same key with `--salt-key-file`, and to prove it against a root of an earlier version, pass `--tree-version`, as for `merkle-root`. A proof against a salted root carries the salt of the leaf of its event, right after the hash of the event:
204
+
205
+ ```shell
206
+ eventsourcingdb-merkle get-proof --backup=backup.json --event-id=42 --salt-key-file=merkle.key
207
+ ```
208
+
209
+ ```
210
+ Merkle Proof for Event 42
211
+
212
+ Event Hash: esdb:hash:v2:abc123...
213
+ Salt: 9c1f0b...
214
+ Merkle Root: esdb:merkle:v3:5f0c3e...
215
+ Last Event ID: 99
216
+ ```
217
+
177
218
  #### Outputting JSON
178
219
 
179
220
  To get the proof as JSON for programmatic use, add the `--json` flag:
@@ -187,25 +228,19 @@ The output can then be saved to a file and shared with others:
187
228
  ```json
188
229
  {
189
230
  "eventId": "42",
190
- "eventHash": "abc123...",
231
+ "eventHash": "esdb:hash:v2:abc123...",
191
232
  "siblingHashes": [
192
- {
193
- "hash": "def456...",
194
- "position": "left"
195
- },
196
- {
197
- "hash": "ghi789...",
198
- "position": "right"
199
- },
200
- {
201
- "hash": "jkl012...",
202
- "position": "left"
203
- }
233
+ "def456...",
234
+ "ghi789...",
235
+ "jkl012..."
204
236
  ],
205
- "merkleRoot": "d1f6713af7ba3896ff238bb51f3986b2bad53b09bda7f23370e0a8b6eb97eeab"
237
+ "merkleRoot": "esdb:merkle:v3:6a0e4b2f9d1c...",
238
+ "lastEventId": "99"
206
239
  }
207
240
  ```
208
241
 
242
+ A proof against a salted root holds its salt in `salt`, right after `eventHash`. A proof of version 1 or 2 holds no `lastEventId`, and every sibling in it is an object with its `hash` and its `position`, `left` or `right`.
243
+
209
244
  ### Verifying Merkle Proofs
210
245
 
211
246
  Once you have a Merkle proof, you can verify it without needing the original backup file. This is particularly useful when you receive a proof from someone else and want to verify that the event is indeed part of the backup with the claimed Merkle root.
@@ -219,16 +254,19 @@ eventsourcingdb-merkle verify-proof proof.json
219
254
  Or with a JSON string:
220
255
 
221
256
  ```shell
222
- eventsourcingdb-merkle verify-proof '{"eventId":"42","eventHash":"...","siblingHashes":[],"merkleRoot":"..."}'
257
+ eventsourcingdb-merkle verify-proof '{"eventId":"42","eventHash":"...","siblingHashes":["..."],"merkleRoot":"esdb:merkle:v3:...","lastEventId":"99"}'
223
258
  ```
224
259
 
260
+ The command verifies proofs of every version of the tree, which it tells apart by their root, and takes the salt of a salted proof from the proof itself, so it needs neither the backup nor the key.
261
+
225
262
  If the proof is valid, the command confirms this and displays the event details:
226
263
 
227
264
  ```
228
265
  ✓ Merkle proof is valid
229
266
  Event ID: 42
230
- Event Hash: abc123...
231
- Merkle Root: d1f6713af7ba3896ff238bb51f3986b2bad53b09bda7f23370e0a8b6eb97eeab
267
+ Event Hash: esdb:hash:v2:abc123...
268
+ Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
269
+ Last Event ID: 99
232
270
  ```
233
271
 
234
272
  If the proof is invalid, the command indicates this with a non-zero exit code:
@@ -236,10 +274,13 @@ If the proof is invalid, the command indicates this with a non-zero exit code:
236
274
  ```
237
275
  ✗ Merkle proof is invalid
238
276
  Event ID: 42
239
- Event Hash: abc123...
240
- Merkle Root: d1f6713af7ba3896ff238bb51f3986b2bad53b09bda7f23370e0a8b6eb97eeab
277
+ Event Hash: esdb:hash:v2:abc123...
278
+ Merkle Root: esdb:merkle:v3:6a0e4b2f9d1c...
279
+ Last Event ID: 99
241
280
  ```
242
281
 
282
+ A valid proof shows that the event is part of the tree with the root it names, so compare that root with the one that was published. In version 3, it also shows that the event is the one with its ID, in a tree that ends at the last event ID: its path leads to the root from the leaf at that position only, and the root covers the number of events, so neither ID can be changed without the proof becoming invalid. A proof of version 1 or 2 does not bind the ID of its event, so whoever hands it on could change it, and its ID is only as trustworthy as whoever gave you the proof.
283
+
243
284
  ## Understanding the Technical Details
244
285
 
245
286
  The tool implements Merkle trees using SHA-256 hashing, following the EventSourcingDB specification for event hash calculation, which knows two versions. An event keeps the version of the hash it was written with, so the tool verifies every hash in the version it names:
@@ -249,7 +290,16 @@ The tool implements Merkle trees using SHA-256 hashing, following the EventSourc
249
290
 
250
291
  In both versions, the data is hashed as it appears in the backup or in the event given with `--event`, rather than parsed and serialized again, since that would change escape sequences such as `\u003c` for `<` and the notation of numbers. Only the whitespace between its tokens is left out, which EventSourcingDB never writes, so that an event formatted afterwards, as by a viewer or by `jq` without `-c`, still verifies.
251
292
 
252
- The Merkle tree is constructed by treating event hashes, as they are stored, prefix included, as leaf nodes. Internal nodes are created by hashing the concatenation of their child hashes. If the number of nodes at any level is odd, the last node is duplicated to create a complete binary tree. This process continues until a single root hash remains.
293
+ The Merkle trees of versions 2 and 3 follow [RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), as the tree of [eventsourcingdb-auditing](https://github.com/thenativeweb/eventsourcingdb-auditing) does:
294
+
295
+ - The data of a leaf is the hash of its event as it is stored, prefix included, as text in UTF-8. Where the leaves are salted, its salt comes first: the HMAC-SHA256 of that text under the salt key. Since nothing marks where the salt ends, the tree takes only hashes in the form of version 1 or 2, which give the data of a leaf other lengths with a salt than without one, and refuses an event with any other hash, as `verify-proof` refuses a proof of one.
296
+ - The hash of a leaf is the SHA-256 of the byte `0x00` followed by its data, and the hash of an inner node is the SHA-256 of the byte `0x01` followed by the hashes of its two children, so that neither can pass for the other.
297
+ - The tree splits its leaves at the largest power of two smaller than their number, and never duplicates a node.
298
+ - The root of version 2 is the hash of the top node in hexadecimal, after the prefix `esdb:merkle:v2:`. The root of version 3 is the SHA-256 of the byte `0x02`, the number of leaves as 8 bytes, big-endian, and the hash of the top node, in hexadecimal, after the prefix `esdb:merkle:v3:`.
299
+
300
+ A proof of version 3 is verified as [RFC 9162](https://www.rfc-editor.org/rfc/rfc9162), section 2.1.3.2, verifies an inclusion proof, with the event ID as the index of its leaf and one more than the last event ID as the size of the tree, from which the side of every sibling follows. EventSourcingDB numbers its events from 0 without gaps, so the leaf at an index is the event with that ID, and the tree of version 3 refuses a source whose IDs do not run like that. Certificate Transparency, for which RFC 9162 was written, signs the size of a tree together with its root. The root of version 3 hashes it in instead, since without it, a proof could bring a size of its own, under which its path would lead to the root from the leaf at another index.
301
+
302
+ The tree of version 1 takes the stored hashes as its leaves, and hashes the concatenation of the hashes of its two children, as text, for an inner node. Where a level has an odd number of nodes, it pairs the last one with itself.
253
303
 
254
304
  Backup files are expected to be in one of the two NDJSON formats that EventSourcingDB writes, and the tool reads every backup that a restore of EventSourcingDB takes. It checks the structure of a backup, the hashes, and the chain, but neither the values of the events nor the schemas, which a restore checks with the rules of the server:
255
305
 
@@ -0,0 +1,27 @@
1
+ import { Flags } from '@oclif/core';
2
+ import BaseCommand from '../lib/BaseCommand.js';
3
+ import { describeError } from '../lib/describeError.js';
4
+ import { writeSaltKey } from '../lib/saltKey.js';
5
+ export default class GenerateSaltKey extends BaseCommand {
6
+ static description = 'Generate a key to salt the leaves of Merkle trees with';
7
+ static examples = ['<%= config.bin %> <%= command.id %> --output=merkle.key'];
8
+ static flags = {
9
+ output: Flags.string({
10
+ description: 'File to write the salt key to; an existing file is never overwritten',
11
+ required: true,
12
+ }),
13
+ };
14
+ static args = {};
15
+ async run() {
16
+ const { flags } = await this.parse(GenerateSaltKey);
17
+ try {
18
+ await writeSaltKey(flags.output);
19
+ this.log(`✓ Salt key written to ${flags.output}`);
20
+ this.log('Keep it safe: without it, no further proof can be generated for a root calculated with it.');
21
+ }
22
+ catch (error) {
23
+ this.log(`✗ Failed to generate salt key: ${describeError(error)}`);
24
+ process.exitCode = 1;
25
+ }
26
+ }
27
+ }
@@ -5,12 +5,14 @@ import { describeError } from '../lib/describeError.js';
5
5
  import { parseEventId } from '../lib/eventId.js';
6
6
  import { generateMerkleProof } from '../lib/merkleTree.js';
7
7
  import { getSource, readEvents, sourceFlags } from '../lib/source.js';
8
+ import { getTree, getTreeFlags, getTreeHashes, readTreeOptions } from '../lib/tree.js';
8
9
  export default class GetProof extends BaseCommand {
9
10
  static description = 'Generate a Merkle proof for a specific event of an EventSourcingDB backup file or a running EventSourcingDB';
10
11
  static examples = [
11
12
  '<%= config.bin %> <%= command.id %> --backup=backup.json --event-id=42',
12
13
  '<%= config.bin %> <%= command.id %> --backup=/path/to/backup.json --event-id=0 --json',
13
14
  '<%= config.bin %> <%= command.id %> --backup=backup.json --event-id=42 --last-event-id=1234566',
15
+ '<%= config.bin %> <%= command.id %> --backup=backup.json --event-id=42 --salt-key-file=merkle.key',
14
16
  '<%= config.bin %> <%= command.id %> --esdb-url=http://localhost:3000 --esdb-api-token="$ESDB_API_TOKEN" --event-id=42',
15
17
  ];
16
18
  static flags = {
@@ -20,6 +22,7 @@ export default class GetProof extends BaseCommand {
20
22
  description: 'ID of the last event the root covers, to prove against the root of an earlier state',
21
23
  }),
22
24
  json: Flags.boolean({ description: 'Output proof as JSON' }),
25
+ ...getTreeFlags('Version of the Merkle tree: 1 to prove against roots that releases before 3.0 calculated, 2 to prove against roots that release 3.0 calculated'),
23
26
  };
24
27
  static args = {};
25
28
  async run() {
@@ -39,7 +42,13 @@ export default class GetProof extends BaseCommand {
39
42
  this.refuseCall(`--last-event-id has to be an event ID, but is '${lastEventId}'`);
40
43
  return;
41
44
  }
45
+ const tree = getTree(flags);
46
+ if ('refusal' in tree) {
47
+ this.refuseCall(tree.refusal);
48
+ return;
49
+ }
42
50
  try {
51
+ const treeOptions = await readTreeOptions(tree);
43
52
  const events = await readEvents(source, { lastEventId });
44
53
  // An event after the last one is not part of the tree, so it has no
45
54
  // proof against its root, although the source holds it.
@@ -58,8 +67,8 @@ export default class GetProof extends BaseCommand {
58
67
  process.exitCode = 1;
59
68
  return;
60
69
  }
61
- const hashes = events.map(e => e.hash);
62
- const proofData = generateMerkleProof(hashes, event.hash);
70
+ const hashes = getTreeHashes(events, treeOptions);
71
+ const proofData = generateMerkleProof(hashes, events.indexOf(event), treeOptions);
63
72
  const proof = {
64
73
  eventId: event.id,
65
74
  ...proofData,
@@ -71,11 +80,21 @@ export default class GetProof extends BaseCommand {
71
80
  this.log(`Merkle Proof for Event ${proof.eventId}`);
72
81
  this.log('');
73
82
  this.log(`Event Hash: ${proof.eventHash}`);
83
+ if (proof.salt !== undefined) {
84
+ this.log(`Salt: ${proof.salt}`);
85
+ }
74
86
  this.log(`Merkle Root: ${proof.merkleRoot}`);
87
+ if ('lastEventId' in proof) {
88
+ this.log(`Last Event ID: ${proof.lastEventId}`);
89
+ }
75
90
  this.log('');
76
91
  this.log(`Sibling Hashes (${proof.siblingHashes.length}):`);
92
+ // The siblings of a proof of version 3 have no sides, since they follow
93
+ // from the position of the event and the size of the tree.
77
94
  for (const [index, sibling] of proof.siblingHashes.entries()) {
78
- this.log(` ${index + 1}. ${sibling.hash} (${sibling.position})`);
95
+ this.log(typeof sibling === 'string'
96
+ ? ` ${index + 1}. ${sibling}`
97
+ : ` ${index + 1}. ${sibling.hash} (${sibling.position})`);
79
98
  }
80
99
  }
81
100
  catch (error) {
@@ -2,13 +2,16 @@ import { Flags } from '@oclif/core';
2
2
  import BaseCommand from '../lib/BaseCommand.js';
3
3
  import { describeError } from '../lib/describeError.js';
4
4
  import { parseEventId } from '../lib/eventId.js';
5
- import { buildMerkleRoot } from '../lib/merkleTree.js';
5
+ import { calculateMerkleRoot } from '../lib/merkleTree.js';
6
6
  import { getSource, readEvents, sourceFlags } from '../lib/source.js';
7
+ import { getTree, getTreeFlags, getTreeHashes, readTreeOptions } from '../lib/tree.js';
7
8
  export default class MerkleRoot extends BaseCommand {
8
9
  static description = 'Calculate the Merkle root for an EventSourcingDB backup file or a running EventSourcingDB';
9
10
  static examples = [
10
11
  '<%= config.bin %> <%= command.id %> --backup=backup.json',
11
12
  '<%= config.bin %> <%= command.id %> --backup=/path/to/backup.json --last-event-id=41',
13
+ '<%= config.bin %> <%= command.id %> --backup=backup.json --salt-key-file=merkle.key',
14
+ '<%= config.bin %> <%= command.id %> --backup=backup.json --tree-version=1',
12
15
  '<%= config.bin %> <%= command.id %> --esdb-url=http://localhost:3000 --esdb-api-token="$ESDB_API_TOKEN"',
13
16
  ];
14
17
  static flags = {
@@ -16,6 +19,7 @@ export default class MerkleRoot extends BaseCommand {
16
19
  'last-event-id': Flags.string({
17
20
  description: 'ID of the last event to cover, to calculate the root of an earlier state again',
18
21
  }),
22
+ ...getTreeFlags('Version of the Merkle tree: 1 for roots that releases before 3.0 calculated, 2 for roots that release 3.0 calculated'),
19
23
  };
20
24
  static args = {};
21
25
  async run() {
@@ -30,10 +34,16 @@ export default class MerkleRoot extends BaseCommand {
30
34
  this.refuseCall(`--last-event-id has to be an event ID, but is '${lastEventId}'`);
31
35
  return;
32
36
  }
37
+ const tree = getTree(flags);
38
+ if ('refusal' in tree) {
39
+ this.refuseCall(tree.refusal);
40
+ return;
41
+ }
33
42
  try {
43
+ const treeOptions = await readTreeOptions(tree);
34
44
  const events = await readEvents(source, { lastEventId });
35
- const hashes = events.map(event => event.hash);
36
- const merkleRoot = buildMerkleRoot(hashes);
45
+ const hashes = getTreeHashes(events, treeOptions);
46
+ const merkleRoot = calculateMerkleRoot(hashes, treeOptions);
37
47
  // The ID of the last event is what calculates this root again later,
38
48
  // from a source that has grown since.
39
49
  this.log(`Merkle Root: ${merkleRoot}`);
@@ -1,9 +1,10 @@
1
1
  import { Args } from '@oclif/core';
2
2
  import BaseCommand from '../lib/BaseCommand.js';
3
- import { verifyMerkleProof } from '../lib/merkleTree.js';
3
+ import { findProofStructureProblem, verifyMerkleProof } from '../lib/merkleTree.js';
4
4
  export default class VerifyProof extends BaseCommand {
5
5
  static description = 'Verify a Merkle proof';
6
6
  static examples = [
7
+ '<%= config.bin %> <%= command.id %> \'{"eventId":"42","eventHash":"...","siblingHashes":["..."],"merkleRoot":"esdb:merkle:v3:...","lastEventId":"99"}\'',
7
8
  '<%= config.bin %> <%= command.id %> \'{"eventId":"42","eventHash":"...","siblingHashes":[],"merkleRoot":"..."}\'',
8
9
  '<%= config.bin %> <%= command.id %> proof.json',
9
10
  ];
@@ -27,26 +28,26 @@ export default class VerifyProof extends BaseCommand {
27
28
  const fileContent = readFileSync(args.proof, 'utf-8');
28
29
  proof = JSON.parse(fileContent);
29
30
  }
30
- // Validate proof structure
31
- if (!proof.eventHash || !proof.merkleRoot || !proof.siblingHashes) {
31
+ const structureProblem = findProofStructureProblem(proof);
32
+ if (structureProblem !== undefined) {
32
33
  this.log('✗ Invalid proof structure');
33
- this.log('Must contain eventHash, merkleRoot, and siblingHashes');
34
+ this.log(structureProblem);
34
35
  process.exitCode = 1;
35
36
  return;
36
37
  }
37
38
  const isValid = verifyMerkleProof(proof);
38
- if (!isValid) {
39
- this.log(`✗ Merkle proof is invalid`);
40
- this.log(`Event ID: ${proof.eventId}`);
41
- this.log(`Event Hash: ${proof.eventHash}`);
42
- this.log(`Merkle Root: ${proof.merkleRoot}`);
43
- process.exitCode = 1;
44
- return;
45
- }
46
- this.log(`✓ Merkle proof is valid`);
39
+ this.log(isValid ? '✓ Merkle proof is valid' : '✗ Merkle proof is invalid');
47
40
  this.log(`Event ID: ${proof.eventId}`);
48
41
  this.log(`Event Hash: ${proof.eventHash}`);
49
42
  this.log(`Merkle Root: ${proof.merkleRoot}`);
43
+ // The root of version 3 covers the size of its tree, so the ID of the last
44
+ // event tells which state of the store it is the root of.
45
+ if ('lastEventId' in proof) {
46
+ this.log(`Last Event ID: ${proof.lastEventId}`);
47
+ }
48
+ if (!isValid) {
49
+ process.exitCode = 1;
50
+ }
50
51
  }
51
52
  catch (error) {
52
53
  this.log(`✗ Failed to verify Merkle proof: ${error instanceof Error ? error.message : String(error)}`);
package/dist/lib/hash.js CHANGED
@@ -50,6 +50,10 @@ export const assertKnownHashVersion = (storedHash) => {
50
50
  const [version = ''] = storedHash.slice(hashPrefix.length).split(':');
51
51
  throw new UnsupportedHashVersionError(version);
52
52
  };
53
+ // A stored hash of version 1 is a SHA-256 in hexadecimal, and one of version 2
54
+ // the same after its prefix, as EventSourcingDB writes them.
55
+ const knownStoredHashPattern = new RegExp(`^(${hashVersion2Prefix})?[0-9a-f]{64}$`, 'u');
56
+ export const isKnownStoredHash = (text) => knownStoredHashPattern.test(text);
53
57
  // calculateEventHash computes the hash of an event in the version that its
54
58
  // stored hash names, since an event keeps the version of the hash it was
55
59
  // written with. An event without a stored hash names no version, which is what
@@ -1,81 +1,184 @@
1
- import { hashPair } from './hash.js';
2
- export const buildMerkleRoot = (hashes) => {
3
- if (hashes.length === 0) {
4
- throw new Error('Cannot build Merkle root from empty list');
5
- }
6
- let currentLevel = [...hashes];
7
- while (currentLevel.length > 1) {
8
- const nextLevel = [];
9
- for (let i = 0; i < currentLevel.length; i += 2) {
10
- const left = currentLevel[i];
11
- const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
12
- if (left === undefined || right === undefined) {
13
- throw new Error('Unexpected undefined hash while building Merkle tree');
14
- }
15
- nextLevel.push(hashPair(left, right));
16
- }
17
- currentLevel = nextLevel;
1
+ import { isKnownStoredHash } from './hash.js';
2
+ import { isObject } from './isObject.js';
3
+ import { buildVersion1Root, generateVersion1Proof, verifyVersion1Proof, } from './merkleTreeVersion1.js';
4
+ import { buildTree, calculateRootFromAuditPath, hashLeaf, } from './merkleTreeVersion2.js';
5
+ import { buildVersion3Tree, calculateVersion3Root } from './merkleTreeVersion3.js';
6
+ import { deriveSalt } from './saltKey.js';
7
+ // A root that names the version of its tree starts with this prefix. A root
8
+ // without it is of version 1, which predates the prefix.
9
+ const merkleRootPrefix = 'esdb:merkle:';
10
+ const merkleRootVersion2Prefix = `${merkleRootPrefix}v2:`;
11
+ const merkleRootVersion3Prefix = `${merkleRootPrefix}v3:`;
12
+ // A salt and the hash of a node are SHA-256 digests, which a proof carries in
13
+ // hexadecimal. Decoding takes text that is not exactly that only as far as it
14
+ // can, so a proof that holds more than the digest would still verify.
15
+ const digestPattern = /^[0-9a-f]{64}$/iu;
16
+ export class UnsupportedMerkleTreeVersionError extends Error {
17
+ constructor(version) {
18
+ super(`Unsupported Merkle tree version: ${version}`);
19
+ this.name = 'UnsupportedMerkleTreeVersionError';
20
+ }
21
+ }
22
+ // The data of a leaf of versions 2 and 3 is the stored hash of its event, as
23
+ // text, which covers both versions of the hash alike, preceded by its salt where
24
+ // the leaves are salted.
25
+ const getLeafHash = (storedHash, salt) => hashLeaf(Buffer.concat([salt ?? Buffer.alloc(0), Buffer.from(storedHash, 'utf-8')]));
26
+ const getSalt = (saltKey, storedHash) => saltKey === undefined ? undefined : deriveSalt(saltKey, storedHash);
27
+ const getLeafHashes = (hashes, saltKey) => hashes.map(hash => getLeafHash(hash, getSalt(saltKey, hash)));
28
+ const formatVersion2Root = (root) => `${merkleRootVersion2Prefix}${root.toString('hex')}`;
29
+ const formatVersion3Root = (root) => `${merkleRootVersion3Prefix}${root.toString('hex')}`;
30
+ export const calculateMerkleRoot = (hashes, options) => {
31
+ if (options.treeVersion === 1) {
32
+ return buildVersion1Root(hashes);
18
33
  }
19
- const [root] = currentLevel;
20
- if (root === undefined) {
21
- throw new Error('Failed to compute Merkle root');
34
+ const leafHashes = getLeafHashes(hashes, options.saltKey);
35
+ if (options.treeVersion === 2) {
36
+ return formatVersion2Root(buildTree(leafHashes).root);
22
37
  }
23
- return root;
38
+ return formatVersion3Root(buildVersion3Tree(leafHashes).root);
24
39
  };
25
- export const generateMerkleProof = (hashes, targetHash) => {
26
- const targetIndex = hashes.indexOf(targetHash);
27
- if (targetIndex === -1) {
40
+ export const generateMerkleProof = (hashes, index, options) => {
41
+ const eventHash = hashes[index];
42
+ if (eventHash === undefined) {
28
43
  throw new Error('Target hash not found in the list');
29
44
  }
30
- const siblingHashes = [];
31
- let currentLevel = [...hashes];
32
- let currentIndex = targetIndex;
33
- while (currentLevel.length > 1) {
34
- const nextLevel = [];
35
- let nextIndex = 0;
36
- for (let i = 0; i < currentLevel.length; i += 2) {
37
- const left = currentLevel[i];
38
- const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
39
- if (left === undefined || right === undefined) {
40
- throw new Error('Unexpected undefined hash while generating Merkle proof');
41
- }
42
- if (i === currentIndex || i + 1 === currentIndex) {
43
- if (currentIndex % 2 === 0) {
44
- // Target is on the left
45
- if (i + 1 < currentLevel.length) {
46
- siblingHashes.push({ hash: right, position: 'right' });
47
- }
48
- }
49
- else {
50
- // Target is on the right
51
- siblingHashes.push({ hash: left, position: 'left' });
52
- }
53
- nextIndex = Math.floor(currentIndex / 2);
54
- }
55
- nextLevel.push(hashPair(left, right));
56
- }
57
- currentLevel = nextLevel;
58
- currentIndex = nextIndex;
45
+ if (options.treeVersion === 1) {
46
+ return generateVersion1Proof(hashes, eventHash);
59
47
  }
60
- const [merkleRoot] = currentLevel;
61
- if (merkleRoot === undefined) {
62
- throw new Error('Failed to compute Merkle root');
48
+ const salt = getSalt(options.saltKey, eventHash);
49
+ const leaf = {
50
+ eventHash,
51
+ ...(salt === undefined ? {} : { salt: salt.toString('hex') }),
52
+ };
53
+ const leafHashes = getLeafHashes(hashes, options.saltKey);
54
+ if (options.treeVersion === 2) {
55
+ const { root, auditPath } = buildTree(leafHashes, index);
56
+ return {
57
+ ...leaf,
58
+ siblingHashes: auditPath.map(({ hash, position }) => ({
59
+ hash: hash.toString('hex'),
60
+ position,
61
+ })),
62
+ merkleRoot: formatVersion2Root(root),
63
+ };
63
64
  }
65
+ // The leaf at an index of the tree of version 3 is the event with that ID, so
66
+ // the last event of the tree has the ID one below the number of its leaves.
67
+ const { root, auditPath } = buildVersion3Tree(leafHashes, index);
64
68
  return {
65
- eventHash: targetHash,
66
- siblingHashes,
67
- merkleRoot,
69
+ ...leaf,
70
+ siblingHashes: auditPath.map(hash => hash.toString('hex')),
71
+ merkleRoot: formatVersion3Root(root),
72
+ lastEventId: String(hashes.length - 1),
68
73
  };
69
74
  };
70
- export const verifyMerkleProof = (proof) => {
71
- let currentHash = proof.eventHash;
72
- for (const { hash, position } of proof.siblingHashes) {
73
- if (position === 'left') {
74
- currentHash = hashPair(hash, currentHash);
75
+ // getMerkleTreeVersion tells the version of the tree that a root names, the way
76
+ // the version of a hash is told: a root without the prefix is of version 1, and
77
+ // every other root with it of a version that the tool does not know.
78
+ export const getMerkleTreeVersion = (merkleRoot) => {
79
+ if (!merkleRoot.startsWith(merkleRootPrefix)) {
80
+ return 1;
81
+ }
82
+ if (merkleRoot.startsWith(merkleRootVersion2Prefix)) {
83
+ return 2;
84
+ }
85
+ if (merkleRoot.startsWith(merkleRootVersion3Prefix)) {
86
+ return 3;
87
+ }
88
+ const [version = ''] = merkleRoot.slice(merkleRootPrefix.length).split(':');
89
+ throw new UnsupportedMerkleTreeVersionError(version);
90
+ };
91
+ const isDigest = (value) => typeof value === 'string' && digestPattern.test(value);
92
+ const isSiblingWithSide = (sibling) => isObject(sibling) &&
93
+ isDigest(sibling.hash) &&
94
+ (sibling.position === 'left' || sibling.position === 'right');
95
+ // An ID in a proof of version 3 is written as EventSourcingDB writes IDs, since
96
+ // get-proof writes it rather than anyone typing it, so that the same proof has
97
+ // one notation only.
98
+ const eventIdPattern = /^(0|[1-9][0-9]*)$/u;
99
+ const isEventId = (value) => typeof value === 'string' && eventIdPattern.test(value);
100
+ // findProofStructureProblem describes what keeps a proof from being one that
101
+ // can be verified, or returns undefined if nothing does. Which fields a proof may
102
+ // or has to have depends on the version of its tree alone, so they are part of
103
+ // the structure, as is the form of every field. The form of the event hash in
104
+ // versions 2 and 3 matters for another reason: the data of a leaf is its salt
105
+ // followed by the stored hash, with nothing between them that tells where the
106
+ // salt ends, so without that form, a proof whose salt took the first bytes of
107
+ // the hash would pass for a proof of the rest of it. With it, the data of a leaf
108
+ // has other lengths with a salt than without one. Whether the fields fit one
109
+ // another, such as an event ID beyond the last one, is left to the verification,
110
+ // as RFC 9162 leaves it.
111
+ export const findProofStructureProblem = (proof) => {
112
+ if (!isObject(proof) ||
113
+ typeof proof.eventHash !== 'string' ||
114
+ proof.eventHash === '' ||
115
+ typeof proof.merkleRoot !== 'string' ||
116
+ proof.merkleRoot === '' ||
117
+ !Array.isArray(proof.siblingHashes)) {
118
+ return 'Must contain eventHash, merkleRoot, and siblingHashes';
119
+ }
120
+ const treeVersion = getMerkleTreeVersion(proof.merkleRoot);
121
+ const hasSalt = proof.salt !== undefined;
122
+ const hasLastEventId = proof.lastEventId !== undefined;
123
+ if (treeVersion === 1) {
124
+ if (hasSalt) {
125
+ return 'A proof of Merkle tree version 1 has no salt';
126
+ }
127
+ if (hasLastEventId) {
128
+ return 'A proof of Merkle tree version 1 has no last event ID';
129
+ }
130
+ return undefined;
131
+ }
132
+ if (treeVersion === 2 && hasLastEventId) {
133
+ return 'A proof of Merkle tree version 2 has no last event ID';
134
+ }
135
+ if (treeVersion === 3) {
136
+ if (proof.eventId === undefined || !hasLastEventId) {
137
+ return 'A proof of Merkle tree version 3 has to contain eventId and lastEventId';
75
138
  }
76
- else {
77
- currentHash = hashPair(currentHash, hash);
139
+ if (!isEventId(proof.eventId)) {
140
+ return 'The event ID has to be digits alone, without a sign or leading zeros';
78
141
  }
79
142
  }
80
- return currentHash === proof.merkleRoot;
143
+ if (!isKnownStoredHash(proof.eventHash)) {
144
+ return 'The event hash has to be a hash of version 1 or 2';
145
+ }
146
+ if (hasSalt && !isDigest(proof.salt)) {
147
+ return 'The salt has to be 64 hexadecimal digits';
148
+ }
149
+ if (treeVersion === 2) {
150
+ return proof.siblingHashes.every(isSiblingWithSide)
151
+ ? undefined
152
+ : 'Every sibling has to have a hash of 64 hexadecimal digits, and the position left or right';
153
+ }
154
+ if (!proof.siblingHashes.every(isDigest)) {
155
+ return 'Every sibling has to be a hash of 64 hexadecimal digits';
156
+ }
157
+ if (!isEventId(proof.lastEventId)) {
158
+ return 'The last event ID has to be digits alone, without a sign or leading zeros';
159
+ }
160
+ return undefined;
161
+ };
162
+ // verifyMerkleProof tells whether a proof leads from the hash of its event to
163
+ // its root, in the version of the tree that the root names, and in version 3,
164
+ // whether it does so from the leaf of its event in a tree that ends at its last
165
+ // event. A proof in which findProofStructureProblem finds a problem proves
166
+ // nothing, and one that passes it has the fields of its version.
167
+ export const verifyMerkleProof = (proof) => {
168
+ if (findProofStructureProblem(proof) !== undefined) {
169
+ return false;
170
+ }
171
+ const treeVersion = getMerkleTreeVersion(proof.merkleRoot);
172
+ if (treeVersion === 1) {
173
+ return verifyVersion1Proof(proof);
174
+ }
175
+ const salt = proof.salt === undefined ? undefined : Buffer.from(proof.salt, 'hex');
176
+ const leafHash = getLeafHash(proof.eventHash, salt);
177
+ if (treeVersion === 2) {
178
+ const auditPath = proof.siblingHashes.map(({ hash, position }) => ({ hash: Buffer.from(hash, 'hex'), position }));
179
+ return formatVersion2Root(calculateRootFromAuditPath(leafHash, auditPath)) === proof.merkleRoot;
180
+ }
181
+ const { eventId, lastEventId, siblingHashes } = proof;
182
+ const root = calculateVersion3Root(leafHash, BigInt(eventId), BigInt(lastEventId) + 1n, siblingHashes.map(hash => Buffer.from(hash, 'hex')));
183
+ return root !== undefined && formatVersion3Root(root) === proof.merkleRoot;
81
184
  };
@@ -0,0 +1,86 @@
1
+ import { hashPair } from './hash.js';
2
+ // The Merkle tree of version 1, which releases before 3.0 calculated. It hashes
3
+ // leaves and inner nodes alike, and pairs a node that is left without a partner
4
+ // with itself, so that a list of hashes has the same root as the same list with
5
+ // its last hash twice. It stays only to calculate the roots that were published
6
+ // with it again, and to verify their proofs.
7
+ export const buildVersion1Root = (hashes) => {
8
+ if (hashes.length === 0) {
9
+ throw new Error('Cannot build Merkle root from empty list');
10
+ }
11
+ let currentLevel = [...hashes];
12
+ while (currentLevel.length > 1) {
13
+ const nextLevel = [];
14
+ for (let i = 0; i < currentLevel.length; i += 2) {
15
+ const left = currentLevel[i];
16
+ const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
17
+ if (left === undefined || right === undefined) {
18
+ throw new Error('Unexpected undefined hash while building Merkle tree');
19
+ }
20
+ nextLevel.push(hashPair(left, right));
21
+ }
22
+ currentLevel = nextLevel;
23
+ }
24
+ const [root] = currentLevel;
25
+ if (root === undefined) {
26
+ throw new Error('Failed to compute Merkle root');
27
+ }
28
+ return root;
29
+ };
30
+ export const generateVersion1Proof = (hashes, targetHash) => {
31
+ const targetIndex = hashes.indexOf(targetHash);
32
+ if (targetIndex === -1) {
33
+ throw new Error('Target hash not found in the list');
34
+ }
35
+ const siblingHashes = [];
36
+ let currentLevel = [...hashes];
37
+ let currentIndex = targetIndex;
38
+ while (currentLevel.length > 1) {
39
+ const nextLevel = [];
40
+ let nextIndex = 0;
41
+ for (let i = 0; i < currentLevel.length; i += 2) {
42
+ const left = currentLevel[i];
43
+ const right = i + 1 < currentLevel.length ? currentLevel[i + 1] : currentLevel[i];
44
+ if (left === undefined || right === undefined) {
45
+ throw new Error('Unexpected undefined hash while generating Merkle proof');
46
+ }
47
+ if (i === currentIndex || i + 1 === currentIndex) {
48
+ if (currentIndex % 2 === 0) {
49
+ // Target is on the left
50
+ if (i + 1 < currentLevel.length) {
51
+ siblingHashes.push({ hash: right, position: 'right' });
52
+ }
53
+ }
54
+ else {
55
+ // Target is on the right
56
+ siblingHashes.push({ hash: left, position: 'left' });
57
+ }
58
+ nextIndex = Math.floor(currentIndex / 2);
59
+ }
60
+ nextLevel.push(hashPair(left, right));
61
+ }
62
+ currentLevel = nextLevel;
63
+ currentIndex = nextIndex;
64
+ }
65
+ const [merkleRoot] = currentLevel;
66
+ if (merkleRoot === undefined) {
67
+ throw new Error('Failed to compute Merkle root');
68
+ }
69
+ return {
70
+ eventHash: targetHash,
71
+ siblingHashes,
72
+ merkleRoot,
73
+ };
74
+ };
75
+ export const verifyVersion1Proof = (proof) => {
76
+ let currentHash = proof.eventHash;
77
+ for (const { hash, position } of proof.siblingHashes) {
78
+ if (position === 'left') {
79
+ currentHash = hashPair(hash, currentHash);
80
+ }
81
+ else {
82
+ currentHash = hashPair(currentHash, hash);
83
+ }
84
+ }
85
+ return currentHash === proof.merkleRoot;
86
+ };
@@ -0,0 +1,51 @@
1
+ import { createHash } from 'node:crypto';
2
+ // RFC 6962 puts a byte of its own in front of what a leaf and an inner node
3
+ // hash, so that neither can pass for the other: without them, the hash of an
4
+ // inner node would also be the hash of a leaf whose data are the hashes of its
5
+ // children.
6
+ const leafPrefix = Buffer.from([0x00]);
7
+ const nodePrefix = Buffer.from([0x01]);
8
+ export const hashLeaf = (data) => createHash('sha256').update(leafPrefix).update(data).digest();
9
+ export const hashNode = (left, right) => createHash('sha256').update(nodePrefix).update(left).update(right).digest();
10
+ // buildTree calculates the root of the tree over the hashes of its leaves, as
11
+ // RFC 6962 defines it, and the audit path of the leaf at the given index. RFC
12
+ // 6962 defines the tree by splitting the leaves at the largest power of two
13
+ // smaller than their number. Pairing the nodes level by level and moving a node
14
+ // that is left without a partner up unchanged builds the same tree, without
15
+ // copying the leaves for every subtree. The node is moved rather than paired
16
+ // with itself, since that would give a list of leaves the same root as the same
17
+ // list with its last leaf twice.
18
+ export const buildTree = (leafHashes, index) => {
19
+ if (leafHashes.length === 0) {
20
+ throw new Error('Cannot build Merkle root from empty list');
21
+ }
22
+ const auditPath = [];
23
+ let level = leafHashes;
24
+ let currentIndex = index;
25
+ while (level.length > 1) {
26
+ if (currentIndex !== undefined) {
27
+ const isLeftChild = currentIndex % 2 === 0;
28
+ const sibling = level[isLeftChild ? currentIndex + 1 : currentIndex - 1];
29
+ if (sibling !== undefined) {
30
+ auditPath.push({ hash: sibling, position: isLeftChild ? 'right' : 'left' });
31
+ }
32
+ currentIndex = Math.floor(currentIndex / 2);
33
+ }
34
+ const nextLevel = [];
35
+ for (let position = 0; position < level.length; position += 2) {
36
+ const left = level[position];
37
+ const right = level[position + 1];
38
+ if (left === undefined) {
39
+ throw new Error('Unexpected undefined hash while building Merkle tree');
40
+ }
41
+ nextLevel.push(right === undefined ? left : hashNode(left, right));
42
+ }
43
+ level = nextLevel;
44
+ }
45
+ const [root] = level;
46
+ if (root === undefined) {
47
+ throw new Error('Failed to compute Merkle root');
48
+ }
49
+ return { root, auditPath };
50
+ };
51
+ export const calculateRootFromAuditPath = (leafHash, auditPath) => auditPath.reduce((hash, sibling) => sibling.position === 'left' ? hashNode(sibling.hash, hash) : hashNode(hash, sibling.hash), leafHash);
@@ -0,0 +1,65 @@
1
+ import { createHash } from 'node:crypto';
2
+ import { buildTree, hashNode } from './merkleTreeVersion2.js';
3
+ // The tree of version 3 is the tree of version 2, but its root covers the size
4
+ // of the tree as well. A proof of version 3 locates its leaf by the index and
5
+ // the size of the tree, as RFC 9162 does, and the index of an event is its ID.
6
+ // RFC 9162 binds the index only where the size is known, though: a path that
7
+ // leads to the root from one leaf of a tree of one size can lead to it from a
8
+ // leaf at another index of a tree of another size. Certificate Transparency
9
+ // signs the size along with the root, but a root that the tool calculates is
10
+ // published as it is, so the root hashes the size in, behind a byte of its own,
11
+ // as RFC 6962 puts one in front of a leaf and of an inner node.
12
+ const rootPrefix = Buffer.from([0x02]);
13
+ // RFC 6962 encodes the size of a tree in 8 bytes, as it does in a tree head, so
14
+ // a tree has at most 2^64 - 1 leaves.
15
+ const treeSizeLength = 8;
16
+ const maximumTreeSize = 2n ** 64n - 1n;
17
+ const hashRoot = (treeSize, root) => {
18
+ const size = Buffer.alloc(treeSizeLength);
19
+ size.writeBigUInt64BE(treeSize);
20
+ return createHash('sha256').update(rootPrefix).update(size).update(root).digest();
21
+ };
22
+ export const buildVersion3Tree = (leafHashes, index) => {
23
+ const { root, auditPath } = buildTree(leafHashes, index);
24
+ return {
25
+ root: hashRoot(BigInt(leafHashes.length), root),
26
+ auditPath: auditPath.map(({ hash }) => hash),
27
+ };
28
+ };
29
+ // calculateVersion3Root calculates the root from the audit path of the leaf at
30
+ // the given index of a tree of the given size, as section 2.1.3.2 of RFC 9162
31
+ // verifies an inclusion proof, or returns undefined where the path does not fit
32
+ // that leaf in that tree. The side of each sibling follows from the index of the
33
+ // node on the path and the last index of its level: a node with an odd index is
34
+ // a right child, and so is the last node of a level, which has no partner on its
35
+ // right and is moved up unchanged until it is one. Both indexes halve from one
36
+ // level to the next, and the path fits where it ends at the level of the root.
37
+ export const calculateVersion3Root = (leafHash, leafIndex, treeSize, auditPath) => {
38
+ if (leafIndex >= treeSize || treeSize > maximumTreeSize) {
39
+ return undefined;
40
+ }
41
+ let index = leafIndex;
42
+ let lastIndex = treeSize - 1n;
43
+ let hash = leafHash;
44
+ for (const sibling of auditPath) {
45
+ if (lastIndex === 0n) {
46
+ return undefined;
47
+ }
48
+ if (index % 2n === 1n || index === lastIndex) {
49
+ hash = hashNode(sibling, hash);
50
+ while (index % 2n === 0n && index !== 0n) {
51
+ index /= 2n;
52
+ lastIndex /= 2n;
53
+ }
54
+ }
55
+ else {
56
+ hash = hashNode(hash, sibling);
57
+ }
58
+ index /= 2n;
59
+ lastIndex /= 2n;
60
+ }
61
+ if (lastIndex !== 0n) {
62
+ return undefined;
63
+ }
64
+ return hashRoot(treeSize, hash);
65
+ };
@@ -0,0 +1,65 @@
1
+ import { createHmac, randomBytes } from 'node:crypto';
2
+ import { access, open, readFile, unlink } from 'node:fs/promises';
3
+ // A salt key is 32 random bytes: RFC 2104 discourages keys for HMAC that are
4
+ // shorter than the output of the hash, and longer ones add nothing. It is kept
5
+ // as text of 64 hexadecimal digits, so that it can be stored wherever a password
6
+ // can, and whitespace around them is left out, so that a final newline, which
7
+ // one editor adds and another does not, does not make the same key another one.
8
+ const saltKeyLength = 32;
9
+ const saltKeyPattern = /^[0-9a-f]{64}$/iu;
10
+ export const readSaltKey = async (filePath) => {
11
+ try {
12
+ await access(filePath);
13
+ }
14
+ catch {
15
+ throw new Error(`File not found: ${filePath}`);
16
+ }
17
+ const digits = (await readFile(filePath, 'utf-8')).trim();
18
+ // The message does not show what the file holds, since that may be a key
19
+ // that was only written down wrong.
20
+ if (!saltKeyPattern.test(digits)) {
21
+ throw new Error(`the salt key in ${filePath} has to be 64 hexadecimal digits`);
22
+ }
23
+ return Buffer.from(digits, 'hex');
24
+ };
25
+ // writeSaltKey never overwrites a file, since a root whose leaves were salted
26
+ // with a key gets no further proof once the key is gone. Nobody may write the
27
+ // file, so that it is not replaced by accident either, as by a command that
28
+ // redirects its output into it, and on Linux and macOS, only its owner may read
29
+ // it. Windows knows only whether a file is read-only, so there, who may read it
30
+ // follows from the permissions of its folder.
31
+ export const writeSaltKey = async (filePath) => {
32
+ const digits = randomBytes(saltKeyLength).toString('hex');
33
+ let file;
34
+ try {
35
+ file = await open(filePath, 'wx', 0o400);
36
+ }
37
+ catch (error) {
38
+ if (error.code === 'EEXIST') {
39
+ throw new Error(`${filePath} already exists`);
40
+ }
41
+ throw error;
42
+ }
43
+ // A root may be calculated and published with the key right after it was
44
+ // written, so the content of the file reaches the disk before the key counts as
45
+ // written. Its entry in the folder is not flushed on its own, so whether a key
46
+ // written right before a power failure is found afterwards is left to the file
47
+ // system. A file that could not be written completely is removed again, so that
48
+ // it neither passes for a key nor keeps the next attempt from writing one. It
49
+ // is closed first, which Windows needs to remove it, but a failure to close it
50
+ // keeps it neither from being removed nor from reporting why writing it failed.
51
+ try {
52
+ await file.writeFile(`${digits}\n`);
53
+ await file.sync();
54
+ }
55
+ catch (error) {
56
+ await file.close().catch(() => undefined);
57
+ await unlink(filePath);
58
+ throw error;
59
+ }
60
+ await file.close();
61
+ };
62
+ // deriveSalt derives the salt of a leaf from the key and the stored hash of its
63
+ // event, so that no salt has to be kept: whoever holds the key gets the same
64
+ // salts again, and whoever does not cannot tell them from random ones.
65
+ export const deriveSalt = (saltKey, storedHash) => createHmac('sha256', saltKey).update(storedHash, 'utf-8').digest();
@@ -0,0 +1,63 @@
1
+ import { Flags } from '@oclif/core';
2
+ import { isKnownStoredHash } from './hash.js';
3
+ import { readSaltKey } from './saltKey.js';
4
+ // The flags that name the tree a command calculates: its version, and the file
5
+ // with the key that salts its leaves. Version 3 is the default, and versions 1
6
+ // and 2 stay for the roots that the releases before 4.0 calculated. Only the
7
+ // commands that calculate a tree take them, since a proof names its version and
8
+ // carries its salt.
9
+ export const getTreeFlags = (treeVersionDescription) => ({
10
+ 'salt-key-file': Flags.string({
11
+ description: 'File with the salt key to salt the leaves of the tree with',
12
+ }),
13
+ 'tree-version': Flags.string({
14
+ description: treeVersionDescription,
15
+ options: ['1', '2', '3'],
16
+ default: '3',
17
+ }),
18
+ });
19
+ // getTree returns the tree that the flags name, or why they name none: the tree
20
+ // of version 1 has no salts, so a salt key file for it would be ignored, which
21
+ // would leave a root unsalted that was meant to be salted.
22
+ export const getTree = (flags) => {
23
+ const saltKeyFile = flags['salt-key-file'];
24
+ if (flags['tree-version'] === '1') {
25
+ if (saltKeyFile !== undefined) {
26
+ return { refusal: '--salt-key-file and --tree-version=1 cannot be used together' };
27
+ }
28
+ return { treeVersion: 1 };
29
+ }
30
+ return { treeVersion: flags['tree-version'] === '2' ? 2 : 3, saltKeyFile };
31
+ };
32
+ // readTreeOptions reads the salt key of a tree, if it has one. A command reads it
33
+ // before its source, so that a key file that cannot be read stops it before it
34
+ // reads a large backup or database for nothing.
35
+ export const readTreeOptions = async (tree) => {
36
+ if (tree.treeVersion === 1 || tree.saltKeyFile === undefined) {
37
+ return { treeVersion: tree.treeVersion };
38
+ }
39
+ return { treeVersion: tree.treeVersion, saltKey: await readSaltKey(tree.saltKeyFile) };
40
+ };
41
+ // getTreeHashes returns the stored hashes of the events, as the leaves of the
42
+ // tree, or throws where the tree could not prove each of them. The trees of
43
+ // versions 2 and 3 take only hashes of a form they know, since only they keep
44
+ // the data of a leaf unambiguous, which is why verify-proof refuses any other.
45
+ // The tree of version 3 also takes the events only where their IDs are their
46
+ // positions, as EventSourcingDB numbers them, in the notation in which it
47
+ // writes them, since a proof of version 3 names the position of its leaf by the
48
+ // ID of its event. Otherwise, the root would cover leaves that no proof could
49
+ // prove. The tree of version 1 takes any events, as it always did, and so does
50
+ // the tree of version 2 as far as their IDs are concerned, as release 3.0 did.
51
+ export const getTreeHashes = (events, treeOptions) => {
52
+ if (treeOptions.treeVersion !== 1) {
53
+ for (const [position, event] of events.entries()) {
54
+ if (treeOptions.treeVersion === 3 && event.id !== String(position)) {
55
+ throw new Error(`event ${event.id} is at position ${position} of the source, but the IDs of the events have to run from 0 without gaps`);
56
+ }
57
+ if (!isKnownStoredHash(event.hash)) {
58
+ throw new Error(`event ${event.id} has no hash of version 1 or 2`);
59
+ }
60
+ }
61
+ }
62
+ return events.map(({ hash }) => hash);
63
+ };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "eventsourcingdb-merkle",
3
- "version": "2.1.0",
3
+ "version": "4.0.0",
4
4
  "description": "Merkle tree utilities for EventSourcingDB backups and databases.",
5
5
  "author": {
6
6
  "name": "the native web GmbH",