eventsourcingdb-merkle 1.1.13 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -5
- package/dist/commands/validate-event-hash.js +30 -16
- package/dist/lib/chainValidator.js +5 -3
- package/dist/lib/eventText.js +43 -0
- package/dist/lib/hash.js +54 -6
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -45,6 +45,14 @@ Event 42: Hash mismatch: expected abc..., got def...
|
|
|
45
45
|
Event 43: Predecessor hash mismatch: expected def..., got ghi...
|
|
46
46
|
```
|
|
47
47
|
|
|
48
|
+
An event whose hash names a version of the hash that the tool does not know is listed as such, rather than as a mismatch:
|
|
49
|
+
|
|
50
|
+
```
|
|
51
|
+
Event 44: Unsupported hash version: v3
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Such an event cannot be verified, which is not the same as being tampered with, and a newer version of the tool may know its version.
|
|
55
|
+
|
|
48
56
|
*Note that the command will exit with a non-zero exit code if the chain is invalid, which makes it suitable for use in CI/CD pipelines.*
|
|
49
57
|
|
|
50
58
|
### Calculating the Merkle Root
|
|
@@ -93,13 +101,13 @@ Calculated hash: def456...
|
|
|
93
101
|
|
|
94
102
|
#### Validating an Event from JSON
|
|
95
103
|
|
|
96
|
-
You can also validate an event provided as a JSON string. This is useful when you receive an event from an external source and want to verify its hash without having access to the backup file:
|
|
104
|
+
You can also validate an event provided as a JSON string. This is useful when you receive an event from an external source and want to verify its hash without having access to the backup file. The command takes a line of a backup, a line of `read-events`, or an event on its own, with its hash:
|
|
97
105
|
|
|
98
106
|
```shell
|
|
99
|
-
eventsourcingdb-merkle validate-event-hash --event='{"specversion":"1.0","id":"0"
|
|
107
|
+
eventsourcingdb-merkle validate-event-hash --event='{"specversion":"1.0","id":"0",...,"hash":"esdb:hash:v2:..."}'
|
|
100
108
|
```
|
|
101
109
|
|
|
102
|
-
If you provide a CloudEvent without a stored hash, the command simply calculates and displays the hash:
|
|
110
|
+
If you provide a CloudEvent without a stored hash, the command simply calculates and displays the hash of version 1, since nothing tells which version is meant:
|
|
103
111
|
|
|
104
112
|
```
|
|
105
113
|
Calculated hash: abc123...
|
|
@@ -195,9 +203,14 @@ Merkle Root: d1f6713af7ba3896ff238bb51f3986b2bad53b09bda7f23370e0a8b6eb97eeab
|
|
|
195
203
|
|
|
196
204
|
## Understanding the Technical Details
|
|
197
205
|
|
|
198
|
-
The tool implements Merkle trees using SHA-256 hashing, following the EventSourcingDB specification for event hash calculation
|
|
206
|
+
The tool implements Merkle trees using SHA-256 hashing, following the EventSourcingDB specification for event hash calculation, which knows two versions. An event keeps the version of the hash it was written with, so the tool verifies every hash in the version it names:
|
|
207
|
+
|
|
208
|
+
- A hash of version 2, which EventSourcingDB writes from release 1.3 on, starts with `esdb:hash:v2:`. It is the SHA-256 of that prefix and of the fields `specversion`, `id`, `predecessorhash`, `time`, `source`, `subject`, `type`, `datacontenttype`, `traceparent`, `tracestate`, and `data`, each on a line of its own, with `-` for a field that the event does not have.
|
|
209
|
+
- A hash without a prefix is of version 1. It is computed by first creating a metadata string from the CloudEvents fields, then hashing the metadata and the data separately, and finally hashing the concatenation of both hashes. It covers neither `traceparent` nor `tracestate`.
|
|
210
|
+
|
|
211
|
+
In both versions, the data is hashed as it appears in the backup or in the event given with `--event`, rather than parsed and serialized again, since that would change escape sequences such as `\u003c` for `<` and the notation of numbers. Only the whitespace between its tokens is left out, which EventSourcingDB never writes, so that an event formatted afterwards, as by a viewer or by `jq` without `-c`, still verifies.
|
|
199
212
|
|
|
200
|
-
The Merkle tree is constructed by treating event hashes as leaf nodes. Internal nodes are created by hashing the concatenation of their child hashes. If the number of nodes at any level is odd, the last node is duplicated to create a complete binary tree. This process continues until a single root hash remains.
|
|
213
|
+
The Merkle tree is constructed by treating event hashes, as they are stored, prefix included, as leaf nodes. Internal nodes are created by hashing the concatenation of their child hashes. If the number of nodes at any level is odd, the last node is duplicated to create a complete binary tree. This process continues until a single root hash remains.
|
|
201
214
|
|
|
202
215
|
Backup files are expected to be in the NDJSON format that EventSourcingDB writes on `/api/v1/backup`, where each line contains a JSON object with a `type` and a `payload` field. A line of the type `event` holds a CloudEvents-compliant event and its hash. A line of the type `schema` holds the schema of an event type, which the tool skips, since neither the hashes nor the chain cover it. A backup that holds a line of the type `error` was cut short, and the tool refuses it, as it refuses a line of any other type or one that is not valid JSON, naming the line.
|
|
203
216
|
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
import { Flags } from '@oclif/core';
|
|
2
2
|
import BaseCommand from '../lib/BaseCommand.js';
|
|
3
3
|
import { findEventById, readBackupFile } from '../lib/backupReader.js';
|
|
4
|
-
import {
|
|
5
|
-
import {
|
|
4
|
+
import { readEventText } from '../lib/eventText.js';
|
|
5
|
+
import { calculateEventHash, UnsupportedHashVersionError } from '../lib/hash.js';
|
|
6
6
|
export default class ValidateEventHash extends BaseCommand {
|
|
7
7
|
static description = 'Validate the hash of a specific event (with or without backup file)';
|
|
8
8
|
static examples = [
|
|
@@ -39,27 +39,26 @@ export default class ValidateEventHash extends BaseCommand {
|
|
|
39
39
|
const filePath = flags.file;
|
|
40
40
|
const eventId = flags['event-id'];
|
|
41
41
|
try {
|
|
42
|
-
let entry;
|
|
43
42
|
let event;
|
|
44
43
|
let storedHash;
|
|
45
44
|
let originalDataString;
|
|
46
45
|
if (flags.event) {
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
storedHash = entry.payload.hash;
|
|
54
|
-
originalDataString = getCompactValue(flags.event, ['payload', 'event', 'data']);
|
|
46
|
+
const eventFromText = readEventText(flags.event);
|
|
47
|
+
if (eventFromText === undefined) {
|
|
48
|
+
this.log('✗ --event is neither an event nor a line of a backup or of read-events');
|
|
49
|
+
this.log('See more help with --help');
|
|
50
|
+
process.exitCode = 2;
|
|
51
|
+
return;
|
|
55
52
|
}
|
|
56
|
-
|
|
57
|
-
event = parsed;
|
|
53
|
+
if (eventFromText.storedHash === undefined) {
|
|
58
54
|
// No stored hash to compare against
|
|
59
|
-
const calculatedHash = calculateEventHash(event,
|
|
55
|
+
const calculatedHash = calculateEventHash(eventFromText.event, eventFromText.originalDataString);
|
|
60
56
|
this.log(`Calculated hash: ${calculatedHash}`);
|
|
61
57
|
return;
|
|
62
58
|
}
|
|
59
|
+
event = eventFromText.event;
|
|
60
|
+
storedHash = eventFromText.storedHash;
|
|
61
|
+
originalDataString = eventFromText.originalDataString;
|
|
63
62
|
}
|
|
64
63
|
else {
|
|
65
64
|
// Read from file
|
|
@@ -69,7 +68,7 @@ export default class ValidateEventHash extends BaseCommand {
|
|
|
69
68
|
return;
|
|
70
69
|
}
|
|
71
70
|
const entries = await readBackupFile(filePath);
|
|
72
|
-
entry = findEventById(entries, eventId);
|
|
71
|
+
const entry = findEventById(entries, eventId);
|
|
73
72
|
if (!entry) {
|
|
74
73
|
this.log(`✗ Event with ID ${eventId} not found`);
|
|
75
74
|
process.exitCode = 1;
|
|
@@ -79,7 +78,22 @@ export default class ValidateEventHash extends BaseCommand {
|
|
|
79
78
|
storedHash = entry.payload.hash;
|
|
80
79
|
originalDataString = entry.payload.originalDataString;
|
|
81
80
|
}
|
|
82
|
-
|
|
81
|
+
// A hash of a version the tool does not know is no mismatch, so it is
|
|
82
|
+
// reported apart from one, with the hash that names the version.
|
|
83
|
+
let calculatedHash;
|
|
84
|
+
try {
|
|
85
|
+
calculatedHash = calculateEventHash(event, originalDataString, storedHash);
|
|
86
|
+
}
|
|
87
|
+
catch (error) {
|
|
88
|
+
if (!(error instanceof UnsupportedHashVersionError)) {
|
|
89
|
+
throw error;
|
|
90
|
+
}
|
|
91
|
+
this.log(`✗ ${error.message}`);
|
|
92
|
+
this.log(`Event ID: ${event.id}`);
|
|
93
|
+
this.log(`Stored hash: ${storedHash}`);
|
|
94
|
+
process.exitCode = 1;
|
|
95
|
+
return;
|
|
96
|
+
}
|
|
83
97
|
if (calculatedHash !== storedHash) {
|
|
84
98
|
this.log(`✗ Hash mismatch`);
|
|
85
99
|
this.log(`Event ID: ${event.id}`);
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { calculateEventHash } from './hash.js';
|
|
1
|
+
import { calculateEventHash, UnsupportedHashVersionError } from './hash.js';
|
|
2
2
|
export const validateChain = (entries) => {
|
|
3
3
|
const errors = [];
|
|
4
4
|
for (const [i, entry] of entries.entries()) {
|
|
@@ -6,7 +6,7 @@ export const validateChain = (entries) => {
|
|
|
6
6
|
const storedHash = entry.payload.hash;
|
|
7
7
|
// Validate hash
|
|
8
8
|
try {
|
|
9
|
-
const calculatedHash = calculateEventHash(event, entry.payload.originalDataString);
|
|
9
|
+
const calculatedHash = calculateEventHash(event, entry.payload.originalDataString, storedHash);
|
|
10
10
|
if (calculatedHash !== storedHash) {
|
|
11
11
|
errors.push({
|
|
12
12
|
eventId: event.id,
|
|
@@ -17,7 +17,9 @@ export const validateChain = (entries) => {
|
|
|
17
17
|
catch (error) {
|
|
18
18
|
errors.push({
|
|
19
19
|
eventId: event.id,
|
|
20
|
-
message:
|
|
20
|
+
message: error instanceof UnsupportedHashVersionError
|
|
21
|
+
? error.message
|
|
22
|
+
: `Failed to calculate hash: ${error instanceof Error ? error.message : String(error)}`,
|
|
21
23
|
});
|
|
22
24
|
}
|
|
23
25
|
// Validate predecessor hash
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
import { getCompactValue } from './rawJson.js';
|
|
2
|
+
const isObject = (value) => {
|
|
3
|
+
return typeof value === 'object' && value !== null && !Array.isArray(value);
|
|
4
|
+
};
|
|
5
|
+
const isEvent = (value) => {
|
|
6
|
+
return isObject(value) && typeof value.specversion === 'string';
|
|
7
|
+
};
|
|
8
|
+
// readEventText reads an event from text in every form one can copy it from:
|
|
9
|
+
// a line of a backup, which holds the event and its hash side by side, a line
|
|
10
|
+
// of read-events, which holds the event with its hash, or the event alone,
|
|
11
|
+
// with its hash or without. The data is taken from the text
|
|
12
|
+
// rather than from what JSON.parse makes of it, since the hash covers it as
|
|
13
|
+
// EventSourcingDB wrote it. Text in any other form gives undefined, a line
|
|
14
|
+
// whose payload is neither of the two included, since a line carries no spec
|
|
15
|
+
// version of its own and so is never taken for an event alone.
|
|
16
|
+
export const readEventText = (text) => {
|
|
17
|
+
const parsed = JSON.parse(text);
|
|
18
|
+
if (isObject(parsed) && parsed.type === 'event' && isObject(parsed.payload)) {
|
|
19
|
+
const { payload } = parsed;
|
|
20
|
+
if (isEvent(payload.event) && typeof payload.hash === 'string') {
|
|
21
|
+
return {
|
|
22
|
+
event: payload.event,
|
|
23
|
+
storedHash: payload.hash,
|
|
24
|
+
originalDataString: getCompactValue(text, ['payload', 'event', 'data']),
|
|
25
|
+
};
|
|
26
|
+
}
|
|
27
|
+
if (isEvent(payload) && typeof payload.hash === 'string') {
|
|
28
|
+
return {
|
|
29
|
+
event: payload,
|
|
30
|
+
storedHash: payload.hash,
|
|
31
|
+
originalDataString: getCompactValue(text, ['payload', 'data']),
|
|
32
|
+
};
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
if (isEvent(parsed)) {
|
|
36
|
+
return {
|
|
37
|
+
event: parsed,
|
|
38
|
+
storedHash: typeof parsed.hash === 'string' ? parsed.hash : undefined,
|
|
39
|
+
originalDataString: getCompactValue(text, ['data']),
|
|
40
|
+
};
|
|
41
|
+
}
|
|
42
|
+
return undefined;
|
|
43
|
+
};
|
package/dist/lib/hash.js
CHANGED
|
@@ -1,12 +1,60 @@
|
|
|
1
1
|
import crypto from 'node:crypto';
|
|
2
|
-
|
|
2
|
+
// A hash that names its version starts with this prefix. A hash without it is
|
|
3
|
+
// of version 1, which predates the prefix.
|
|
4
|
+
const hashPrefix = 'esdb:hash:';
|
|
5
|
+
const hashVersion2Prefix = `${hashPrefix}v2:`;
|
|
6
|
+
// The input of a hash of version 2 writes a field that the event does not have
|
|
7
|
+
// as this, which no value of such a field can be.
|
|
8
|
+
const missingField = '-';
|
|
9
|
+
export class UnsupportedHashVersionError extends Error {
|
|
10
|
+
constructor(version) {
|
|
11
|
+
super(`Unsupported hash version: ${version}`);
|
|
12
|
+
this.name = 'UnsupportedHashVersionError';
|
|
13
|
+
}
|
|
14
|
+
}
|
|
15
|
+
const getSha256 = (input) => {
|
|
16
|
+
return crypto.createHash('sha256').update(input).digest('hex');
|
|
17
|
+
};
|
|
18
|
+
// The hash of version 1 covers neither traceparent nor tracestate.
|
|
19
|
+
const calculateEventHashOfVersion1 = (event, dataString) => {
|
|
3
20
|
const metadata = `${event.specversion}|${event.id}|${event.predecessorhash}|${event.time}|${event.source}|${event.subject}|${event.type}|${event.datacontenttype}`;
|
|
4
|
-
|
|
5
|
-
|
|
21
|
+
return getSha256(`${getSha256(metadata)}${getSha256(dataString)}`);
|
|
22
|
+
};
|
|
23
|
+
// The hash of version 2 covers the prefix and every field, each on a line of its
|
|
24
|
+
// own, without a final line break, as EventSourcingDB computes it.
|
|
25
|
+
const calculateEventHashOfVersion2 = (event, dataString) => {
|
|
26
|
+
const input = [
|
|
27
|
+
hashVersion2Prefix,
|
|
28
|
+
event.specversion,
|
|
29
|
+
event.id,
|
|
30
|
+
event.predecessorhash,
|
|
31
|
+
event.time,
|
|
32
|
+
event.source,
|
|
33
|
+
event.subject,
|
|
34
|
+
event.type,
|
|
35
|
+
event.datacontenttype,
|
|
36
|
+
event.traceparent ?? missingField,
|
|
37
|
+
event.tracestate ?? missingField,
|
|
38
|
+
dataString,
|
|
39
|
+
].join('\n');
|
|
40
|
+
return `${hashVersion2Prefix}${getSha256(input)}`;
|
|
41
|
+
};
|
|
42
|
+
// calculateEventHash computes the hash of an event in the version that its
|
|
43
|
+
// stored hash names, since an event keeps the version of the hash it was
|
|
44
|
+
// written with, or in version 1 without a stored hash to go by. A stored hash
|
|
45
|
+
// whose prefix names a version that the tool does not know cannot be verified,
|
|
46
|
+
// which is not the same as being tampered with, so it is refused rather than
|
|
47
|
+
// computed in another version that could only fail to match.
|
|
48
|
+
export const calculateEventHash = (event, originalDataString, storedHash) => {
|
|
6
49
|
const dataString = originalDataString ?? JSON.stringify(event.data);
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
50
|
+
if (storedHash === undefined || !storedHash.startsWith(hashPrefix)) {
|
|
51
|
+
return calculateEventHashOfVersion1(event, dataString);
|
|
52
|
+
}
|
|
53
|
+
if (storedHash.startsWith(hashVersion2Prefix)) {
|
|
54
|
+
return calculateEventHashOfVersion2(event, dataString);
|
|
55
|
+
}
|
|
56
|
+
const [version = ''] = storedHash.slice(hashPrefix.length).split(':');
|
|
57
|
+
throw new UnsupportedHashVersionError(version);
|
|
10
58
|
};
|
|
11
59
|
export const hashPair = (left, right) => {
|
|
12
60
|
return crypto.createHash('sha256').update(`${left}${right}`).digest('hex');
|