hypercore 10.0.0-alpha.4 → 10.0.0-alpha.40
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +58 -3
- package/index.js +504 -166
- package/lib/bitfield.js +9 -5
- package/lib/block-encryption.js +68 -0
- package/lib/block-store.js +3 -1
- package/lib/caps.js +34 -0
- package/lib/core.js +84 -27
- package/lib/errors.js +42 -0
- package/lib/merkle-tree.js +186 -113
- package/lib/messages.js +249 -168
- package/lib/oplog.js +4 -3
- package/lib/replicator.js +1288 -548
- package/lib/streams.js +56 -0
- package/package.json +18 -9
- package/.github/workflows/test-node.yml +0 -23
- package/CHANGELOG.md +0 -37
- package/UPGRADE.md +0 -9
- package/examples/announce.js +0 -19
- package/examples/basic.js +0 -10
- package/examples/http.js +0 -123
- package/examples/lookup.js +0 -20
- package/lib/extensions.js +0 -76
- package/lib/protocol.js +0 -524
- package/lib/random-iterator.js +0 -46
- package/test/basic.js +0 -78
- package/test/bitfield.js +0 -71
- package/test/core.js +0 -290
- package/test/encodings.js +0 -18
- package/test/extension.js +0 -71
- package/test/helpers/index.js +0 -23
- package/test/merkle-tree.js +0 -518
- package/test/mutex.js +0 -137
- package/test/oplog.js +0 -399
- package/test/preload.js +0 -72
- package/test/replicate.js +0 -333
- package/test/sessions.js +0 -173
- package/test/user-data.js +0 -47
package/README.md
CHANGED
|
@@ -63,12 +63,16 @@ Note that `tree`, `data`, and `bitfield` are normally heavily sparse files.
|
|
|
63
63
|
createIfMissing: true, // create a new Hypercore key pair if none was present in storage
|
|
64
64
|
overwrite: false, // overwrite any old Hypercore that might already exist
|
|
65
65
|
valueEncoding: 'json' | 'utf-8' | 'binary', // defaults to binary
|
|
66
|
-
|
|
66
|
+
encodeBatch: batch => { ... }, // optionally apply an encoding to complete batches
|
|
67
|
+
keyPair: kp, // optionally pass the public key and secret key as a key pair
|
|
68
|
+
encryptionKey: k // optionally pass an encryption key to enable block encryption
|
|
67
69
|
}
|
|
68
70
|
```
|
|
69
71
|
|
|
70
72
|
You can also set valueEncoding to any [abstract-encoding](https://github.com/mafintosh/abstract-encoding) or [compact-encoding](https://github.com/compact-encoding) instance.
|
|
71
73
|
|
|
74
|
+
valueEncodings will be applied to individually blocks, even if you append batches. If you want to control encoding at the batch-level, you can use the `encodeBatch` option, which is a function that takes a batch and returns a binary-encoded batch. If you provide a custom valueEncoding, it will not be applied prior to `encodeBatch`.
|
|
75
|
+
|
|
72
76
|
#### `const seq = await core.append(block)`
|
|
73
77
|
|
|
74
78
|
Append a block of data (or an array of blocks) to the core.
|
|
@@ -83,7 +87,7 @@ Options include
|
|
|
83
87
|
|
|
84
88
|
``` js
|
|
85
89
|
{
|
|
86
|
-
wait: true, // wait for
|
|
90
|
+
wait: true, // wait for block to be downloaded
|
|
87
91
|
onwait: () => {}, // hook that is called if the get is waiting for download
|
|
88
92
|
timeout: 0, // wait at max some milliseconds (0 means no timeout)
|
|
89
93
|
valueEncoding: 'json' | 'utf-8' | 'binary' // defaults to the core's valueEncoding
|
|
@@ -97,6 +101,23 @@ Truncate the core to a smaller length.
|
|
|
97
101
|
Per default this will update the fork id of the core to `+ 1`, but you can set the fork id you prefer with the option.
|
|
98
102
|
Note that the fork id should be monotonely incrementing.
|
|
99
103
|
|
|
104
|
+
#### `const hash = await core.treeHash([length])`
|
|
105
|
+
|
|
106
|
+
Get the Merkle Tree hash of the core at a given length, defaulting to the current length of the core.
|
|
107
|
+
|
|
108
|
+
#### `const stream = core.createReadStream([options])`
|
|
109
|
+
|
|
110
|
+
Make a read stream. Options include:
|
|
111
|
+
|
|
112
|
+
``` js
|
|
113
|
+
{
|
|
114
|
+
start: 0,
|
|
115
|
+
end: core.length,
|
|
116
|
+
live: false,
|
|
117
|
+
snapshot: true // auto set end to core.length on open or update it on every read
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
100
121
|
#### `const range = core.download([range])`
|
|
101
122
|
|
|
102
123
|
Download a range of data.
|
|
@@ -113,6 +134,7 @@ A range can have the following properties:
|
|
|
113
134
|
{
|
|
114
135
|
start: startIndex,
|
|
115
136
|
end: nonInclusiveEndIndex,
|
|
137
|
+
blocks: [index1, index2, ...],
|
|
116
138
|
linear: false // download range linearly and not randomly
|
|
117
139
|
}
|
|
118
140
|
```
|
|
@@ -125,6 +147,12 @@ To download the full core continously (often referred to as non sparse mode) do
|
|
|
125
147
|
core.download({ start: 0, end: -1 })
|
|
126
148
|
```
|
|
127
149
|
|
|
150
|
+
To downloaded a discrete range of blocks pass a list of indices.
|
|
151
|
+
|
|
152
|
+
```js
|
|
153
|
+
core.download({ blocks: [4, 9, 7] });
|
|
154
|
+
```
|
|
155
|
+
|
|
128
156
|
To cancel downloading a range simply destroy the range instance.
|
|
129
157
|
|
|
130
158
|
``` js
|
|
@@ -188,6 +216,12 @@ Buffer containing the public key identifying this core.
|
|
|
188
216
|
|
|
189
217
|
Populated after `ready` has been emitted. Will be `null` before the event.
|
|
190
218
|
|
|
219
|
+
#### `core.keyPair`
|
|
220
|
+
|
|
221
|
+
Object containing buffers of the core's public and secret key
|
|
222
|
+
|
|
223
|
+
Populated after `ready` has been emitted. Will be `null` before the event.
|
|
224
|
+
|
|
191
225
|
#### `core.discoveryKey`
|
|
192
226
|
|
|
193
227
|
Buffer containing a key derived from the core's public key.
|
|
@@ -195,6 +229,10 @@ In contrast to `core.key` this key does not allow you to verify the data but can
|
|
|
195
229
|
|
|
196
230
|
Populated after `ready` has been emitted. Will be `null` before the event.
|
|
197
231
|
|
|
232
|
+
#### `core.encryptionKey`
|
|
233
|
+
|
|
234
|
+
Buffer containing the optional block encryption key of this core. Will be `null` unless block encryption is enabled.
|
|
235
|
+
|
|
198
236
|
#### `core.length`
|
|
199
237
|
|
|
200
238
|
How many blocks of data are available on this core?
|
|
@@ -207,12 +245,22 @@ How much data is available on this core in bytes?
|
|
|
207
245
|
|
|
208
246
|
Populated after `ready` has been emitted. Will be `0` before the event.
|
|
209
247
|
|
|
248
|
+
#### `core.contiguousLength`
|
|
249
|
+
|
|
250
|
+
How many blocks are contiguously available starting from the first block of this core?
|
|
251
|
+
|
|
252
|
+
Populated after `ready` has been emitted. Will be `0` before the event.
|
|
253
|
+
|
|
210
254
|
#### `core.fork`
|
|
211
255
|
|
|
212
256
|
What is the current fork id of this core?
|
|
213
257
|
|
|
214
258
|
Populated after `ready` has been emitted. Will be `0` before the event.
|
|
215
259
|
|
|
260
|
+
#### `core.padding`
|
|
261
|
+
|
|
262
|
+
How much padding is applied to each block of this core? Will be `0` unless block encryption is enabled.
|
|
263
|
+
|
|
216
264
|
#### `const stream = core.replicate(isInitiatorOrReplicationStream)`
|
|
217
265
|
|
|
218
266
|
Create a replication stream. You should pipe this to another Hypercore instance.
|
|
@@ -238,10 +286,17 @@ const socket = net.connect(...)
|
|
|
238
286
|
socket.pipe(localCore.replicate(true)).pipe(socket)
|
|
239
287
|
```
|
|
240
288
|
|
|
289
|
+
#### `const done = core.findingPeers()`
|
|
290
|
+
|
|
291
|
+
Create a hook that tells Hypercore you are finding peers for this core in the background. Call `done` when your current discovery iteration is done.
|
|
292
|
+
If you're using Hyperswarm, you'd normally call this after a `swarm.flush()` finishes.
|
|
293
|
+
|
|
294
|
+
This allows `core.update` to wait for either the `findingPeers` hook to finish or one peer to appear before deciding whether it should wait for a merkle tree update before returning.
|
|
295
|
+
|
|
241
296
|
#### `core.on('append')`
|
|
242
297
|
|
|
243
298
|
Emitted when the core has been appended to (i.e. has a new length / byteLength), either locally or remotely.
|
|
244
299
|
|
|
245
|
-
#### `core.on('truncate')`
|
|
300
|
+
#### `core.on('truncate', ancestors, forkId)`
|
|
246
301
|
|
|
247
302
|
Emitted when the core has been truncated, either locally or remotely.
|