tool-db 2.2.0 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (237) hide show
  1. package/README.md +12 -139
  2. package/bundle.js +1 -1
  3. package/dist/adapters/toolDbIndexedb.d.ts +10 -0
  4. package/dist/adapters/toolDbIndexedb.js +128 -0
  5. package/dist/adapters/toolDbIndexedb.js.map +1 -0
  6. package/dist/adapters/toolDbLeveldb.d.ts +9 -0
  7. package/dist/adapters/toolDbLeveldb.js +112 -0
  8. package/dist/adapters/toolDbLeveldb.js.map +1 -0
  9. package/dist/adapters/toolDbWeb3User.d.ts +20 -0
  10. package/dist/adapters/toolDbWeb3User.js +75 -0
  11. package/dist/adapters/toolDbWeb3User.js.map +1 -0
  12. package/dist/adapters/toolDbWebrtc.d.ts +59 -0
  13. package/dist/adapters/toolDbWebrtc.js +492 -0
  14. package/dist/adapters/toolDbWebrtc.js.map +1 -0
  15. package/dist/adapters/toolDbWebsocket.d.ts +19 -0
  16. package/dist/adapters/toolDbWebsocket.js +181 -0
  17. package/dist/adapters/toolDbWebsocket.js.map +1 -0
  18. package/dist/adapters-base/networkAdapter.d.ts +57 -0
  19. package/dist/adapters-base/networkAdapter.js +184 -0
  20. package/dist/adapters-base/networkAdapter.js.map +1 -0
  21. package/dist/adapters-base/storageAdapter.d.ts +9 -0
  22. package/dist/adapters-base/storageAdapter.js +26 -0
  23. package/dist/adapters-base/storageAdapter.js.map +1 -0
  24. package/dist/adapters-base/userAdapter.d.ts +14 -0
  25. package/dist/adapters-base/userAdapter.js +35 -0
  26. package/dist/adapters-base/userAdapter.js.map +1 -0
  27. package/dist/index.d.ts +8 -4
  28. package/dist/index.js +15 -7
  29. package/dist/index.js.map +1 -1
  30. package/dist/logger.d.ts +2 -0
  31. package/dist/logger.js +32 -0
  32. package/dist/logger.js.map +1 -0
  33. package/dist/messageHandlers/handleCrdtGet.js +19 -22
  34. package/dist/messageHandlers/handleCrdtGet.js.map +1 -1
  35. package/dist/messageHandlers/handleCrdtPut.js +23 -14
  36. package/dist/messageHandlers/handleCrdtPut.js.map +1 -1
  37. package/dist/messageHandlers/handleGet.js +19 -21
  38. package/dist/messageHandlers/handleGet.js.map +1 -1
  39. package/dist/messageHandlers/handleJoin.js +2 -2
  40. package/dist/messageHandlers/handleJoin.js.map +1 -1
  41. package/dist/messageHandlers/handlePing.js +1 -0
  42. package/dist/messageHandlers/handlePing.js.map +1 -1
  43. package/dist/messageHandlers/handlePong.js +1 -23
  44. package/dist/messageHandlers/handlePong.js.map +1 -1
  45. package/dist/messageHandlers/handlePut.js +29 -25
  46. package/dist/messageHandlers/handlePut.js.map +1 -1
  47. package/dist/messageHandlers/handleQuery.js +7 -2
  48. package/dist/messageHandlers/handleQuery.js.map +1 -1
  49. package/dist/messageHandlers/handleServers.js +1 -1
  50. package/dist/messageHandlers/handleServers.js.map +1 -1
  51. package/dist/messageHandlers/handleSubscribe.js +30 -29
  52. package/dist/messageHandlers/handleSubscribe.js.map +1 -1
  53. package/dist/sendJoin.d.ts +2 -0
  54. package/dist/sendJoin.js +31 -0
  55. package/dist/sendJoin.js.map +1 -0
  56. package/dist/toolDbClientOnMessage.js +5 -8
  57. package/dist/toolDbClientOnMessage.js.map +1 -1
  58. package/dist/toolDbCrdtGet.js +29 -32
  59. package/dist/toolDbCrdtGet.js.map +1 -1
  60. package/dist/toolDbCrdtPut.js +25 -26
  61. package/dist/toolDbCrdtPut.js.map +1 -1
  62. package/dist/toolDbGet.js +29 -30
  63. package/dist/toolDbGet.js.map +1 -1
  64. package/dist/toolDbKeysSignIn.d.ts +1 -1
  65. package/dist/toolDbKeysSignIn.js +8 -4
  66. package/dist/toolDbKeysSignIn.js.map +1 -1
  67. package/dist/toolDbNetworkAdapter.js +5 -9
  68. package/dist/toolDbNetworkAdapter.js.map +1 -1
  69. package/dist/toolDbPut.js +33 -30
  70. package/dist/toolDbPut.js.map +1 -1
  71. package/dist/toolDbQueryKeys.js +8 -7
  72. package/dist/toolDbQueryKeys.js.map +1 -1
  73. package/dist/toolDbSignIn.js +6 -3
  74. package/dist/toolDbSignIn.js.map +1 -1
  75. package/dist/toolDbSignUp.js +37 -33
  76. package/dist/toolDbSignUp.js.map +1 -1
  77. package/dist/toolDbSubscribe.d.ts +2 -1
  78. package/dist/toolDbSubscribe.js +20 -16
  79. package/dist/toolDbSubscribe.js.map +1 -1
  80. package/dist/toolDbVerificationWrapper.js +15 -10
  81. package/dist/toolDbVerificationWrapper.js.map +1 -1
  82. package/dist/toolDbWebrtc.d.ts +0 -6
  83. package/dist/toolDbWebrtc.js +16 -15
  84. package/dist/toolDbWebrtc.js.map +1 -1
  85. package/dist/toolDbWebsocket.js +9 -15
  86. package/dist/toolDbWebsocket.js.map +1 -1
  87. package/dist/tooldb.d.ts +9 -3
  88. package/dist/tooldb.js +12 -9
  89. package/dist/tooldb.js.map +1 -1
  90. package/dist/types/message.d.ts +1 -1
  91. package/dist/types/tooldb.d.ts +7 -27
  92. package/dist/types/tooldb.js +0 -33
  93. package/dist/types/tooldb.js.map +1 -1
  94. package/dist/utils/getPeerSignature.d.ts +2 -2
  95. package/dist/utils/indexedb.js +4 -4
  96. package/dist/utils/indexedb.js.map +1 -1
  97. package/dist/utils/leveldb.js +4 -4
  98. package/dist/utils/leveldb.js.map +1 -1
  99. package/dist/utils/verifyMessage.js +37 -33
  100. package/dist/utils/verifyMessage.js.map +1 -1
  101. package/dist/utils/verifyPeer.js +1 -1
  102. package/dist/utils/verifyPeer.js.map +1 -1
  103. package/package.json +1 -1
  104. package/test-alice/000145.ldb +0 -0
  105. package/test-alice/000147.ldb +0 -0
  106. package/test-alice/000150.ldb +0 -0
  107. package/test-alice/000153.ldb +0 -0
  108. package/test-alice/000154.log +0 -0
  109. package/test-alice/CURRENT +1 -1
  110. package/test-alice/LOG +5 -5
  111. package/test-alice/LOG.old +5 -5
  112. package/test-alice/MANIFEST-000152 +0 -0
  113. package/test-alice-speed/000022.ldb +0 -0
  114. package/test-alice-speed/000024.ldb +0 -0
  115. package/test-alice-speed/000027.ldb +0 -0
  116. package/test-alice-speed/000030.ldb +0 -0
  117. package/test-alice-speed/000031.log +0 -0
  118. package/test-alice-speed/CURRENT +1 -1
  119. package/test-alice-speed/LOG +5 -13
  120. package/test-alice-speed/LOG.old +5 -5
  121. package/test-alice-speed/MANIFEST-000029 +0 -0
  122. package/{test-verify-a/000250.log → test-base-client/000099.log} +0 -0
  123. package/test-base-client/CURRENT +1 -0
  124. package/test-base-client/LOCK +0 -0
  125. package/test-base-client/LOG +3 -0
  126. package/test-base-client/LOG.old +3 -0
  127. package/test-base-client/MANIFEST-000098 +0 -0
  128. package/test-base-server/000101.log +0 -0
  129. package/test-base-server/CURRENT +1 -0
  130. package/test-base-server/LOCK +0 -0
  131. package/test-base-server/LOG +3 -0
  132. package/test-base-server/LOG.old +3 -0
  133. package/test-base-server/MANIFEST-000100 +0 -0
  134. package/test-bob/000160.ldb +0 -0
  135. package/test-bob/000162.ldb +0 -0
  136. package/test-bob/000163.log +0 -0
  137. package/test-bob/CURRENT +1 -1
  138. package/test-bob/LOG +5 -5
  139. package/test-bob/LOG.old +14 -14
  140. package/test-bob/MANIFEST-000161 +0 -0
  141. package/test-chris/000017.ldb +0 -0
  142. package/test-chris/000020.ldb +0 -0
  143. package/test-chris/000023.ldb +0 -0
  144. package/test-chris/000026.ldb +0 -0
  145. package/test-chris/000029.ldb +0 -0
  146. package/test-chris/000086.ldb +0 -0
  147. package/test-chris/000089.ldb +0 -0
  148. package/test-chris/000092.ldb +0 -0
  149. package/test-chris/000095.ldb +0 -0
  150. package/test-chris/000098.ldb +0 -0
  151. package/test-chris/000103.ldb +0 -0
  152. package/test-chris/000106.ldb +0 -0
  153. package/test-chris/000109.ldb +0 -0
  154. package/test-chris/000112.ldb +0 -0
  155. package/test-chris/000115.ldb +0 -0
  156. package/test-chris/000118.ldb +0 -0
  157. package/test-chris/000121.ldb +0 -0
  158. package/test-chris/000124.ldb +0 -0
  159. package/test-chris/000127.ldb +0 -0
  160. package/test-chris/000130.ldb +0 -0
  161. package/test-chris/000133.ldb +0 -0
  162. package/test-chris/000136.ldb +0 -0
  163. package/test-chris/000139.ldb +0 -0
  164. package/test-chris/000142.ldb +0 -0
  165. package/test-chris/000145.ldb +0 -0
  166. package/test-chris/000148.ldb +0 -0
  167. package/test-chris/000151.ldb +0 -0
  168. package/test-chris/000152.log +0 -0
  169. package/test-chris/CURRENT +1 -0
  170. package/test-chris/LOCK +0 -0
  171. package/test-chris/LOG +6 -0
  172. package/test-chris/LOG.old +6 -0
  173. package/test-chris/MANIFEST-000150 +0 -0
  174. package/test-node-a/000175.ldb +0 -0
  175. package/test-node-a/000177.ldb +0 -0
  176. package/test-node-a/000178.log +0 -0
  177. package/test-node-a/CURRENT +1 -1
  178. package/test-node-a/LOG +5 -5
  179. package/test-node-a/LOG.old +14 -5
  180. package/test-node-a/MANIFEST-000176 +0 -0
  181. package/test-node-b/000175.ldb +0 -0
  182. package/test-node-b/000177.ldb +0 -0
  183. package/test-node-b/000178.log +0 -0
  184. package/test-node-b/CURRENT +1 -1
  185. package/test-node-b/LOG +5 -5
  186. package/test-node-b/LOG.old +14 -5
  187. package/test-node-b/MANIFEST-000176 +0 -0
  188. package/test-store/000007.ldb +0 -0
  189. package/test-store/000010.ldb +0 -0
  190. package/test-store/000013.ldb +0 -0
  191. package/test-store/000016.ldb +0 -0
  192. package/test-store/000019.ldb +0 -0
  193. package/test-store/000022.ldb +0 -0
  194. package/test-store/000025.ldb +0 -0
  195. package/test-store/000028.ldb +0 -0
  196. package/test-store/000031.ldb +0 -0
  197. package/test-store/000034.ldb +0 -0
  198. package/test-store/000037.ldb +0 -0
  199. package/test-store/000038.log +0 -0
  200. package/test-store/CURRENT +1 -0
  201. package/test-store/LOCK +0 -0
  202. package/test-store/LOG +6 -0
  203. package/test-store/LOG.old +6 -0
  204. package/test-store/MANIFEST-000036 +0 -0
  205. package/test-verify-a/000049.log +0 -0
  206. package/test-verify-a/CURRENT +1 -1
  207. package/test-verify-a/LOG +3 -3
  208. package/test-verify-a/LOG.old +3 -3
  209. package/test-verify-a/MANIFEST-000048 +0 -0
  210. package/toolDbLibP2p.ts +12 -11
  211. package/test-alice/000663.ldb +0 -0
  212. package/test-alice/000665.ldb +0 -0
  213. package/test-alice/000668.ldb +0 -0
  214. package/test-alice/000669.log +0 -0
  215. package/test-alice/MANIFEST-000667 +0 -0
  216. package/test-alice-speed/000035.log +0 -0
  217. package/test-alice-speed/000036.ldb +0 -0
  218. package/test-alice-speed/MANIFEST-000033 +0 -0
  219. package/test-bob/000665.ldb +0 -0
  220. package/test-bob/000667.ldb +0 -0
  221. package/test-bob/000668.log +0 -0
  222. package/test-bob/MANIFEST-000666 +0 -0
  223. package/test-node-a/000663.ldb +0 -0
  224. package/test-node-a/000665.ldb +0 -0
  225. package/test-node-a/000668.ldb +0 -0
  226. package/test-node-a/000669.log +0 -0
  227. package/test-node-a/MANIFEST-000667 +0 -0
  228. package/test-node-b/000663.ldb +0 -0
  229. package/test-node-b/000665.ldb +0 -0
  230. package/test-node-b/000668.ldb +0 -0
  231. package/test-node-b/000669.log +0 -0
  232. package/test-node-b/MANIFEST-000667 +0 -0
  233. package/test-verify-a/000026.ldb +0 -0
  234. package/test-verify-a/000029.ldb +0 -0
  235. package/test-verify-a/000031.ldb +0 -0
  236. package/test-verify-a/000033.ldb +0 -0
  237. package/test-verify-a/MANIFEST-000249 +0 -0
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Tool Db
2
2
 
3
- ToolDb is a peer-to-peer model for a decentralized database, inspired by Gun (Mark Nadal): https://gun.eco/
3
+ ToolDb is a peer-to-peer model for a decentralized database, inspired by GundB
4
4
 
5
5
 
6
6
  It is the mix of a lot of cool concepts into one;
@@ -29,9 +29,8 @@ We do that using [discovery-channel](https://www.npmjs.com/package/discovery-cha
29
29
 
30
30
  While the database is currently functioning as expected, there are many things that could be added or improved to make it even better!
31
31
 
32
- - Use a common web3 format for users identity, or a more standarized key pair.
33
- - Allow messages encryption, add methods for Elliptic Curves, shared keys, etc.
34
- - Allow adding Noise/encryption to connections (probably based on the ECC?)
32
+ - Allow data encryption (proably built in), add methods for ECC encryption, shared keys, etc.
33
+ - Allow adding Noise/encryption to connections (probably based on the peer identity, requires initial keys exchange on connection)
35
34
 
36
35
  # Install
37
36
 
@@ -54,148 +53,22 @@ const { ToolDb, sha256 } = tooldb;
54
53
  ```
55
54
 
56
55
  ## Base usage
57
- Connect to the selected toolDb peers;
58
- ```
59
- const client = new ToolDb(options);
60
- ```
61
56
 
62
- These are the options you can pass to the constructor:
57
+ Creating a webrtc peers network is as easy as;
63
58
 
64
59
  ```
65
- {
66
- // Database name to use (default "tooldb")
67
- db: string;
68
-
69
- // Show debug console logs (default false)
70
- debug: boolean;
71
-
72
- // Array of peers to connect to, each one in the form of { host: "127.0.0.1", port: 9000 }
73
- peers: { host: "127.0.0.1", port: 9000 }[];
74
-
75
- // Max number of tries when a connection fails (default 5)
76
- maxRetries: number;
77
-
78
- // How long to wait (max) for a debounced key listener recv (default 100)
79
- triggerDebouce: number;
80
-
81
- // How long to wait between retries (default 2000)
82
- wait: number;
83
-
84
- // If you want to force a Proof of Work on all messages, set how much (zero is no POW, default 0)
85
- pow: number;
86
-
87
- // Whether we are a server or not (default false)
88
- server: boolean;
89
-
90
- // Our hostname (server only)
91
- host: string
92
-
93
- // Port to listen incoming connections (server only, default is 8080)
94
- port: number;
95
-
96
- // A server instance like Express (server only)
97
- httpServer: HTTPServer | HTTPSServer | undefined;
98
-
99
- // Our storage namespace (default is "tooldb")
100
- storageName: string;
101
-
102
- // A custom network adapter class
103
- networkAdapter: typeof ToolDbNetworkAdapter;
60
+ import { ToolDb, ToolDbWebrtc } from "tool-db";
104
61
 
105
- // A custom storage adapter function
106
- storageAdapter: ToolDbStorageAdapter;
107
-
108
- // The namespace/topic of our app (default is "tool-db-default")
109
- topic: string;
110
- }
62
+ const db = new ToolDb({
63
+ networkAdapter: ToolDbWebrtc,
64
+ debug: true,
65
+ });
111
66
  ```
112
- Notice you can use your own network and storage modules if you would like to implement custom solutions for peers discovery, connections, storage, etc, the default adapters will work both on nodejs and the browser.
113
-
114
67
 
115
- You can create a user, sign in or create a random set of keys for anonymous usage;
116
- ```
117
- client.signUp(user, pass).then();
118
- client.signIn(user, pass).then(keys);
119
- client.anonSignIn().then();
120
- ```
121
-
122
- To retrieve your address; (only if logged in)
123
- ```
124
- client.getAddress();
125
- ```
68
+ From there you can put and get data using the api;
126
69
 
127
- You can check if you are correctly logged in by checking if the user field exists or not;
128
70
  ```
129
- if (client.user) {
130
- // Ok!
131
- }
71
+ db.putData("foo-key", "var").then(console.log);
132
72
  ```
133
73
 
134
- ## Putting and getting data
135
-
136
- Core methods are very straighforward:
137
- ```
138
- client.getData("key", userNamespaced?, timeout?).then();
139
- client.putData("key", value, userNamespaced?).then();
140
- client.putCrdt("key", documentChanges, userNamespaced?).then();
141
- ```
142
- UserNamespaced will check for the user namespace, so it will transform the key used to ":user.key", This way all peers reading this entry will understand the key belongs to this user and will enforce the verifications required to namespaced entries.
143
-
144
- You can use `client.getUserNamespacedKey(key)` to convert any key to a private namespaced key of the current logged in user.
145
-
146
- Note the third function (putCrdt); For p2p networks some times you want to have conflict resolution on certain documents, to do this we use [automerge](https://github.com/automerge/automerge). Please take a look at it to know how it works in detail! The only thing you need to know on Tool Db is that you have to send an `Automerge.BinaryChange[]` (you get it using `Automerge.change()`). The recieving peer will process the changes, compare it to its stored documents and generate a new document with JSON CRDT applied to it, then relay the final document back to you.
147
-
148
- To listen for a value changes you can set up a listener on a key. Beware the listener will check for all keys *starting with* the supplied key, so for example, if you use "value." as your listener if will execute on every key that starts with "value.". This is useful for checking against an address or namespace.
149
- ```
150
- const listenerId = client.addKeyListener("value", console.log);
151
- client.removeKeyListener(listenerId);
152
- ```
153
-
154
- Similarly, for using a custom verification on a key (or subset of keys) you can create a new function that returns a Promise boolean;
155
- ```
156
- const validateFn = (msg) => {
157
- return new Promise((resolve) => {
158
- console.log("Custom verification: ", msg);
159
- if (typeof msg.value === "string") resolve(true);
160
- else resolve(false);
161
- });
162
- }
163
-
164
- const validatorId = client.addCustomVerification("value", validateFn);
165
-
166
- client.removeCustomVerification(validatorId);
167
- ```
168
-
169
- Keep in mind custom validators should run on all client and server nodes, and even though the nodes not running your validator will be able to store and relay invalid messages its up to each peer to check these messages on arrival. ToolDb does this automatically when using the custom validation, but its important to make sure every peer runs the same code to avoid tampered messages flowing in the network.
170
-
171
- ## Query
172
-
173
- To make queries or just create indexes you can make a query, it simply asks all server or peers connected a list of all keys starting with a prefix. You can later use these keys to get the data itself.
174
-
175
- ```
176
- client.queryKeys(keyPrefix, userNamespaced?, timeout?).then()
177
- ```
178
-
179
- Just like previous methods you can configure the namespace and timeout and returns a promise, but in this case the value is always an array with the found keys.
180
- Using for example ":" + address as our prefix would return all keys stored for that specific user namespace; you can also create keys for very specific use cases like "post-1" and query again "post-" to get a list of all available posts.
181
-
182
- Keep in mind this can be an intensive thing to do if your indexes are too big, and not recommended to be used very frequently.
183
-
184
-
185
- ## Listen for changes
186
-
187
- Some times you want to subscribe to the changes made on a certain key, this is possible via subscribeData;
188
- ```
189
- client.subscribeData("key", userNamespaced?);
190
- ```
191
-
192
- This will relay back to you all Put and CrdtPut messages on that key.
193
-
194
- ## Events
195
-
196
- If you need to check when you are connected to a server peer or not you can use the following method replacements;
197
-
198
- ```
199
- client.onConnect = (remotePeerId: string) => { /* Your code here */ };
200
- client.onDisconnect = () => { /* Your code here */ };
201
- ```
74
+ There is a lot more you can do, like subscribe for updates, built-in users credentials validation based on ECC, create federated networks, run servers on Nodejs, auto-replicate data trough them, etc. Make sure more you read trough the [documentation](https://github.com/Manwe-777/tool-db-docs)!