@streamr/node 100.0.0-rc.0

package/LICENSE

@@ -0,0 +1,78 @@
+STREAMR NETWORK OPEN SOURCE LICENSE
+-----------------------------------
+
+PREAMBLE
+
+This software is licensed under the Streamr Network Open 
+Source License. The Streamr Network Open Source License is 
+based on the GNU Affero General Public License, version 3 
+(19 November 2007), but it contains a few additional 
+limitations and requirements.
+
+The main addition is the introduction of operating parameters 
+which apply to the software. These parameters may induce the 
+use of specific cryptographic tokens (such as the Streamr
+DATAcoin) in the operations of the Streamr Network. Doing so 
+protects the considerable effort already undertaken by the
+Streamr community.
+
+These license terms allow you to create, modify and propagate 
+new versions of the Streamr Network software (“Program”). Any
+copy or modified version shall always comply with certain 
+network operating parameters, as amended from time to time. 
+In a manner to be determined in the future, it is Streamr’s 
+intention to decentralize the governance and decision-making 
+processes relating to the network operating parameters. The 
+Streamr Network is also available under a proprietary 
+commercial license pursuant to a dual licensing scheme. 
+Please contact Streamr directly at contact@streamr.network if 
+you are interested in such a proprietary license. 
+
+ADDITIONAL TERMS
+
+1. These additional terms (“Additional Terms”) shall be in 
+addition to and have precedence over the GNU Affero General 
+Public License, Version 3 (“License Terms”). The License Terms 
+shall be deemed to be incorporated herein as a material part 
+of these Additional Terms. Any capitalized terms herein shall 
+have the meaning given to them in the License Terms. 
+
+2. Unless explicitly otherwise stated in these Additional 
+Terms, the License Terms shall be applicable on and govern 
+your use, modification and propagation of the Program. 
+
+3. Definitions:
+
+    a. “Supported Parameters” means the operating parameters 
+    of the Program that are indicated in the smart contract 
+    pointed to by Ethereum Name Service (ENS) registry entry 
+    network-parameters.streamr.eth from time to time. 
+    
+    b. “Reference Versions” means the minimum version number 
+    of the Program that is indicated in the Supported Parameters 
+    as well as any newer version of the Program (as indicated 
+    by a higher version number). 
+    
+    c. “DATA” means the Streamr DATAcoin cryptographic token, 
+    in each case as set out in and as supported by the Supported 
+    Parameters from time to time.
+    
+4. Subject to Section 5, any modified version of the Program 
+shall incorporate and implement all of, and only operate in
+accordance with, the Supported Parameters, as implemented in 
+the Reference Versions. For the avoidance of doubt, as an 
+example, if the modified version involves the use of crypto-
+graphic tokens (such as DATA), such tokens must be set out in
+the Supported Parameters. Any breach of this Section 4 is 
+considered a material breach of these Additional Terms. 
+
+5. Notwithstanding anything to the contrary in Section 4, 
+modification and propagation of the Program for development 
+and testing purposes (meaning e.g., creation of small-scale 
+prototypes or proofs of concept) is allowed without implementing 
+the Supported Parameters; provided, however, that the use of 
+such modified version shall be limited only to the time period 
+necessary for such development or testing purposes. 
+
+6. Any violation or breach of these Additional Terms shall be 
+deemed a violation or breach of the License Terms.

package/README.md

@@ -0,0 +1,54 @@
+<p align="center">
+  <a href="https://streamr.network">
+    <img alt="Streamr" src="https://raw.githubusercontent.com/streamr-dev/network-monorepo/main/packages/client/readme-header.png" width="1320" />
+  </a>
+</p>
+
+# @streamr/node
+Broker nodes are Streamr Network nodes that run external to your application. You start up a broker node locally or on a
+server, and interface with it remotely using one of the supported protocols.
+
+The broker node ships with protocol support for HTTP, WebSocket, and MQTT. As libraries for these common protocols exist
+in practically every programming language, you can conveniently publish data to and subscribe to streams in the Streamr Network
+using the programming language of your choice.
+
+Broker nodes can also perform other tasks in addition to (or instead of) serving applications, such as mining.
+
+## Table of Contents
+- [Install](#install)
+- [Run](#run)
+- [Configuration](#configuration)
+- [Plugins](#plugins)
+
+## Install
+```
+npm install -g @streamr/node
+```
+
+For more information on the different ways of setting up a broker node, see
+[setting up a Broker node](https://streamr.network/docs/streamr-network/installing-broker-node).
+
+## Run
+
+First [install](#install) the package globally if you have not yet.
+
+Create a configuration file with interactive tool:
+```
+streamr-broker-init 
+```
+Then run the command broker with the desired configuration file:
+```
+streamr-broker <configFile>
+```
+
+## Configuration
+
+See [configuration](configuration.md) for a description of the configuration options.
+
+## Plugins
+
+The broker node ships with a number of plugins that can be enabled and configured selectively to match your specific
+needs. For easy data integration from any programming language environment, plugins for HTTP, WebSocket, and MQTT are
+provided.
+
+Read more about available [plugins](plugins.md).

package/bin/broker.ts

@@ -0,0 +1,36 @@
+#!/usr/bin/env node
+import { program } from 'commander'
+import pkg from '../package.json'
+
+import { createBroker } from '../src/broker'
+import { readConfigAndMigrateIfNeeded } from '../src/config/migration'
+import { overrideConfigToEnvVarsIfGiven } from '../src/config/config'
+
+program
+    .version(pkg.version)
+    .name('broker')
+    .description('Run broker under environment specified by given configuration file.')
+    .arguments('[configFile]')
+    .option('--test', 'test the configuration (does not start the broker)')
+    .action(async (configFile) => {
+        try {
+            const config = readConfigAndMigrateIfNeeded(configFile)
+            overrideConfigToEnvVarsIfGiven(config)
+            const broker = await createBroker(config)
+            if (!program.opts().test) {
+                await broker.start()
+            } else {
+                // eslint-disable-next-line no-console
+                console.info('the configuration is valid')
+                // TODO remove process.exit(0)
+                // We should not need explicit exit call if all setTimeouts are cleared.
+                // Currently there is only one leaking timeout in PingPongWs (created
+                // by NodeClientWsEndpoint from the createNetworkNode() call)
+                process.exit(0)
+            }
+        } catch (err) {
+            console.error(err)
+            process.exit(1)
+        }
+    })
+    .parse(process.argv)

package/bin/config-wizard.js

@@ -0,0 +1,4 @@
+// Placeholder script so we don't have to update docs yet
+// https://streamr.network/docs/streamr-network/installing-broker-node.
+// Remove in next version update
+require('child_process').fork('dist/bin/config-wizard.js')

package/bin/config-wizard.ts

@@ -0,0 +1,17 @@
+#!/usr/bin/env node
+import { program } from 'commander'
+import pkg from '../package.json'
+import { start } from '../src/config/ConfigWizard'
+
+program
+    .version(pkg.version)
+    .name('broker-config-wizard')
+    .description('Run the configuration wizard for the broker')
+
+;(async () => {
+    try {
+        await start()
+    } catch (e) {
+        console.error('Streamr Node Config Wizard encountered an error:\n', e)
+    }
+})()

package/bin/delete-expired-data.ts

@@ -0,0 +1,41 @@
+#!/usr/bin/env node
+import { program } from 'commander'
+import pkg from '../package.json'
+import { DeleteExpiredCmd } from '../src/plugins/storage/DeleteExpiredCmd'
+
+program
+    .version(pkg.version)
+    .requiredOption('--cassandra-username <username>')
+    .requiredOption('--cassandra-password <password>')
+    .requiredOption('--cassandra-hosts <hosts_delimited_by_comma>')
+    .requiredOption('--cassandra-datacenter <datacenter>')
+    .requiredOption('--cassandra-keyspace <keyspace>')
+    .requiredOption('--streamr-base-url <baseUrl>')
+    .requiredOption('--bucket-limit <bucketLimit>', 'max number of buckets to fetch')
+    .option('--real-run', 'delete data for real', false)
+    .description('Delete expired data')
+    .parse(process.argv)
+
+const deleteExpiredCommand = new DeleteExpiredCmd({
+    cassandraUsername: program.opts().cassandraUsername,
+    cassandraPassword: program.opts().cassandraPassword,
+    cassandraHosts: program.opts().cassandraHosts.split(','),
+    cassandraDatacenter: program.opts().cassandraDatacenter,
+    cassandraKeyspace: program.opts().cassandraKeyspace,
+    streamrBaseUrl: program.opts().streamrBaseUrl,
+    bucketLimit: program.opts().bucketLimit,
+    dryRun: !program.opts().realRun
+})
+
+async function run() {
+    try {
+        // @ts-expect-error TODO fix
+        await deleteExpiredCommand.run()
+        return {}
+    } catch (err) {
+        console.error(err)
+        process.exit(1)
+    }
+}
+
+run()

package/bin/entry-point.ts

@@ -0,0 +1,27 @@
+#!/usr/bin/env node
+import { config as CHAIN_CONFIG } from '@streamr/config'
+import { DhtAddress, DhtNode, NodeType, getRawFromDhtAddress } from '@streamr/dht'
+
+const main = async () => {
+    const entryPoint = CHAIN_CONFIG.dev2.entryPoints[0]
+    const peerDescriptor = {
+        ...entryPoint,
+        nodeId: getRawFromDhtAddress(entryPoint.nodeId as DhtAddress),
+        type: NodeType.NODEJS  // TODO remove this when NET-1070 done
+    }
+    const dhtNode = new DhtNode({
+        nodeId: entryPoint.nodeId as DhtAddress,
+        websocketHost: entryPoint.websocket.host,
+        websocketPortRange: {
+            min: entryPoint.websocket.port,
+            max: entryPoint.websocket.port
+        },
+        websocketServerEnableTls: false,
+        entryPoints: [peerDescriptor]
+    })
+    await dhtNode.start()
+    await dhtNode.joinDht([peerDescriptor])
+    console.info('Entry point started')
+}
+
+main()

package/configs/development-1.env.json

@@ -0,0 +1,68 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0xaa7a3b3bb9b4a662e756e978ad8c6464412e7eef1b871f19e5120d4747bce966"
+        },
+        "network": {
+            "controlLayer": {
+                "peerDescriptor": {
+                    "nodeId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb1",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40401,
+                        "tls": false
+                    }
+                },
+                "entryPoints": [{
+                    "nodeId": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40500,
+                        "tls": false
+                    }
+                }],
+                "iceServers": [],
+                "webrtcAllowPrivateAddresses": true,
+                "websocketServerEnableTls": false
+            },
+            "node": {
+                "id": "0xde1112f631486CfC759A50196853011528bC5FA0"
+            }
+        },
+        "contracts": {
+            "streamRegistryChainAddress": "0xd04af489677001444280366Dd0885B03dAaDe71D",
+            "streamStorageRegistryChainAddress": "0xB9372284e0D61607aF3B7EF5f022e7D599Ed2a37",
+            "storageNodeRegistryChainAddress": "0xCBAcfA0592B3D809aEc805d527f8ceAe9307D9C0",
+            "streamRegistryChainRPCs": {
+                "chainId": 8995,
+                "rpcs": [
+                    {
+                        "url": "http://10.200.10.1:8547"
+                    }
+                ]
+            },
+            "theGraphUrl": "http://10.200.10.1:8800/subgraphs/name/streamr-dev/network-subgraphs"
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8891
+    },
+    "plugins": {
+        "storage": {
+            "cassandra": {
+                "hosts": [
+                    "10.200.10.1"
+                ],
+                "username": "",
+                "password": "",
+                "keyspace": "streamr_dev_v2",
+                "datacenter": "datacenter1"
+            },
+            "storageConfig": {
+                "refreshInterval": 10000
+            }
+        }
+    }
+}

package/configs/development-2.env.json

@@ -0,0 +1,53 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0xa51ef2b7a1f160e53cc3a17c5a74ea0563a50030644e9f6e1b1f4a7d0afd088e"
+        },
+        "network": {
+            "controlLayer": {
+                "peerDescriptor": {
+                    "nodeId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb2",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40402,
+                        "tls": false
+                    }
+                },
+                "entryPoints": [{
+                    "nodeId": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40500,
+                        "tls": false
+                    }
+                }],
+                "iceServers": [],
+                "webrtcAllowPrivateAddresses": true,
+                "websocketServerEnableTls": false
+            },
+            "node": {
+                "id": "0xde222E8603FCf641F928E5F66a0CBf4de70d5352"
+            }
+        },
+        "contracts": {
+            "streamRegistryChainAddress": "0xd04af489677001444280366Dd0885B03dAaDe71D",
+            "streamStorageRegistryChainAddress": "0xB9372284e0D61607aF3B7EF5f022e7D599Ed2a37",
+            "storageNodeRegistryChainAddress": "0xCBAcfA0592B3D809aEc805d527f8ceAe9307D9C0",
+            "streamRegistryChainRPCs": {
+                "chainId": 8995,
+                "rpcs": [
+                    {
+                        "url": "http://10.200.10.1:8547"
+                    }
+                ]
+            },
+            "theGraphUrl": "http://10.200.10.1:8800/subgraphs/name/streamr-dev/network-subgraphs"
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8791
+    },
+    "plugins": {}
+}

package/configs/development-3.env.json

@@ -0,0 +1,53 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0x2ba0e218041e81b8e5c88ee4134995c1e358c2c01664676d975792057c57e333"
+        },
+        "network": {
+            "controlLayer": {
+                "peerDescriptor": {
+                    "nodeId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb3",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40403,
+                        "tls": false
+                    }
+                },
+                "entryPoints": [{
+                    "nodeId": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40500,
+                        "tls": false
+                    }
+                }],
+                "iceServers": [],
+                "webrtcAllowPrivateAddresses": true,
+                "websocketServerEnableTls": false
+            },
+            "node": {
+                "id": "0xde3331cA6B8B636E0b82Bf08E941F727B8927442"
+            }
+        },
+        "contracts": {
+            "streamRegistryChainAddress": "0xd04af489677001444280366Dd0885B03dAaDe71D",
+            "streamStorageRegistryChainAddress": "0xB9372284e0D61607aF3B7EF5f022e7D599Ed2a37",
+            "storageNodeRegistryChainAddress": "0xCBAcfA0592B3D809aEc805d527f8ceAe9307D9C0",
+            "streamRegistryChainRPCs": {
+                "chainId": 8995,
+                "rpcs": [
+                    {
+                        "url": "http://10.200.10.1:8547"
+                    }
+                ]
+            },
+            "theGraphUrl": "http://10.200.10.1:8800/subgraphs/name/streamr-dev/network-subgraphs"
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8691
+    },
+    "plugins": {}
+}

package/configs/development-prod-resend.env.json

@@ -0,0 +1,21 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0x87ba653d4320abb5231d10a4abfad9ea1371dcbcdc247ab2f961e2754ae781ca"
+        },
+        "network": {
+            "node": {
+                "id": "0xa9D6428B05a65053B276AeCc917ED07ACF219aE2"
+            },
+            "controlLayer": {
+                "websocketServerEnableTls": false
+            }
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8791
+    },
+    "plugins": {}
+}

package/configs/docker-1.env.json

@@ -0,0 +1,72 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0xaa7a3b3bb9b4a662e756e978ad8c6464412e7eef1b871f19e5120d4747bce966"
+        },
+        "network": {
+            "controlLayer": {
+                "peerDescriptor": {
+                    "nodeId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb1",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40401,
+                        "tls": false
+                    }
+                },
+                "entryPoints": [{
+                    "nodeId": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40500,
+                        "tls": false
+                    }
+                }],
+                "iceServers": [],
+                "webrtcAllowPrivateAddresses": true,
+                "websocketServerEnableTls": false
+            },
+            "node": {
+                "id": "0xde1112f631486CfC759A50196853011528bC5FA0"
+            }
+        },
+        "contracts": {
+            "streamRegistryChainAddress": "0xd04af489677001444280366Dd0885B03dAaDe71D",
+            "streamStorageRegistryChainAddress": "0xB9372284e0D61607aF3B7EF5f022e7D599Ed2a37",
+            "storageNodeRegistryChainAddress": "0xCBAcfA0592B3D809aEc805d527f8ceAe9307D9C0",
+            "streamRegistryChainRPCs": {
+                "chainId": 8995,
+                "rpcs": [
+                    {
+                        "url": "http://10.200.10.1:8547"
+                    }
+                ]
+            },
+            "theGraphUrl": "http://10.200.10.1:8800/subgraphs/name/streamr-dev/network-subgraphs"
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8891
+    },
+    "plugins": {
+        "consoleMetrics": {
+            "interval": 30
+        },
+        "info": {},
+        "storage": {
+            "cassandra": {
+                "hosts": [
+                    "cassandra"
+                ],
+                "username": "",
+                "password": "",
+                "keyspace": "streamr_dev_v2",
+                "datacenter": "datacenter1"
+            },
+            "storageConfig": {
+                "refreshInterval": 10000
+            }
+        }
+    }
+}

package/configs/docker-2.env.json

@@ -0,0 +1,58 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0xa51ef2b7a1f160e53cc3a17c5a74ea0563a50030644e9f6e1b1f4a7d0afd088e"
+        },
+        "network": {
+            "controlLayer": {
+                "peerDescriptor": {
+                    "nodeId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb2",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40402,
+                        "tls": false
+                    }
+                },
+                "entryPoints": [{
+                    "nodeId": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40500,
+                        "tls": false
+                    }
+                }],
+                "iceServers": [],
+                "webrtcAllowPrivateAddresses": true,
+                "websocketServerEnableTls": false
+            },
+            "node": {
+                "id": "0xde222E8603FCf641F928E5F66a0CBf4de70d5352"
+            }
+        },
+        "contracts": {
+            "streamRegistryChainAddress": "0xd04af489677001444280366Dd0885B03dAaDe71D",
+            "streamStorageRegistryChainAddress": "0xB9372284e0D61607aF3B7EF5f022e7D599Ed2a37",
+            "storageNodeRegistryChainAddress": "0xCBAcfA0592B3D809aEc805d527f8ceAe9307D9C0",
+            "streamRegistryChainRPCs": {
+                "chainId": 8995,
+                "rpcs": [
+                    {
+                        "url": "http://10.200.10.1:8547"
+                    }
+                ]
+            },
+            "theGraphUrl": "http://10.200.10.1:8800/subgraphs/name/streamr-dev/network-subgraphs"
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8791
+    },
+    "plugins": {
+        "consoleMetrics": {
+            "interval": 30
+        },
+        "info": {}
+    }
+}

package/configs/docker-3.env.json

@@ -0,0 +1,58 @@
+{
+    "$schema": "https://schema.streamr.network/config-v2.schema.json",
+    "client": {
+        "auth": {
+            "privateKey": "0x2ba0e218041e81b8e5c88ee4134995c1e358c2c01664676d975792057c57e333"
+        },
+        "network": {
+            "controlLayer": {
+                "peerDescriptor": {
+                    "nodeId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb3",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40403,
+                        "tls": false
+                    }
+                },
+                "entryPoints": [{
+                    "nodeId": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
+                    "websocket": {
+                        "host": "10.200.10.1",
+                        "port": 40500,
+                        "tls": false
+                    }
+                }],
+                "iceServers": [],
+                "webrtcAllowPrivateAddresses": true,
+                "websocketServerEnableTls": false
+            },
+            "node": {
+                "id": "0xde3331cA6B8B636E0b82Bf08E941F727B8927442"
+            }
+        },
+        "contracts": {
+            "streamRegistryChainAddress": "0xd04af489677001444280366Dd0885B03dAaDe71D",
+            "streamStorageRegistryChainAddress": "0xB9372284e0D61607aF3B7EF5f022e7D599Ed2a37",
+            "storageNodeRegistryChainAddress": "0xCBAcfA0592B3D809aEc805d527f8ceAe9307D9C0",
+            "streamRegistryChainRPCs": {
+                "chainId": 8995,
+                "rpcs": [
+                    {
+                        "url": "http://10.200.10.1:8547"
+                    }
+                ]
+            },
+            "theGraphUrl": "http://10.200.10.1:8800/subgraphs/name/streamr-dev/network-subgraphs"
+        },
+        "metrics": false
+    },
+    "httpServer": {
+        "port": 8691
+    },
+    "plugins": {
+        "consoleMetrics": {
+            "interval": 30
+        },
+        "info": {}
+    }
+}