libp2p 0.42.2 → 0.43.0

package/src/circuit/IMPLEMENTATION_NOTES.md

@@ -1,128 +0,0 @@
-EDIT: This document is outdated and here only for historical purposes
-
-NOTE: This document is structured in an `if-then/else[if]-then` manner, each line is a precondition for following lines with a higher number of indentation
-
-Example:
-
-- if there are apples
-    - eat them
-- if not, check for pears
-    - then eat them
-- if not, check for cherries
-    - then eat them
-
-Or,
-
-- if there are apples
-    - eat them
-- if not
-    - check for pears
-        - then eat them
-- if not
-    - check for cherries
-        - then eat them
-
-In order to minimize nesting, the first example is preferred
-
-# Relay flow
-
-## Relay transport (dialer/listener)
-
-- ### Dial over a relay
-    - See if there is a relay that's already connected to the destination peer, if not
-        - Ask all the peer's known relays to dial the destination peer until an active relay (one that can dial on behalf of other peers), or a relay that may have recently acquired a connection to the destination peer is successful. 
-            - If successful
-                - Write the `/ipfs/relay/circuit/1.0.0` header to the relay, followed by the destination address
-                    -  e.g. `/ipfs/relay/circuit/1.0.0\n/p2p-circuit/ipfs/QmDest`.
-            - If no relays could connect, fail the same way a regular transport would
-    - Once the connection has been established, the swarm should treat it as a regular connection, 
-        - i.e. muxing, encrypt, etc should all be performed on the relayed connection
-
-- ### Listen for relayed connections
-    - Peer mounts the `/ipfs/relay/circuit/1.0.0` proto and listens for relayed connections
-        - A connection arrives
-            - read the address of the source peer from the incoming connection stream
-                - if valid, create a PeerInfo object for that peer and add the incoming address to its multiaddresses list
-                - pass the connection to `protocolMuxer(swarm.protocols, conn)` to have it go through the regular muxing/encryption flow
-                
-- ### Relay discovery and static relay addresses in swarm config
-
-    - #### Relay address in swarm config
-       - A peer has relay addresses in its swarm config section
-           - On node startup, connect to the relays in swarm config
-                - if successful add address to swarms PeerInfo's multiaddresses
-                    - `identify` should take care of announcing that the peer is reachable over the listed relays
-
-    - #### Passive relay discovery
-        - A peer that can dial over `/ipfs/relay/circuit/1.0.0` listens for the `peer-mux-established` swarm event, every time a new muxed connection arrives, it checks if the incoming peer is a relay. (How would this work? Some way of discovering if its a relay is required.)
-           - *Useful in cases when the peer/node doesn't know of any relays on startup and also, to learn of as many additional relays in the network as possible*
-           - *Useful during startup, when connecting to bootstrap nodes. It allows us to implicitly learn if its a relay without having to explicitly add `/p2p-circuit` addresses to the bootstrap list*
-           - *Also useful if the relay communicates its capabilities upon connecting to it, as to avoid additional unnecessary requests/queries. I.e. if it supports weather its able to forward connections and weather it supports the `ls` or other commands.*
-           - *Should it be possible to disable passive relay discovery?*
-               - This could be useful when the peer wants to be reachable **only** over the listed relays
-       - If the incoming peer is a relay, send an `ls` and record its peers
-
-## Relay Nodes
-
-- ### Passive relay node
-    - *A passive relay does not explicitly dial into any requested peer, only those that it's swarm already has connections to.*
-        - When the relay gets a request, read the the destination peer's multiaddr from the connection stream and if its a valid address and peer id
-            - check its swarm's peerbook(?) see if its a known peer, if it is
-                - use the swarms existing connection and
-                    - send the multistream header and the source peer address to the dest peer
-                        - e.g. `/ipfs/relay/circuit/1.0.0\n/p2p-circuit/ipfs/QmSource`
-                    - circuit the source and dest connections
-                - if couldn't dial, or the connection/stream to the dest peer closed prematurelly
-                    - close the src stream
-
-
-- ### Active relay node
-    - *An active relay node can dial other peers even if its swarm doesnt know about those peers*
-        - When the relay gets a request, read the the destination peer's multiaddr from the connection stream and if its a valid address and peer id
-            - use the swarm to dial to the dest node
-                - send the multistream header and the source peer address to the dest peer
-                    - e.g. `/ipfs/relay/circuit/1.0.0\n/p2p-circuit/ipfs/QmSource`
-                - circuit the source and dest connections
-            - if couldn't dial, or the connection/stream to the dest peer closed prematurely
-                - close the src stream
-
-- ### `ls` command
-    - *A relay node can allow the peers known to it's swarm to be listed*
-        - *this should be possible to enable/disable from the config*
-            - when a relay gets the `ls` request
-                - if enabled, get its swarm's peerbook's known peers and return their ids and multiaddrs
-                    - e.g `[{id: /ipfs/QmPeerId, addrs: ['ma1', 'ma2', 'ma3']}, ...]`
-                - if disabled, respond with `na`
-
-
-## Relay Implementation notes
-
-- ### Relay transport
-    - Currently I've implemented the dialer and listener parts of the relay as a transport, meaning that it *tries* to implement the `interface-transport` interface as closely as possible. This seems to work pretty well and it's makes the dialer/listener parts really easy to plug in into the swarm. I think this is the cleanest solution.
-
-- ### `circuit-relay`
-    - This is implemented as a separate piece (not a transport), and it can be enabled/disabled with a config. The transport listener however, will do the initial parsing of the incoming header and figure out weather it's a connection that's needs to be handled by the circuit-relay, or its a connection that is being relayed from a circuit-relay.
-
-## Relay swarm integration
-
-- The relay transport is mounted explicitly by calling the `swarm.connection.relay(config.relay)` from libp2p
-    - Swarm will register the dialer and listener using the swarm `transport.add` and `transport.listen` methods
-
-    - ### Listener 
-        - the listener registers itself as a multistream handler on the `/ipfs/relay/circuit/1.0.0` proto
-            - if `circuit-relay` is enabled, the listener will delegate connections to it if appropriate
-        - when the listener receives a connection, it will read the multiaddr and determine if its a connection that needs to be relayed, or its a connection that is being relayed
-
-    - ### Dialer
-        - When the swarm attempts to dial to a peer, it will filter the protocols that the peer can be reached on
-            - *The relay will be used in two cases*
-            - If the peer has an explicit relay address that it can be reached on
-                -  no other transport is available
-                    - The relay will attempt to dial the peer over that relay
-            - If no explicit relay address is provided
-                -  no other transport is available
-                    -  A generic circuit address will be added to the peers multiaddr list
-                        - i.e. `/p2p-circuit/ipfs/QmDest`
-            - If another transport is available, then use that instead of the relay
-    
-

package/src/circuit/README.md

@@ -1,111 +0,0 @@
-# js-libp2p-circuit
-
-> Node.js implementation of the Circuit module that libp2p uses, which implements the [interface-connection](https://github.com/libp2p/js-libp2p-interfaces/tree/master/src/connection) interface for dial/listen.
-
-**Note**: git history prior to merging into js-libp2p can be found in the original repository, https://github.com/libp2p/js-libp2p-circuit.
-
-`libp2p-circuit` implements the circuit-relay mechanism that allows nodes that don't speak the same protocol to communicate using a third _relay_ node. You can read more about this in its [spec](https://github.com/libp2p/specs/tree/master/relay).
-
-## Table of Contents
-
-- [js-libp2p-circuit](#js-libp2p-circuit)
-  - [Table of Contents](#table-of-contents)
-    - [Why?](#why)
-    - [libp2p-circuit and IPFS](#libp2p-circuit-and-ipfs)
-  - [Usage](#usage)
-  - [API](#api)
-    - [Implementation rational](#implementation-rational)
-
-### Why?
-
-`circuit-relaying` uses additional nodes in order to transfer traffic between two otherwise unreachable nodes. This allows nodes that don't speak the same protocols or are running in limited environments, e.g. browsers and IoT devices, to communicate, which would otherwise be impossible given the fact that for example browsers don't have any socket support and as such cannot be directly dialed.
-
-The use of circuit-relaying is not limited to routing traffic between browser nodes, other uses include:
- - routing traffic between private nets and circumventing NAT layers
- - route mangling for better privacy (matreshka/shallot dialing).
-
-It's also possible to use it for clients that implement exotic transports such as  devices that only have bluetooth radios to be reachable over bluetooth enabled relays and become full p2p nodes.
-
-### libp2p-circuit and IPFS
-
-Prior to `libp2p-circuit` there was a rift in the IPFS network, were IPFS nodes could only access content from nodes that speak the same protocol, for example TCP only nodes could only dial to other TCP only nodes, same for any other protocol combination. In practice, this limitation was most visible in JS-IPFS browser nodes, since they can only dial out but not be dialed in over WebRTC or WebSockets, hence any content that the browser node held was not reachable by the rest of the network even through it was announced on the DHT. Non browser IPFS nodes would usually speak more than one protocol such as TCP, WebSockets and/or WebRTC, this made the problem less severe outside of the browser. `libp2p-circuit` solves this problem completely, as long as there are `relay nodes` capable of routing traffic between those nodes their content should be available to the rest of the IPFS network.
-
-## Usage
-
-Libp2p circuit configuration can be seen at [Setup with Relay](../../doc/CONFIGURATION.md#setup-with-relay).
-
-Once you have a circuit relay node running, you can configure other nodes to use it as a relay as follows:
-
-```js
-import { multiaddr } from '@multiformats/multiaddr'
-import Libp2p from 'libp2p'
-import { tcp } from '@libp2p/tcp'
-import { mplex } from '@libp2p/mplex'
-import { noise } from '@chainsafe/libp2p-noise'
-
-const relayAddr = ...
-
-const node = await createLibp2p({
-  addresses: {
-    listen: [multiaddr(`${relayAddr}/p2p-circuit`)]
-  },
-  transports: [
-    tcp()
-  ],
-  streamMuxers: [
-    mplex()
-    ],
-  connectionEncryption: [
-    noise()
-  ]
-  },
-  config: {
-    relay: {                   // Circuit Relay options (this config is part of libp2p core configurations)
-      enabled: true           // Allows you to dial and accept relayed connections. Does not make you a relay.
-    }
-  }
-})
-```
-
-## API
-
-[![](https://raw.githubusercontent.com/libp2p/interface-transport/master/img/badge.png)](https://github.com/libp2p/interface-transport)
-
-`libp2p-circuit` accepts Circuit addresses for both IPFS and non IPFS encapsulated addresses, i.e:
-
-`/p2p-circuit/ip4/127.0.0.1/tcp/4001/p2p/QmHash`
-
-Both for dialing and listening.
-
-### Implementation rational
-
-This module is not a transport, however it implements `interface-transport` interface in order to allow circuit to be plugged with `libp2p`. The rational behind it is that, `libp2p-circuit` has a dial and listen flow, which fits nicely with other transports, moreover, it requires the _raw_ connection to be encrypted and muxed just as a regular transport's connection does. All in all, `interface-transport` ended up being the correct level of abstraction for circuit, as well as allowed us to reuse existing integration points in `libp2p` and `libp2p` without adding any ad-hoc logic. All parts of `interface-transport` are used, including `.getAddr` which returns a list of `/p2p-circuit` addresses that circuit is currently listening.
-
-```
-libp2p                                                                                  libp2p-circuit (transport)
-+-------------------------------------------------+                                     +--------------------------+
-|        +---------------------------------+      |                                     |                          |
-|        |                                 |      |                                     |   +------------------+   |
-|        |                                 |      |  circuit-relay listens for the HOP  |   |                  |   |
-|        |              libp2p             <------------------------------------------------|  circuit-relay   |   |
-|        |                                 |      |  message to handle incomming relay  |   |                  |   |
-|        |                                 |      |  requests from other nodes          |   +------------------+   |
-|        +---------------------------------+      |                                     |                          |
-|         ^     ^   ^  ^   ^           ^          |                                     |   +------------------+   |
-|         |     |   |  |   |           |          |                                     |   | +-------------+  |   |
-|         |     |   |  |   |           |          |      dialer uses libp2p to dial     |   | |             |  |   |
-|         |     |   |  +---------------------------------------------------------------------->   dialer    |  |   |
-|         |     | transports           |          |  to a circuit-relay node using the  |   | |             |  |   |
-|         |     |   |      |           |          |  HOP message                        |   | +-------------+  |   |
-|         |     |   |      |           |          |                                     |   |                  |   |
-|         v     v   |      v           v          |                                     |   |                  |   |
-|+------------------|----------------------------+|                                     |   |  +-------------+ |   |
-||           |      |    |      |                ||                                     |   |  |             | |   |
-||libp2p-tcp |libp2p-ws  | .... |libp2p-circuit  ||  listener handles STOP messages from|   |  | listener    | |   |
-||           |      +-------------------------------------------------------------------------->             | |   |
-||           |           |      |plugs in just   ||  circuit-relay nodes                |   |  +-------------+ |   |
-||           |           |      |as any other    ||                                     |   |                  |   |
-||           |           |      |transport       ||                                     |   +------------------+   |
-|+-----------------------------------------------+|                                     |                          |
-+-------------------------------------------------+                                     +--------------------------+
-```

package/src/circuit/auto-relay.ts

@@ -1,292 +0,0 @@
-import { logger } from '@libp2p/logger'
-import { fromString as uint8ArrayFromString } from 'uint8arrays/from-string'
-import { toString as uint8ArrayToString } from 'uint8arrays/to-string'
-import { RELAY_CODEC } from './multicodec.js'
-import { canHop } from './circuit/hop.js'
-import { namespaceToCid } from './utils.js'
-import {
-  CIRCUIT_PROTO_CODE,
-  HOP_METADATA_KEY,
-  HOP_METADATA_VALUE,
-  RELAY_RENDEZVOUS_NS
-} from './constants.js'
-import type { PeerId } from '@libp2p/interface-peer-id'
-import type { AddressSorter, PeerProtocolsChangeData } from '@libp2p/interface-peer-store'
-import type { Connection } from '@libp2p/interface-connection'
-import sort from 'it-sort'
-import all from 'it-all'
-import { pipe } from 'it-pipe'
-import { publicAddressesFirst } from '@libp2p/utils/address-sort'
-import type { RelayComponents } from './index.js'
-
-const log = logger('libp2p:auto-relay')
-
-const noop = () => {}
-
-export interface AutoRelayInit {
-  addressSorter?: AddressSorter
-  maxListeners?: number
-  onError?: (error: Error, msg?: string) => void
-}
-
-export class AutoRelay {
-  private readonly components: RelayComponents
-  private readonly addressSorter: AddressSorter
-  private readonly maxListeners: number
-  private readonly listenRelays: Set<string>
-  private readonly onError: (error: Error, msg?: string) => void
-
-  constructor (components: RelayComponents, init: AutoRelayInit) {
-    this.components = components
-    this.addressSorter = init.addressSorter ?? publicAddressesFirst
-    this.maxListeners = init.maxListeners ?? 1
-    this.listenRelays = new Set()
-    this.onError = init.onError ?? noop
-
-    this._onProtocolChange = this._onProtocolChange.bind(this)
-    this._onPeerDisconnected = this._onPeerDisconnected.bind(this)
-
-    this.components.peerStore.addEventListener('change:protocols', (evt) => {
-      void this._onProtocolChange(evt).catch(err => {
-        log.error(err)
-      })
-    })
-    this.components.connectionManager.addEventListener('peer:disconnect', this._onPeerDisconnected)
-  }
-
-  /**
-   * Check if a peer supports the relay protocol.
-   * If the protocol is not supported, check if it was supported before and remove it as a listen relay.
-   * If the protocol is supported, check if the peer supports **HOP** and add it as a listener if
-   * inside the threshold.
-   */
-  async _onProtocolChange (evt: CustomEvent<PeerProtocolsChangeData>) {
-    const {
-      peerId,
-      protocols
-    } = evt.detail
-    const id = peerId.toString()
-
-    // Check if it has the protocol
-    const hasProtocol = protocols.find(protocol => protocol === RELAY_CODEC)
-
-    // If no protocol, check if we were keeping the peer before as a listenRelay
-    if (hasProtocol == null) {
-      if (this.listenRelays.has(id)) {
-        await this._removeListenRelay(id)
-      }
-
-      return
-    }
-
-    if (this.listenRelays.has(id)) {
-      return
-    }
-
-    // If protocol, check if can hop, store info in the metadataBook and listen on it
-    try {
-      const connections = this.components.connectionManager.getConnections(peerId)
-
-      if (connections.length === 0) {
-        return
-      }
-
-      const connection = connections[0]
-
-      // Do not hop on a relayed connection
-      if (connection.remoteAddr.protoCodes().includes(CIRCUIT_PROTO_CODE)) {
-        log(`relayed connection to ${id} will not be used to hop on`)
-        return
-      }
-
-      const supportsHop = await canHop({ connection })
-
-      if (supportsHop) {
-        await this.components.peerStore.metadataBook.setValue(peerId, HOP_METADATA_KEY, uint8ArrayFromString(HOP_METADATA_VALUE))
-        await this._addListenRelay(connection, id)
-      }
-    } catch (err: any) {
-      this.onError(err)
-    }
-  }
-
-  /**
-   * Peer disconnects
-   */
-  _onPeerDisconnected (evt: CustomEvent<Connection>) {
-    const connection = evt.detail
-    const peerId = connection.remotePeer
-    const id = peerId.toString()
-
-    // Not listening on this relay
-    if (!this.listenRelays.has(id)) {
-      return
-    }
-
-    this._removeListenRelay(id).catch(err => {
-      log.error(err)
-    })
-  }
-
-  /**
-   * Attempt to listen on the given relay connection
-   */
-  async _addListenRelay (connection: Connection, id: string): Promise<void> {
-    try {
-      // Check if already listening on enough relays
-      if (this.listenRelays.size >= this.maxListeners) {
-        return
-      }
-
-      // Get peer known addresses and sort them with public addresses first
-      const remoteAddrs = await pipe(
-        await this.components.peerStore.addressBook.get(connection.remotePeer),
-        (source) => sort(source, this.addressSorter),
-        async (source) => await all(source)
-      )
-
-      // Attempt to listen on relay
-      const result = await Promise.all(
-        remoteAddrs.map(async addr => {
-          try {
-            let multiaddr = addr.multiaddr
-
-            if (multiaddr.getPeerId() == null) {
-              multiaddr = multiaddr.encapsulate(`/p2p/${connection.remotePeer.toString()}`)
-            }
-
-            multiaddr = multiaddr.encapsulate('/p2p-circuit')
-
-            // Announce multiaddrs will update on listen success by TransportManager event being triggered
-            await this.components.transportManager.listen([multiaddr])
-            return true
-          } catch (err: any) {
-            log.error('error listening on circuit address', err)
-            this.onError(err)
-          }
-
-          return false
-        })
-      )
-
-      if (result.includes(true)) {
-        this.listenRelays.add(id)
-      }
-    } catch (err: any) {
-      this.onError(err)
-      this.listenRelays.delete(id)
-    }
-  }
-
-  /**
-   * Remove listen relay
-   */
-  async _removeListenRelay (id: string) {
-    if (this.listenRelays.delete(id)) {
-      // TODO: this should be responsibility of the connMgr
-      await this._listenOnAvailableHopRelays([id])
-    }
-  }
-
-  /**
-   * Try to listen on available hop relay connections.
-   * The following order will happen while we do not have enough relays.
-   * 1. Check the metadata store for known relays, try to listen on the ones we are already connected.
-   * 2. Dial and try to listen on the peers we know that support hop but are not connected.
-   * 3. Search the network.
-   */
-  async _listenOnAvailableHopRelays (peersToIgnore: string[] = []) {
-    // TODO: The peer redial issue on disconnect should be handled by connection gating
-    // Check if already listening on enough relays
-    if (this.listenRelays.size >= this.maxListeners) {
-      return
-    }
-
-    const knownHopsToDial = []
-    const peers = await this.components.peerStore.all()
-
-    // Check if we have known hop peers to use and attempt to listen on the already connected
-    for (const { id, metadata } of peers) {
-      const idStr = id.toString()
-
-      // Continue to next if listening on this or peer to ignore
-      if (this.listenRelays.has(idStr)) {
-        continue
-      }
-
-      if (peersToIgnore.includes(idStr)) {
-        continue
-      }
-
-      const supportsHop = metadata.get(HOP_METADATA_KEY)
-
-      // Continue to next if it does not support Hop
-      if ((supportsHop == null) || uint8ArrayToString(supportsHop) !== HOP_METADATA_VALUE) {
-        continue
-      }
-
-      const connections = this.components.connectionManager.getConnections(id)
-
-      // If not connected, store for possible later use.
-      if (connections.length === 0) {
-        knownHopsToDial.push(id)
-        continue
-      }
-
-      await this._addListenRelay(connections[0], idStr)
-
-      // Check if already listening on enough relays
-      if (this.listenRelays.size >= this.maxListeners) {
-        return
-      }
-    }
-
-    // Try to listen on known peers that are not connected
-    for (const peerId of knownHopsToDial) {
-      await this._tryToListenOnRelay(peerId)
-
-      // Check if already listening on enough relays
-      if (this.listenRelays.size >= this.maxListeners) {
-        return
-      }
-    }
-
-    // Try to find relays to hop on the network
-    try {
-      const cid = await namespaceToCid(RELAY_RENDEZVOUS_NS)
-      for await (const provider of this.components.contentRouting.findProviders(cid)) {
-        if (provider.multiaddrs.length === 0) {
-          continue
-        }
-
-        const peerId = provider.id
-
-        if (peerId.equals(this.components.peerId)) {
-          // Skip the provider if it's us as dialing will fail
-          continue
-        }
-
-        await this.components.peerStore.addressBook.add(peerId, provider.multiaddrs)
-
-        await this._tryToListenOnRelay(peerId)
-
-        // Check if already listening on enough relays
-        if (this.listenRelays.size >= this.maxListeners) {
-          return
-        }
-      }
-    } catch (err: any) {
-      this.onError(err)
-    }
-  }
-
-  async _tryToListenOnRelay (peerId: PeerId) {
-    try {
-      const connection = await this.components.connectionManager.openConnection(peerId)
-      await this._addListenRelay(connection, peerId.toString())
-    } catch (err: any) {
-      log.error('Could not use %p as relay', peerId, err)
-      this.onError(err, `could not connect and listen on known hop relay ${peerId.toString()}`)
-    }
-  }
-}