Libp2p connect to nodes with pubsub enabled

Hi,

I’ve been at this before but i’m taking a bit of a different approach this time. For those who remember it, when i asked about this before i was trying to get IPFS’s PubSub mechanism working on websites which brought me the webrtc-star nightmare and all together just not a concept that seemed stable enough to use.

The approach i’m taking now is simply making a socket.io proxy endpoint nodes to which users can connect. You can find the code for that here. That however is still based on a full IPFS node running. For cheap $5 servers that’s a bit taxing. So i’m trying to accomplish the same with only libp2p.

As a “simple” proof of concept if i can receive pubsub data i try the following code (99% copy/paste from examples, please do tell me if i do something totally wrong):

'use strict'

const Libp2p = require('libp2p')
const TCP = require('libp2p-tcp')
const Mplex = require('libp2p-mplex')
const { NOISE } = require('libp2p-noise')
const Bootstrap = require('libp2p-bootstrap')
const DHT = require('libp2p-kad-dht')
const Gossipsub = require('libp2p-gossipsub')
const uint8ArrayFromString = require('uint8arrays/from-string')
const uint8ArrayToString = require('uint8arrays/to-string')

const bootstrapers = [
    '/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN',
    '/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb',
    '/dnsaddr/bootstrap.libp2p.io/p2p/QmZa1sAxajnQjVM8WjWXoMbmPd7NsWhfKsPkErzpm9wGkp',
    '/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa',
    '/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt'
  ]

;(async () => {
  const node = await Libp2p.create({
    addresses: {
      listen: ['/ip4/0.0.0.0/tcp/0']
    },
    modules: {
      transport: [TCP],
      streamMuxer: [Mplex],
      connEncryption: [NOISE],
      peerDiscovery: [Bootstrap],
      dht: DHT,
      pubsub: Gossipsub
    },
    config: {
      peerDiscovery: {
        bootstrap: {
          interval: 60e3,
          enabled: true,
          list: bootstrapers,
          autoDial: true
        }
      },
      dht: {                        // The DHT options (and defaults) can be found in its documentation
        kBucketSize: 20,
        enabled: true,
        randomWalk: {
          enabled: true,            // Allows to disable discovery (enabled by default)
        }
      }
    }
  })

  const topic = 'news'
  await node.start()
  node.pubsub.on(topic, (msg) => {
    console.log(`node received: ${uint8ArrayToString(msg.data)}`)
  })
  await node.pubsub.subscribe(topic)
})();

The annoying thing here is that this works, but not immediately.
It’s my suspicion that the bootstrap nodes don’t have pubsub enabled.
With DHT i’m getting a lot more node connections. After, say, 2 minutes or so i finally have a connection to a node with pubsub enabled after which the above code receives pubsub messages just fine.

So this works :slight_smile: That’s a good thing!

I’m wondering if i can somehow instruct this code to make a connection to pubsub enabled nodes and drop the other connections?

What i did try is add a bootstrap node with pubsub enabled and that worked too.

I also tried adding the bootstrap nodes from IPFS, thinking they “surely” had pubsub enabled… Turns out that apparently isn’t the case either. That does kinda make me wonder how enabled pubsub is on the IPFS network if their own bootstrap nodes apparently don’t have it enabled.

Lastly, how can i dial a node by only it’s CID? I have a few nodes where i know pubsub is running, but i don’t want to add them in the bootstrap list because their IP is dynamic and could therefore change.

Cheers,
Mark

Edit.
In fact, i want to spin up half a dozen of these IPFS nodes handling pubsub. How can i make them connect to each other without specifying them in the bootstrap? Would they find each other via DHT in the above code? Also, which ports do i need to open for that? For IPFS that’s 4001, is that the same for libp2p?

Edit 2.
I do notice each instance of the above code is creating a new peerid. Probably because there isn’t a keychain? Is there an example somewhere to setup libp2p with keychain?

cc ping @vasco-santos :slight_smile: (as you probably know a load more about all of this)

Hey @markg85
I could not go through your code yet. But I will answer the questions for now to try to unblock you.

Every instance is a new peer unless you create a peerId yourself and save it. You can use peer-id CLI to generate a peer and save it as a File and then create the PeerId programatically by loading the file and provide it in the libp2p constructor.

I’m wondering if i can somehow instruct this code to make a connection to pubsub enabled nodes and drop the other connections?

The solution I see here is to iterate to the PeerStore peers and see what peers have pubsub protocol. Then go through the connection manager connections and close all of them except to the peer with pubsub. You can see the API.md to get all these methods. Another more reactive solution would be to listen on changes in the protobook and react when you a peer with pubsub protocol appears.

We want in the future to allow more configuration on the peers you want to be connected to, which will ease this flow. But for now, this needs to be handled in the application layer.

Yes they should not have. go-ipfs does not have pubsub enabled by default. You can check their protocols by checking the protobook content after establishing the connections.
Pubsub at this moment, until we enable it by default is more suitable for specific overlay networks. This means, you would setup your own bootstrap nodes with pubsub enabled to boost your network, where your application nodes would use them to get to know other peers.

You can use findPeer method where you provide the peerId and receive their multiaddrs, which you can then use to connect to them.

We don’t have an out of the box solution here. In the application layer, you will need to rely on the DHT/Delegates and use provide/findProviders. The logic would be, you create a given namespace, create a CID from it and every time a node starts it will provide that CID. When further nodes start, they will also provide the CID and findProviders of that CID. With that, it will obtain the multiaddrs to connect to the other nodes.


As a general comment, bear in mind that the DHT in JS is highly experimental and still not reliable enough. We have been recommending people to use the Delegate Routers (which js-ipfs also uses). Feel free to experiment with the DHT if you like, but you might expect a few gaps there.

Hope these comments help you

2 Likes

Hi @vasco-santos,

That helps a lot! Thank you for that detailed reply!

Based on your last remark:

I take it that i’m better off by not using DHT and instead just use the bootstrap list + dialing a few peer id’s i know to be having pubsub enabled?

Regarding pubsub and the protocol array. It confuses me a little as i don’t see what i’m expecting. Fir is an example protocol array from a node with pubsub enabled (gossipsub):

[
  '/p2p/id/delta/1.0.0',
  '/ipfs/id/1.0.0',
  '/ipfs/id/push/1.0.0',
  '/ipfs/ping/1.0.0',
  '/libp2p/circuit/relay/0.1.0',
  '/ipfs/lan/kad/1.0.0',
  '/libp2p/autonat/1.0.0',
  '/meshsub/1.1.0',
  '/meshsub/1.0.0',
  '/floodsub/1.0.0',
  '/libp2p/fetch/0.0.1',
  '/ipfs/bitswap/1.2.0',
  '/ipfs/bitswap/1.1.0',
  '/ipfs/bitswap/1.0.0',
  '/ipfs/bitswap',
  '/x/',
  '/ipfs/kad/1.0.0'
]

The PubSub spec mentions gossipsub and floodsub but doesn’t mention how they would be exposed as a protocol. That’s the first point of confusion.

Then the array mentions “meshsub” a couple of times and “floodsub”.
The node this array is coming from is started with: “–enable-pubsub-experiment --enable-namesys-pubsub” and the node’s config does specify gossipsub:

  "Pubsub": {
    "DisableSigning": false,
    "Router": "gossipsub"
  },

This gives me many more quiestions…

  1. Just to confirm. What is the proper way to start a go-ipfs instance with gossipsub enabled?
  2. Is meshsub gossipsub? If that’s the case, why is it called like that? If that’s not the case, where is gossipsub?
  3. How do i only enable gossipsub. I see that as a superior protocol to floodsub so i want to disable floodsub.
  4. How do i start a libp2p node with gossipsub enabled?

Thank you again for all your help so far!

Edit 1.
Looks like “meshsup” is indeed gossipsup according to this. It clarifies the protocols for me but i’m still curious why there are seemingly 2 names for the same thing.

Yes, but that will also limit your options to find peers with pubsub enabled, or peers you care about. If you can start with that solution it will be helpful. But you can also use the delegates. This basically relies on other nodes to do DHT queries on your behalf. These are go-libp2p DHT servers.

You are right, the spec should mention the protocols! Can you PR or create an issue for this in the libp2p specs repo? Your assumption is correct, /meshsub/ is gossipsub. Regarding all the versions here related, gossipsub 1.0 is '/meshsub/1.0.0' and gossipsub 1.1 is '/meshsub/1.1.0. Looking at the spec, gossipsub 1.1 is backwards compatible, which means it can do pubsub things with peers still running gossipsub 1.0. This is the reason to have both protocols. You can see libp2p protocols as a “decentralized rest api”. Gossipsub 1.1 nodes will listen on both routes 1.0 and 1.1 and according to the other party capabilities will do different stuff. The floodsub protocol is essentially the same, you can read more here. In js gossipsub, we have an option to fallback to floodsub, which is the default behaviour.

You are doing it as expected :slight_smile: