State of pubsub

Hi! At this point, we feel pretty confident in recommending people use Pubsub—it should be stable today and we plan to try and make upcoming improvements backwards compatible. (If you’re actively experiencing serious issues, please file them at go-floodsub/issues or js-libp2p-floodsub/issues!)

In particular, the next major improvement, Gossipsub (https://github.com/libp2p/go-floodsub/pull/67), will be backwards compatible, so current Floodsub peers will be able to communicate with Gossipsub peers. If you start using the current pubsub implementation today, you should be able to slowly and progressively upgrade all your peers/nodes/devices without needing to coordinate anything special. (Gossipsub is also actually a part of the go-floodsub library, not a separate module or package you’ll have to use. It’ll also still be included and accessible within go-ipfs and js-ipfs.) We hope to release for Go in the next few months, then we’ll start working on the JS version (in the mean time, the backwards-compatibility should keep things working).

Gossipsub aims primarily to make pubsub work better in large networks (roughly 10,000-100,000 nodes) and adds the ability for nodes to relay messages, so nodes with fewer connections can still participate in a topic.

After Gossipsub, we’re turning our attention to optimizing performance when there are a small or fixed number of publishers/sources for messages. This’ll really help use cases like streaming live video or individual sensors that stream their data to the world via pubsub, but doesn’t add anything for cases where there are lots of publishers and subscribers, like a blockchain.

In a more general scope, we want to focus on keeping pubsub low-level and simple, with a focus on performance, decentralization, and scale. That means we don’t plan on doing things like long-lived queues (we are adding small buffers to improve reliability), guaranteed ordering, and so on. If you need those features, you can still build them on top of libp2p/IPFS pubsub, but we don’t plan to directly support them. (If you’re working on a higher-level library that adds these sorts of things, we’d love to hear about it!)

8 Likes