IPFS Cluster & public IPFS nodes combination

Hi all,

I am in the progress of learning and researching on IPFS for our next project which is an NFT marketplace.

I have learned that IPFS cluster setup will be a good choice because of the following characteristics of IPFS cluster

  1. Private network will increase the performance (source: Dedicated IPFS Networks. Scaling Through Network Specialization | by Matt Ober | Pinata | Medium)
  2. Easy to manage the pinning behavior through the cluster API

However, since the setup is a private network, so we will lose the decentralization of IPFS which means we can no longer access the content using public gateways, says ipfs.io. Besides, imo, letting the users access our platform’s NFT content through public gateways is also a good point to increase transparency and reliability.

So, I am wondering whether we can achieve the benefits of private networks while still a (controllable, manageable) way to let our content accessible from public network.

Actually I have been thinking of a setup but please help me to verify if is correct or not, in terms of security, feasibility and if the idea going against the nature of IPFS, IPFS cluster.
My thought is that we will maintain a IPFS cluster, which will be connected directly our NFT platform backend to store and pin content, along with a standalone public IPFS node acting as a gateway to outside world. Such IPFS node:

  • is the only public node that can ready data from our IPFS nodes inside the cluster (please verify if this can be configured)
  • only be used to read data, a read-only node.

Forgive me if anything is incorrect. I am quite new to IPFS.

Really appreciate any comment, suggestion, advice and clarification.
Thanks in advance.

You seem to be mixing two things:

  • The “gateway” endpoint (port 8080) that an IPFS node offers can be published in any way you want. It is by default a read-only endpoint. It can be provided by a peer that is part of your cluster (or not), but must be part of the IPFS private network. It provides HTTP access to the content.

  • The “p2p” endpoint (port 4001) that an IPFS node offers would be “encrypted” (part of the private network). It is not possible to setup an IPFS node that is both in a private network and offers a public-network p2p endpoint. Thus, people would not be able to download things for your private network using the IPFS protocol even if you made the endpoint publicly accessible (unless you give your private network key, at which point is not a truly private network anymore).

Hi @hector ,

Thanks for your clarification.

So, we cannot have an IPFS cluster with public access, right?

In short, I am looking for a solution that I can setup an IPFS cluster, which connects directly to our NFT platform to add, pin, read files, and people from outside that cluster can still view our content using public gateways such as ipfs.io.

Hi Hector,

Here is the update. Turn out the content pinned to my private cluster (given that I setup a secret key) can still be accessible through public gateways (I tried ipfs.io/ipfs and it worked).

This mean public nodes can still download content from my nodes but they cannot add & pin contents unless:

  • I share them the cluster secret
  • or, expose the port 5001 of my IPFS nodes (managed by clusters peers) for them to call /api/v0/pin/add API

If possible, please let me know your feedbacks and thoughts. Really appreciate.

The ipfs nodes and the IPFS-Cluster nodes are 2 different things. IPFS-Cluster has it’s own private network and is there to tell IPFS nodes what to pin.

IPFS nodes can be in the public network or form a private network of their own. If they are configured to form a private network, the content they have will not be accessible from the public gateways.

IPFS-Cluster does not play a role in how content is made available, it only serves to pin and track pins in multiple places.

1 Like

Thanks for the clarification Hector.