File replication in IPFS - how to control?

Hi,
In IPFS Cluster, or in IPFS in general, how do we control or to know the number of copies for a file? For example, can we explicitly say that every file will always have 2 copies, but not more?

Thanks.

In cluster, you can specify a replication factor: https://cluster.ipfs.io/documentation/configuration/#the-cluster-main-section

Thanks! What does replication_factor_min mean? For example if it’s set to 2, does it mean we cannot store a file if there is only node online at the moment?

and, is there a way to find out the replication_factor for a particular file, if the min and the max are different?

If you call ipfs-cluster-ctl add FILE, cluster will wait to replicate. If you add the file with IPFS directly and then call ipfs-cluster-ctl pin add CID, this won’t wait.

However, I’m generally not sure how cluster will behave if you have fewer nodes than your replication factor.

ipfs-cluster-ctl status CID

1 Like

Hi,

I will make a tl;dr; Things are a bit elaborated here: https://cluster.ipfs.io/documentation/internals/#pinning-an-item and https://cluster.ipfs.io/documentation/internals/#adding-an-item (obviously, we need to fix the docs to explain this more clearly in a more concise, accessible place <- @pkafei).

Cluster will try to put your pins in replication_factor_max ipfs peers. But as long as there are more than replication_factor_min peers things will work (and no error will be returned). In a peer alert event (when a peer has gone away), no re-allocations of the pin will happen if it is still above it’s minimum replication factor. This allows peers to go away and come back as much as the replication factor allows without the cluster starting to replicate data to some other place when “repinnings” are enabled.

The best way to get the replication factor (and any “pin option”) in a pin is with ipfs-cluster-ctl pin ls $cid). status will query/fetch the actual status from every peer, so you can see if pinning failed. There is more info about this here (it has to do with the shared vs. local state): https://cluster.ipfs.io/documentation/internals/#the-shared-state-the-local-state-and-the-ipfs-state

1 Like