IPFS Cluster How-to: please review!

Help IPFS Cluster
1

I found the documentation for IPFS Cluster to be inadequate for someone of my (low) skill level, so once I got a cluster working, I decided to write up what I did, in case it could help others.

I would appreciate it if anyone with more knowledge of IPFS Cluster could review and critique what I have written before I make it public. Of course, I have pinned it to my own cluster. You can find it here:

https://gateway.ipfs.io/ipfs/QmcXuNQsPqXALRVUAnTf7B1N5dAJKU1uRPrCHvpoEBeW8V/

It is not pretty yet, just some basic styles to make it readable. I am more concerned about whether it is accurate at this point. Thank you!

2 Likes
2

This is really nice!

My comments:

  • When installing ipfs-update you could save the step of installing go-ipfs separately first and do it directly from ipfs-update (i think)
  • Regarding ctl: it will allow you to access cluster statistics from the command line. I’m not sure what that means. I think you mean interacting with your cluster daemon from the command line ?
  • For security considerations mostly, we have the convention of running go-ipfs and ipfs-cluster-service using an ipfs system user, rather than root. That means, the init commands should be run by this user and the configuration etc. are placed in ipfs user home. Also, in cases where your ipfs storage is an additional large disk attached to your machine, this allows you to mount the ipfs home directly on that disk.
  • The ipfs-cluster-service.service file will be very different ? Is this the case? I don’t see differences.
  • It is also important to follow things mentioned here: https://cluster.ipfs.io/documentation/deployment/ or at least be familiar with the configurations options mentioned for both ipfs and cluster.

Would you mind if we include a link to this from the IPFS Cluster website once you have a final version?

3

Thank you! I will make these changes. Of course I’d be happy to have a link to it, once it is correct. :grinning:

4

I have updated it. The new version is here:

https://gateway.ipfs.io/ipfs/QmTqD45V5EFF9DvCpr39vyJDE5m1PJYdhWSjqNdYLAgA37/

I would appreciate any further review. Thank you!

5

I’ve corrected some small typos for the next version, so don’t worry too much about mistakes you find like incorrect capitalization. I am more concerned with technical accuracy. Thanks!

6

Update:

https://gateway.ipfs.io/ipfs/QmeVDPHgN6JMTTiJrsxhC3PEEBvcXJsLPZwwfMnwWxb6YW/

That should be the last version, unless there is an error. Just cleaned up some of the text and images.

Any feedback would be very helpful!

1 Like
7

Thanks for your document. I was doing kind of the same configuration… I think

  • You should use ipfs specific user instead of root to run ipfs and ipfs-cluster daemons

useradd -r -s /usr/bin/nologin ipfs

[Unit]
Description=IPFS daemon
After=network.target

[Service]
User=ipfs
Environment="IPFS_PATH=/data/ipfs"
ExecStart=/usr/bin/ipfs daemon --mount --enable-gc --routing=dhtclient
Restart=on-failure

[Install]
WantedBy=multi-user.target

  • Restrict disk usage on each node depending on their actual capacity

    ipfs config Datastore.StorageMax 200GB

To facilitate automatic pinning by “lead” nodes… Have a look to https://medium.com/textileio/easy-personal-ipfs-pinning-service-with-textile-9d366da4e420

8

So, instead of this:

[Unit]
Description=IPFS Daemon

[Service]
Type=simple
ExecStart=/usr/local/bin/ipfs daemon --enable-gc
Group=root
Restart=always
Environment="IPFS_PATH=/home/username/.ipfs"

[Install]
WantedBy=multi-user.target

I should do this?

[Unit]
Description=IPFS daemon
After=network.target

[Service]
User=ipfs
Environment="IPFS_PATH=/data/ipfs"
ExecStart=/usr/bin/ipfs daemon --mount --enable-gc --routing=dhtclient
Restart=on-failure

[Install]
WantedBy=multi-user.target
9

Is better than root for system security…

Any startup options are ok, depending on your config.
enable-gc: activate garbage collector
routing=dhtclient: makes node more quiet as it doesn’t maintain DHT table but use it from other node (lead)

But I am a beginner user…

10

Since I have already given instructions on how to create a separate user (referred to as “username”) so that root isn’t the one running IPFS, I guess this should work, yes?

[Unit]
Description=IPFS Daemon
After=network.target

[Service]
User=username
Environment="IPFS_PATH=/home/username/.ipfs"
ExecStart=/usr/bin/ipfs daemon --mount --enable-gc --routing=dhtclient
Restart=always

[Install]
WantedBy=multi-user.target
11

it is customary to use a ipfs for username. This should be a user just for use by ipfs/cluster.

12

I’ll change the “username” to ipfs then, thanks :slight_smile:

13

Updated! I hope I have it right this time:

https://gateway.ipfs.io/ipfs/QmfXNrrqdu9WkW46gJJH1JmKjHKFhLYVg8dd71HRFWwUJs/

14

Hey, thanks so much. I’ll go over it when I have some time and do a final check!

15

Thank you! I am looking forward to getting it published if it is ready.

16

Does this look good to go? I want to publish it before anything changes that makes it need updating :slight_smile:

17

Thanks for the ping. I can’t load it up though :confused:

18

Thanks, I’ll check it…

19

I can’t get it either. IPFS is running on my remote server, and the hash is right. I am adding it to a local IPFS instance as well, maybe that will help.

In the meantime, here it is as just a regular webpage:

http://ul-qoma.com/IPFS_Cluster_Articlev3/

Thanks!

20

  • Build a 4 continent IPFS Cluster -> Build a 4-continent though it reads weird. Maybe Build an IPFS Cluster spawning 4 continents... is better.

  • Bootstrap the additional nodes to the join the cluster -> bootstrap additional IPFS Cluster peers to join the cluster.

  • Regarding the ipfs user, best practice would be to not use it to log in (instead login to some other user and switch to it -su - ipfs- or sudo -sudo -u ipfs -i as needed). Definitely do not give sudo power to the ipfs user, this defeats the purpose of having a separate user to isolate possible security issues in ipfs in the first place.

  • All the rest seems fine!