Running Ipfs-cluster-service spikes up cpu usage

From the above screen shot when ever I run ipfs-cluster-service daemon,
I see that it is trying to get/or pins/dag heads.
This is taking a lot of cpu usage. Is there anything should I know about it.
Is it anyway be prevented and only fetch when required.

Thank you.

There is 2 possibilities, if you are syncing it’s just the CRDT calculating.
Else if most of the time it’s at a very low CPU usage and sometimes spiking up it’s very likely the golang GC.

2020-12-03T17:03:08.984+0530 e[34mINFOe[0m crdt crdt/consensus.go:210 new pin added: QmXYj7gqF33czS4eJrGn7Dxkk7aSypufBt4VrBaEydmpLs
2020-12-03T17:03:09.056+0530 e[34mINFOe[0m ipfshttp ipfshttp/ipfshttp.go:378 IPFS Pin request succeeded: QmXYj7gqF33czS4eJrGn7Dxkk7aSypufBt4VrBaEydmpLs

This is what I see in the cmd. Is there any way to control over that syncing CRDT calculation? As I see it takes up a lot of cpu at times.

Thank you

You are receiving pins for pinning at a high rate. Either you are syncing a cluster with a long history from scratch or your cluster is pinning a lot of things.

Note that if you are syncing, you should not interrupt the process until it’s finished. If you do, you will need to ipfs-cluster-service state cleanup and start from scratch to get it fully synced.

Other than that it’s normal operation.

1 Like

Is there a way on how I can turn off the syncing? Would that be a problem?
I’m using cluster just to add pins to one machine. I don’t think syncing is necessary.

Thank you

you can use ipfs-cluster-ctl --host </multiaddress> to talk direclty to that machine.

But otherwise, no, once you are synced, the cluster daemon will have less things to do and use less CPU.

ipfs-cluster-ctl --host /ip4/<ip>/tcp/<port>/p2p/<id> pin add

ip == the target machines public ip address
port == ?
id == is the the cluster id/ipfs daemon id ?

If I’m running above command, I don’t need to join the cluster network ?

correct.

ipfs-cluster-ctl --host /ip4/ip_address/tcp/<cluster_restapi_port, usually 9094> for the HTTP API

ipfs-cluster-ctl --host /ip4/ip_address/tcp/9096/p2p/<cluster_peer_id> for the libp2p HTTP API.

When you start the cluster daemon, it prints the API addresses on which it listens. There is support for basic authentication. Some more info here https://cluster.ipfs.io/documentation/reference/ctl/ .

I have two machines PEER1 and PEER2.

My PEER1 is like my central machine (this runs ipfs daemon & ipfs-cluster-service daemon all time)
So, every other PEER want to pin their file hashes to PEER1.

PEER2 (local machine) which is like other peers who wants to pin the files hash on PEER1.
If PEER2 is joining the cluster, there is lot of resyncing and the cpu spikes hit high.

Instead of joining the cluster network, can I pin my hashes on PEER1 through
ipfs-cluster-ctl pin add?

I have tried the above mentioned command.
ipfs-cluster-clt --host address pin ls.
But it runs into context deadline exceeded

Can you help me out if my scenario is achievable without joining the cluster.

Thank you

Check the ports, IPs, firewalls etc. that means that you cannot reach your PEER1. Make sure your listen address in the configuration are set to other than 127.0.0.1.

See also https://cluster.ipfs.io/documentation/guides/security/#ports-overview . You may want to enable basic authentication by adding some credentials to the config.