Network delay when adding / reading file

[guissou]

Hello,

I am currently trying to measure ipfs’ performances to write / read single block files (not in a folder) in an ipfs cluster of 15 nodes in raft mode. To write a file to a cluster, I use the Add function from ipfs-cluster API, to read a file, I use the cat function from ipfs API.
I am able to change the RTT between nodes, and noticed that the time needed on average to write a file is ~5-6 x maxRTT in the cluster, and reading ~3 x maxRTT in the cluster. Does it make sense ? I made experiments with maxRTT up to 2sec.

As I understand, when writing in Raft mode, the following packets are exchanged:

1. Writing node updates the cluster leader (1/2 RTT)
2. Leader broadcasts update to all peers and they ack (1 RTT)
3. Leader waits for a majority of peers’ acks and broadcast the accepted update, returns to Writing node (~1/2 RTT)
   Total: 2RTT, I got: 5RTT

When reading from IPFS, the following packets are exchanged:

@stebalien

When requesting a file, you’ll:

1. Ask all connected nodes for the first piece (root node) of the file.
2. After that, you’ll create what we call a “session” and put all peers that responded in step one into the session.
3. Finally, you’ll ask peers in the session for subsequent blocks.

I guess 1RTT for step 1.
1 RTT to establish the session in 2. ?
As the file has a single block, I am guessing step 3. takes 1RTT
Total: 3RTT, I got: 3RTT

Are operations performed locally (pinning) negligible compared with network delay ? If not what is the average time it takes for the longest local operations when writing and reading?

Thank you for your help

[hector]

When adding with Cluster:

• 1 round trip to send each block to each IPFS peer on which you are adding (happens in parallel)
• Once all the blocks are in the destinations, the pinning process starts:
  • Request to pin is forwarded to current Raft leader as needed
  • Current Raft leader broadcasts the operation and waits for acks and broadcasts accepted update.
  • Upon sucessful completion the PinAdd method returns to writing node
  • In parallel (async) once the Raft-pinning is successful, cluster peers request IPFS peers (usually on localhost) which have received the blocks to pin the root CID. I don’t know your network delay so can’t say if the local operation is negligible.

[guissou]

@hector

• Upon sucessful completion the PinAdd method returns to writing node

What does exactly the PinAdd method do ? Does it require more RTTs ?

@hector

I don’t know your network delay so can’t say if the local operation is negligible.

My network has a radius of roughly 300 ms.

[hector]

@guissou

Does it require more RTTs ?

No, it should not touch the network again.

[guissou]

Ok thank you for your answer!