Get number of nodes with CID pinned?

I’m trying to diagnose our highly varied load times of a static site with the content pinned to a local IPFS gateway.

One idea I had was to view the number of nodes with the content pinned. Is there a way to do that? Couldn’t find in Google or docs.

Something like ipfs numNodes hash

1 Like

Good idea. I assume it would go into the dht part of ipfs, i.e. ipfs dht countprovs <hash>. You could always determine it like ipfs dht findprovs <hash> | wc -l but that’s one additional command, so a single ipfs command is surely preferable.

1 Like

I think that ipfs dht findprovs <hash> will return any nodes with the content whether or not it’s pinned. I don’t know if there’s a way to tell if someone has something pinned or if it’s just cached on their node and waiting to be garbage collected.

Unless the content is very small and fits in a single chunk, it would also be nice to have a way to tell how many people can provide all of the hashes from the entire DAG and not just the top-level hash.

1 Like

That would then be ipfs dht [findprovs | countprovs] -r <root-hash> with output (a) when only counting nodes like

<root-hash>: 17 <sub-hash1>: 8 <sub-hash2>: 23 ... <sub-hash-n>: 47

or (b) when printing the actual PeerIDs like

<root-hash>: 17
<PeerID-1>
<PeerID-2>
...
<PeerID-n>

<sub-hash-1>: 8
<PeerID-1>
<PeerID-2>
...
<PeerID-n>

...

<sub-hash-n>: 47
<PeerID-1>
<PeerID-2>
...
<PeerID-n>

It would get a little crowded, though, if you use the verbose option. :slight_smile:

1 Like

That would be very handy.

In the meantime, we can probably get at some of the information by mashing together ipfs refs -r <hash> and ipfs dht findprovs like so.

( HASH=QmX8LDhDSYdX3xG6cHFUybXLDSuvo9Lz6wF5NU3UVmJRnB ; echo $HASH ; ipfs refs -r $HASH ) | xargs -l bash -c 'echo -n "$0 - "; ipfs dht findprovs -n 2 $0 | wc -l'

This returns output that looks like this:

QmX8LDhDSYdX3xG6cHFUybXLDSuvo9Lz6wF5NU3UVmJRnB - 2
QmPYVMnpamYrLC9W49eJfieh9Po613UUxaBY9UE8Rmrcm6 - 2
Qmaw9M9C7JKwT3XoucoJyxoEJxPxPSp1bpxQvYcLgGJskb - 2
QmQNvsA6aqAL5GRCRokNq7pkQHeJ19rQ45XLcRGaFPiSbR - 2
QmQ5BJLdLM8HShW35RXHoujbkvQxh9UiDZJs44JxkQxtMg - 2
Qmbmfk69gtUqkXLXP6nSh9C3hEBk4hQGGz1Nc6ET7hZtJt - 2
...

To make this much faster than it would be otherwise, findprovs is limited to only searching for 2 providers with the -n 2 flag.

I’m not sure why this happens, but after running this for a while on a large file (like the one in the example) my daemon almost always seems to crash and then ipfs dht findprovs starts returning “Error: api not running”.

1 Like

I tested just the ipfs refs portion of the script, and it seems to be the -r option. Without it it’s fine & fast, but with -r it sent my CPU to 150%, and I assume it can lead to an ipfs crash.

The ipfs refs -r <hash> definitely took a while the first time I called it. At least when I was testing, it was almost instantaneous during subsequent runs. I’m not sure if it matters but I was testing with v0.4.11-dev.