Ipfs gateway timeouts

Help
21

@ylempereur is this neccessary to expose port 4001, since in the EKS/pod/svc we are not exposing it. also the AWS ALB is internal means, no one can reach https://our..internal.ipfs.com/ipfs/.
we access it via VPN.

now I have doubt if we don’t expose (port-forward) the 4001, how our IPFS see all the peers (around 600) and how the pinned a content can be accessed via other public IPFS gateways?

but I see that deamon is having public IP?
I think IPFS one purpose is also to bypass all the restrictions like firewall rules, …

22

@HamidullahMuslih,

I suggest reading through the following blog post.

When you request a CID from an IPFS node, the request can be broken down into two steps:

  1. Content discovery/routing: asking direct peers and querying the DHT (opens new window)to find the peer IDs and network addresses (opens new window)of peers providing the CID (referred to as providers).
  2. Content retrieval: connecting to one of the providers, fetching the CID’s content, and streaming the response to the client.

Actual debugging steps

Here’s how I debug one of the CIDs you provided: QmeMBccZ4XwZExsPNaGANDDPq4KHEghXAQH91cKA2167pJ:

  1. Find providers in the DHT:
    ipfs dht findprovs QmeMBccZ4XwZExsPNaGANDDPq4KHEghXAQH91cKA2167pJ which returns PeerIDs of providers

Result

12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF
12D3KooWKuSzGoorvsBB8zdXCYVMEfybYKa7sDgXT3AkofAzX3FV
  1. Try to ping the PeerIDs to see if these PeerIDs are reachable
$ ipfs ping 12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF
PING 12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF.
Pong received: time=275.42 ms
Pong received: time=280.18 ms
Pong received: time=273.83 ms
Pong received: time=334.89 ms
Pong received: time=312.37 ms
Pong received: time=271.37 ms
Pong received: time=279.96 ms
Pong received: time=312.39 ms
Pong received: time=277.58 ms
Pong received: time=282.73 ms
Average latency: 290.07ms
----
$ ipfs ping 12D3KooWKuSzGoorvsBB8zdXCYVMEfybYKa7sDgXT3AkofAzX3FV

Looking up peer 12D3KooWKuSzGoorvsBB8zdXCYVMEfybYKa7sDgXT3AkofAzX3FV
Error: peer lookup failed: routing: not found

This means that only one of the two peers is reachable, namely 12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF.

  1. Finally, trying to fetch the blocks from that peer. First find the addresses of the peer:
ipfs dht findpeer 12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF | sort
/ip4/127.0.0.1/tcp/4001
/ip4/127.0.0.1/udp/4001/quic
/ip4/172.31.80.213/tcp/4001
/ip4/172.31.80.213/udp/4001/quic
/ip4/5.161.55.227/tcp/4001/p2p/12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ/p2p-circuit
/ip4/5.161.55.227/udp/4001/quic/p2p/12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ/p2p-circuit
/ip4/84.146.179.248/tcp/4001/p2p/12D3KooWPutck3ASkfcU13Ht5jzgevBo9Fc1sHS5yYberAYupDtq/p2p-circuit
/ip4/84.146.179.248/udp/4001/quic/p2p/12D3KooWPutck3ASkfcU13Ht5jzgevBo9Fc1sHS5yYberAYupDtq/p2p-circuit
/ip6/2a01:4ff:f0:1e5a::1/tcp/4001/p2p/12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ/p2p-circuit
/ip6/2a01:4ff:f0:1e5a::1/udp/4001/quic/p2p/12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ/p2p-circuit
/ip6/64:ff9b::5492:b3f8/tcp/4001/p2p/12D3KooWPutck3ASkfcU13Ht5jzgevBo9Fc1sHS5yYberAYupDtq/p2p-circuit
/ip6/64:ff9b::5492:b3f8/udp/4001/quic/p2p/12D3KooWPutck3ASkfcU13Ht5jzgevBo9Fc1sHS5yYberAYupDtq/p2p-circuit
/ip6/64:ff9b::5a1:37e3/udp/4001/quic/p2p/12D3KooWSW4hoHmDXmY5rW7nCi9XmGTy3foFt72u86jNP53LTNBJ/p2p-circuit

It looks like the node isn’t publicly reachable (only has private IPs)
All the addresses with the p2p-circuit suffix are relay nodes that can help hole punch a connection to the node to retrieve blocks.

Normally I use http://check.ipfs.network/ to debug content retrieval.

Input

@HamidullahMuslih Can you check if that is your PeerID (ipfs id -f '<id>')?

Recommendations

  • Enable accelerated DHT client (seems you already did)

is this neccessary to expose port 4001, since in the EKS/pod/svc we are not exposing it.

Yes! Ensure that port 4001 on your IPFS node is reachable from the IPFS node’s public IP (either port forwarding. This is critical for other nodes to be able to retrieve blocks using the Bitswap protocol.

23

@danieln
yes 12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF id of my IPFS.

but the 4001 is not exposed, then how your node found it?

24

Because your node is publishing to the DHT both peer records (mapping from your peerId to addresses) and provider records for that CID.

See the Content publishing lifecycle section in the blog post.

1 Like
25

As Daniel said, use the following to test if your node is reachable:

IPFS Check

P.S. I just tested your node with it, and it is reachable, but, it’s through a relay, which makes things inefficient. You should open those two ports, things will work better.

2 Likes
26

@danieln and @ylempereur thank you,

for below content my two IPFSs services did not show any provider
ipfs dht findprovs Qma2L7kLniaafUDWRbGhGy22aiHA9qsv6PD8CzZZMipBhj

but it can be retrieved from below link

https://ipfs.io/ipfs/Qma2L7kLniaafUDWRbGhGy22aiHA9qsv6PD8CzZZMipBhj

can you provide any light how and why ?

27

This could be due to a combination of two things:

  • The IPFS node hosting/pinning Qma2L7kLniaafUDWRbGhGy22aiHA9qsv6PD8CzZZMipBhj is not publishing/advertising provider records to the DHT
  • The same IPFS node is directly peering with the ipfs.io gateway so that it asks for the content directly over BitSwap (without having to look up the DHT)
1 Like
28

@ylempereur as relay you mean there will be one more hop to reach our IPFS.
and the adresses will look like below

		"/ip4/188.166.36.145/tcp/4001/p2p/12D3KooWAtQymAZZ1KwaTqr1oLSomEb45C6oQQokMLkwjeAhdwYq/p2p-circuit/p2p/12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF",
		"/ip4/188.166.36.145/udp/4001/quic/p2p/12D3KooWAtQymAZZ1KwaTqr1oLSomEb45C6oQQokMLkwjeAhdwYq/p2p-circuit/p2p/12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF",

without relay will look like below:

		"/ip4/my-public-ip/tcp/4001/p2p/12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF",
		"/ip4/my-public-ip/udp/4001/quic/p2p/12D3KooWFkPkpLWTKDDbE4Dtdknrhj17KTEsduYp4cnS9aYeUuYF"
1 Like
29

Correct.

Using a relay causes the connection establishment to take a lot longer (using hole punching). Once established, it’s a direct connection and works just as fast as any other, though.

So, if it’s an option to set up port forwarding and poke a hole in the firewall, it’s much better than using a relay. Especially if you are running a server, and not just a client on a personal machine.

2 Likes