Public gateways will not retrieve my files 😔

I’ve been trying for hours, forwarding ports, monitoring network traffic, and trying various computers (desktop, laptop, DigitalOcean droplet). I cannot retrieve my files over any public gateway. Port 4001 is open. It still doesn’t work. Just endless loading until an eventual timeout. :pensive: :pensive: :banana:

EDIT: I’ve also used my mobile phone and I tend to have around 120 peers.

EDIT: I’m drunk and I’m going to tear my hair out. :rage:

EDIT: Nevermind. It was just taking like 20 minutes to load. It works now. Just kinda slow on the upload.

That was a roller-coaster of emotions.

1 Like

Hey,

would you be able to verify your peer is diallable from the outside?

  1. ipfs id should show an external IP and port
  2. curl <ext_ip>:<ext_port> should connect (hopefully from a different computer)

Depending on the setup, opening port 4001 in the router may not help, because IPFS sets up upnp on a different port, and/or the detected external port is the NAT one from the router (the main thing here is what is the port shown by ipfs id next to the external IP). If upnp does not work then the peer is not dialable. The actual IP/port can be overriden with the Announce parameter in the config.

Then, assuming your peer is diallable or has found a relay to become diallable, the gateway will have to find it. This is where the public gateway health and the general state of the DHT come into play (both for the content your peer is providing and for the content the gateways are retrieving). For those things, we are working very hard on improving reliability of the network.

Which gateways did you try and from which location?

ok I can confirm my peer is diallable from the outside. Your curl test gets a 404
I opened up the firewall on azure and I can access the IPFS webgui from my laptop which is on a different network.

In all my testing (not very much) whenever I cut and paste a cid QM… into the explore box the three dots just keep spinning. I’ve only ever been able to access XKCD etc I had assumed that if I could discover a peer I could ‘ls’ it etc. Question If someone has set up a node but hasn’t added a file would that cause problems. Can you see my file?

I found the same thing too. It is just really really slow publishing your file hash on the DHT (if it manages it at all). The first file took 49 minutes to publish and become available (and hence for my other node to be able to find it). The second file didn’t publish after 14 hours. I was using DigitalOcean droplets, so no NAT problems. Which seems to be the knee-jerk reaction to any access problems you have.

Any command like ipfs dht or ipfs provide can take anywhere between minutes and hours to complete, if it completes at all.

yes that’s the conclusion I’m coming to as well. I’ve just done a ‘ipfs ls’ from the command line on all the cid’s listed on the Explore tab and they all work. The reason I set up my IPFS node on azure was so I could leave it running all the time. Can you keep trying and see when I’m visible to you?

There are two things here. If you are talking about explore.ipld.io, this seems to use a js-ipfs embedded node. Since js-ipfs does not have DHT-discover/routing, it relies on something we call preloader nodes to fetch that content from the main dht.

explore.ipld.io is different from gateway.ipfs.io on how content is fetched (they use different infrastructure).

The reason why it takes so long to provide records on the DHT is that most peers in it (where those records should go) are not diallable therefore the provider calls keep searching for longer for appropiate peers. Undiallable nodes are a big issue right now even if your single node is but we are working on improving this!

Note that go-ipfs now support different providing strategies. See https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#reprovider and also try https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#strategic-providing . Depending on the amount of content you have, this may improve how fast things are provided. You may also try increasing the connection manager settings to increase the limit on the total number of peers (to 2500 for example).

1 Like

I can browse explore.ipld.io but gateway.ipfs.io gives 404. So I’m adding to my mental model, just because I can see a peer doesn;t mean I can ls|get their files. Correct? I’ll try increasing my peer limit.
Question does IPFS watch the config file ie. do I have to manually restart after re-config?

Ok I increased my peer limit to 1900 and left it for a few hours, but I still can’t ‘explore’ random Qm’s just XKCD etc. I set up on Azure so I can run 24/7 as I suspected that people running for a few hours will not really help the reliability of ipfs. I got into ipfs through torrent paradise and he explains the importance of running 24/7 and pinning. I’m running nohup ipfs daemon& from the command line but I’d appreciate instructions for setting up a service on Windows Mac and Linux. This would help you as much as me

but gateway.ipfs.io gives 404

What gateway url are you calling?

I have to manually restart after re-config?

Yes

I’m running nohup ipfs daemon& from the command line but I’d appreciate instructions for setting up a service on Windows Mac and Linux

There are a few tutorials around. i.e. Set up IPFS node on the server - Michal Zalecki

https://gateway.ipfs.io/ipfs/QmRdzqKJPVzjaUJsfFDyUdvZdAn3eidAAyQtehNczv2hmc my own gateway

Next problem…

echo “

Michal

” > index.html
ipfs add index.html
ipfs cat Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9
Error: invalid path “Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9”: selected encoding not supported

Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9 is clearly Qma1PYYMwbgR3GBscmBV7Zx8YgWdhBUAY6z27TznhrBet9 concatenated twice…

Here is the real problem: You can make a lot of mistakes/typos in the process. And even if you do everything right, content resolution can still be very slow, too slow sometimes …