Directory slow to sync to public node, but single files are not

Help
1

Hello, I’m not sure it’s the right place to look for help, don’t hesitate to tell me where this should go if needed.

I created a public IPFS node on a linux box, and tried to add files and directory on it, before trying to access them from anonther public node like ipfs.io/ipfs/*

The weird thing is that the files are almost accessible immediately (ie. 5 to 10 seconds), but the directories containing these same files takes literally hours before being available on other nodes, which is a real issue considering directories are needed to have predictable file names.

I use the following bash command line to check files, which are all returning a 200 http status code when requested from another public node:

ipfs pin ls --type=indirect | awk '{print $1}' | xargs -I {} curl -s -o /dev/null --max-time 5 -w "%{url_effective} -> http %{http_code}\n" 'http://ipfs.io/ipfs/'{}

http://ipfs.io/ipfs/QmVhqX28jWrbTKzyiLLg5cdUuEboFNRQgcvH4bm85WuoXA -> http 200
http://ipfs.io/ipfs/QmYCvbfNbCwFR45HiNP45rwJgvatpiW38D961L5qAhUM5Y -> http 200
http://ipfs.io/ipfs/QmejvEPop4D7YUadeGqYWmZxHhLc4JBUCzJJHWMzdcMe2y -> http 200
http://ipfs.io/ipfs/QmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB -> http 200
http://ipfs.io/ipfs/QmPfmSphchj5NaX3hATyeUdnZe5EZ32GvE84iNWNYZjDAt -> http 200
http://ipfs.io/ipfs/QmQGiYLVAdSHJQKYFRTJZMG4BXBHqKperaZtyKGmCRLmsF -> http 200
http://ipfs.io/ipfs/QmTUQDQQLdyvdnKDGXD7xc9TH4FvpNqDcogBvC97W62PgM -> http 200
http://ipfs.io/ipfs/QmU5k7ter3RdjZXu3sHghsga1UQtrztnQxmTL22nPnsu3g -> http 200
http://ipfs.io/ipfs/QmQ5vhrL7uv6tuoN9KeVBwd4PwfQkXdVVmDLUZuTNxqgvm -> http 200
http://ipfs.io/ipfs/QmQy6xmJhrcC5QLboAcGFcAE1tC8CrwDVkrHdEYJkLscrQ -> http 200
http://ipfs.io/ipfs/QmUCj5TaQyUjoJyJUW4HUsUREN3A974DSZvf5WCEaGL8a4 -> http 200
http://ipfs.io/ipfs/QmbcCMKD69dbYmT4xXWEyqbd9D8gmef64PYz1oKCJFhXAH -> http 200
http://ipfs.io/ipfs/QmcX36X5c2mitj7LgTTgnpoDFmYniX6C8dVt7vjzhuihbP -> http 200

If I try the same on the directories containing these files, almost all of them return a 504 error code, is it normal, a known problem, or a configuration issue with my public node?

Many thanks for your insights.

ipfs pin ls --type=recursive | awk '{print $1}' | xargs -I {} curl -s -o /dev/null --max-time 5 -w "%{url_effective} %{http_code}\n" 'http://ipfs.io/ipfs/'{}
http://ipfs.io/ipfs/QmRM18zhr9dK1auT24DZ9FE8PTrW7HozRTfZjAUwq624EF 000
http://ipfs.io/ipfs/QmWm6hmW9QaMya2RgsVoXAohrnrC7MSsnAyeRnH6nxnzyv 000
http://ipfs.io/ipfs/QmXsYTMj8FFPQPSkJrtnPzaKS1WrZY5CEVcaNaSmD6h77W 200
http://ipfs.io/ipfs/QmWeUC7H8tEG2siuZEx6gr6co6EqSnP8zmCi6MC94TKUpB 000
http://ipfs.io/ipfs/QmcVy9K3qmpbyrpisnMni7Cc6VTx7np8w3nW5XmoapDzgs 000
http://ipfs.io/ipfs/QmQPeNsJPyVWPFDVHb77w8G42Fvo15z4bG2X8D2GhfbSXc 200
http://ipfs.io/ipfs/QmQRu3LHXrHCQaEALFnwP7ZPj91UeQXaj9BoSfaCDG6wzV 000
http://ipfs.io/ipfs/QmR5oKoPSzPREjtRag3FBcQBQ5Bs95XWwug5TpVfa4PyPh 200
http://ipfs.io/ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn 200
http://ipfs.io/ipfs/QmVaphs79wNm21h7Tn2pn6T786QXaWkWyJ7CY7eUDzVViY 000
2

Directory listings in UI are slower because UI retrieves first block of all linked files to check if it is file or subdirectory and display correct icon. You can check yourself in IPFS UI and see difference between hash browse (fetch leaf) and hash inspect (just root block)

1 Like