I have CI pipeline which checks my playbooks. They install go-ipfs and pins files we need for operations. Those files are pinned on pinata and on our (non-CI) servers.
The time pinnig works varies drastically. Sometime my workflow ends in 4 minutes, sometimes it’s over 20 minutes of retries.
Is there any way to speedup pinning? What is causing slow pinning?
I work on: GitHub - Jorropo/linux2ipfs: This is a tool to upload files to estuary.tech in a VERY lightweigth maner.
Which solves this issue.
It is capable to pin files at gigabytes/s, (capped by the speed of your disks, network or remote servers).
But I suspect it wont be any faster since pinata servers aren’t.
It sounds interesting, but I have a rather small amount of data (< 50MB), and I can’t understand logic of slow pining. Is it something related to the way IPFS works, or I need to debug the issue?
50MB should be almost instant, <3s, my tool is more for many gigs to Tib level of data.
The most likely issue is that pinata is slow.
I think you should change to a faster pinning service.
You should also note that current implementation of bitswap are not fast, in the precise case where you upload to a pinning service, it is faster to stream exporting
.car files (archive of IPLD blocks) and send that over http stream, tls, ssh, …