FileCoin and Pinata

Help
1

I am trying to figure out better the use cases for both file coin and pinata.

Here are my opinions.

Pinata is for pinning. Great. So I can easily pin a file to pinata and pinata will store that file on its nodes(not sure on how many nodes, but i guess more than 1, right ? otherwise it’s still a single point of failure). When I want to retrieve a file, I can just do that simply… All good, all happy. So in the end, pinata is just a network of computers and those computers that pinata owns is pinata’s nodes, nobody else’s…

FileCoin is much better, because on filecoin, you can rent out your space and you can store other people’s files. All clear, all happy.

I am trying to wrap my head around the fact that why I’d need to use both of the services. I read that it’s a good thing that using both of them together is much better. Can I know why ?

I can think of these reasons:

  1. it’s said that pinata is much faster for retrieval, than filecoin. Any idea why ?
  2. pinata has its own nodes so in the end, we still trust pinata’s nodes, while on filecoin, distributed solution is better, because whoever stores our files, submits proofs (if not, gets penalized). What if it still gets penalized ? does it mean our file was lost ? or does our file on filecoin still get stored on multiple nodes ? I know that file gets split into multiple parts and gets stored on multiple nodes, so that each node can’t figure out what the file is, but what i am asking is if even after sharding, 1 whole file ends up being on multiple nodes and not on one.

So those 2 above are my questions and my 3rd question is that is using filecoin with pinata only better because it defeats single point of pinata’s failure if pinata used only without filecoin ? but pinata is still good to be used with filecoin, because it’s faster for retrival, which i don’t understand why.

2

They have 4 public nodes, 2 in 2 locations. You can ask them for private nodes if you pay enough.
By default they pin in 1 node, but if you ask to pin in 3 locations, your file is gonna be counted then 3 times in your used space.

3

On pinata accessing your files is free.
What I mean by that is that once you pinned your files, anyone can retrieve them.
On filecoin, retrieving a file require the retriever to pay the hoster a small fee. (this is since if you were able to pay for other users to fetch your files, a hoster could generate many other accounts and act like they are downloading your files again and again, emptying what you allocated to this hoster).

So in the end, pinata is just a network of computers and those computers that pinata owns is pinata’s nodes, nobody else’s…

Pinata doesn’t run their own network, they make it accessible on the whole IPFS network, so other people running IPFS nodes / clusters (like me) could also pin your files if they find them interresting (or reshare them for a short while, once they have accessed them).
You could also use a concurrent pinning service too, or setup your own IPFS pinning nodes.

it’s said that pinata is much faster for retrieval, than filecoin. Any idea why ?

  • Pinata runs on the IPFS network, it’s basically torrent on steroid and thus simple to implement and use (compared to filecoin). IPFS doesn’t require you to download terabytes of blockchain data for example.
  • Filecoin is a complex protocol, that require lots of overhead on the blockchain first. (filecoin network, include and use the IPFS network, but not the other way around)
4

@Jorropo

Thanks a lot for the nice answers.

One thing i quite didn’t understand is:

  1. so retrieval miner pays the hoster miner a small fee in order to get a file from hoster miner. then retrieval miner sends it to the actual user who asked for a file. Is this true ?

  2. If I bought a contract with a miner so that he stores my file. Imagine now that my friend wants to access this file. so my friend asks for the file. Who pays the retrieval miner ? because retrieval miner also has to pay to hoster miner, so it’s kind of logical that someone also pays to retrival miner. who ?

  3. So it’s like on filecoin, if file is hosted on some hoster miner, my friend still can get it right ? it’s not protected or anything, but I guess that option exists in filecoin so that no one can get it but me and miners who host it.

  4. So, if I buy a contract, it gets stored only on one hoster miner, unless I buy another contract. So my file is in a single point of failure, unless i buy another contract. Now, look at this What is Filecoin? How to Buy FIL (2021) - Decrypt search for Is Filecoin safe? and you will see that it says about sharding the data so that one hoster can not figure out what the data is. I mean, if I bought a contract with only one miner, I guess the whole data is with that miner and if I don’t store encrypted data, that hoster can still get what the data is very easily. Is that true ?

  5. The wrong thing in my opinion is that if I buy one contract, it means one hosting miner. So what if he deletes the data or anything.Then my data is lost. I shouldn’t be needing to buy multiple contracts with different hosters to store my data in multiple places. don’t you agree ?

5

so retrieval miner pays the hoster miner a small fee in order to get a file from hoster miner. then retrieval miner sends it to the actual user who asked for a file

I’m not sure, I’ve not played with filecoin (running a node require very good hardware). I think in practice the retrieval miner and the user who asked for a file would be the same user (since you would not like to pay to retrieve files for other users for free).

If I bought a contract with a miner so that he stores my file. Imagine now that my friend wants to access this file. so my friend asks for the file. Who pays the retrieval miner ? because retrieval miner also has to pay to hoster miner, so it’s kind of logical that someone also pays to retrival miner. who ?

I think that either :

  • Your friend is actually the retriever
  • Your friend has a deal with the retriever (imagine a service where you pay, and they provide you with files from filecoin your node without you having to run filecoin)

So it’s like on filecoin, if file is hosted on some hoster miner, my friend still can get it right ?

Yes, he can fetch the file, but HE has to pay. For your friend this is maybe fine, but imagine you are asking random users coming to see your site to pay, they would probably just walk away.

So, if I buy a contract, it gets stored only on one hoster miner

In practice you would never use 1 contract, siacoin for example uses at least 30 contracts. (*last time I’ve used it).

Now, look at this What Is Filecoin (FIL) ? The Beginner’s Guide - Decrypt search for Is Filecoin safe?

I guess this is probably an unclear sentence or this is in an other context is even the same. Idk what was intent behind this. What I know is that sharding and / or parity is really not a safe way to protect data.

If they tried to hack it, they would find just a stream of meaningless data

This looks a lot how you talk about encryption, not sharding (if you want to protect your files you do both anyway).

The wrong thing in my opinion is that if I buy one contract, it means one hosting miner. So what if he deletes the data or anything.Then my data is lost. I shouldn’t be needing to buy multiple contracts with different hosters to store my data in multiple places.

I think the miner have a penalty and loose a bit of coins (I’m not certain).

But again, you would never buy 1 contract. The UI is likely gonna hide everything from you anyway, what will likely happen is that you will set a maximum price and a target replication factor, then the node is gonna do everything itself (I guess, it’s mostly how sia does it, and this works well).

6

@Jorropo

Do you believe that on filecoin, if upload 200MB file, that file will completely in full size be stored on a hoster miner ? it should be right ? Otherwise, behind the hood, it might be putting different parts of the file on different hoster miners which is also possible too. but, if that’s possible, there’s a problem. Imagine my 200MB was split into 20 file parts. and filecoin stored each part on each different hoster miner that i bought contract with. there’s a chance that 10 of the hoster miners died. They get penalized, but for some reason, imagine that they didn’t care. now all that means, that i can’t ever get my file back , since 10 parts of it is lost.

This is why I think it doesn’t make sense to me to store different parts of the files on different nodes. I think each file should be getting stored in full size on the node.

WDYT ? which one actually does filecoin handle ?

7

Maybe filecoin has an option for single contract IDK.

I know that what I would do is :

  • First split the file in lots of chunks
  • Generates parity for thoses chunks (this mean that you can loose any of the chunks independently from each other, at least as long as you still have a complete replication total)
  • Sends thoses chunks + parity chunks to multiple nodes. So you would be able to loose any combination of nodes and still recover the file.

So for me what would happen is that your file would be splited in 20 part, then 40 parity parts would be generated, and you would be able to loose up to 40 nodes and still recover the file.

There are reasons to not go for full on with the parity, mainly that it’s hard on CPU.

1 Like
8

@Jorropo is there any way I can better understand parity ?

Let’s say my file is document.pdf. Okay, let’s say that it gets split into chunks. chunk-1-binary, chunk-2-binary , chunk-3-binary.

Okay, so what do we mean by parity now from these 3 files ?

9

Concretely parity allows you to loose any part of the data and still recover the whole set, it’s the same thing raid 5 and 6 does (and probably other raids).
So if you have 3 chunks and a parity, if you loose the parity well this changes nothing as your data is still there. If you loose a chunk, the parity + the 2 left chunks will allow you regenerate your data, any combination of left chunks well be enough, you can push the process to 2 parity, loosing up to 2 chunks, …

You may look at :

It’s the same idea pushed to the extreme. For something like this you wouldn’t use any of the algorithm Linked there, you would a bigger variation of thoses, as they are not well suited for computing large amount of parity.

10

To clarify, Filecoin, or rather the main implementation – Lotus –, currently doesn’t do any sharding. It’s up to your application to do that. It does support replication though, so the same data can be stored on multiple storage nodes.