Can we use ipfs for big data like hdfs?

I have an idea for a thesis on how to optimize the big data architecture using blockchain / IPFS, and analyze the data using smart contracts, is it possible to do?

Netflix is using IPFS for pushing software update: New improvements to IPFS Bitswap for faster container image distribution | IPFS Blog & News

Some guy tried and succeeded a long time ago: https://www.cse.unsw.edu.au/~hpaik/thesis/showcases/16s2/scott_brisbane.pdf
https://s3-ap-southeast-2.amazonaws.com/scott-brisbane-thesis/decentralising-big-data-processing.pdf

Hope it helps.

1 Like

thank you for the reply and the thesis document that you have included is very helpful for me.

I see the thesis that they are working on combining ipfs and hdfs, where ipfs as storage and hdfs in this context is map reduce as processing.

what I want to do is how ipfs optimizes hdfs, ipfs as data storage as well as data processing with the help of smart contracts. I expect faster runtime performance, and higher availability

1 Like