I don’t know if you’ve seen this paper about Hadoop on IPFS? Maybe it helps.
Don’t forget to report back when it’s out
That’s technically true, but yet you can built great things with small blocks.
@Yaxir I will list the projects built on IPFS I know loosely related to data management. As I’m not really in the field myself, a lot will be irrelevant, but some may help.
There may be more.
Depending on your model (does every peer want to edit and access the same piece of data
? Is eventual consistency enough? etc?) , you may want to use gossipsub to have a topic per data that interested peers want to follow, to get updates quickly and have real-time updates. CRDT’s also come to mind.
Your design will also depend on the frequency of updates.
Information being accessed and referred to is not a problem with IPFS. You can access the CID for a static version, or the CID pointed to by an IPNS record for an updatable version. See qri.io above to see how they approach version management of datasets.
The “consistently updated” part will depend on whether the updates are frequent, how dynamic is the group of peers having the right to update, do all the pieces of data have the same group of peers able to update them, and is peer accessing a non-super-fresh data ok (eventual consistency).
It’s hard to help you further without knowing a bit more.