Social Media Architecture with IPFS

I wish I had funding to provide ā€¦ Iā€™m a terrible UX designer and a moderately poor programmerā€¦ but fairly proficient at running everything else.

Itā€™s really great workā€¦ Alas! In the end funding seems to actually matter.

I think, or would like to think, that within a few years the goal of a non-Corporately controlled social media platform will reach the ā€œnetworkā€ tipping point. There are several options available, but each requires a separate account and most the interlinking protocols have been disabled or are non-operative.

For awhile, I ran my own XMPP serviceā€¦ but then FB and Google shutdown external linkage and suddenly thereā€™s no one on the other of the lineā€¦ as per Big Tech intentions of squashing everyone else.

I think the solution is going to require blockchain and/or IPFS identities. However, the basic problem Iā€™ve had is that most people I try to communicate with simply ignore anything not posted to Google, FB, or Twitterā€¦

Alternative channels already exist and have existed for a very long time. However, very few people are using those alternative channels and itā€™s difficult to find people who are willing to navigate multiple platforms for the sole purpose of communicating with the one person who isnā€™t ā€œonā€ a non-Corporate controlled service.

I agree with all of that @ipfsme. The world needs something like a ā€œpublic general-purpose blockchainā€ that can ā€œhostā€ (but decentralized as IPFS) all social media data in the world, so that no one ā€œposts to Twitterā€ or ā€œPosts to Facebookā€, but they are posting ā€œto the internetā€, so to speak, with the important thing being no central point of control or failure (no BigTech monopolies). Iā€™m probably repeating a lot of what has already been said, but is definitely the consensus of opinion of the entire IPFS community.

So the Fediverse, ActivityPub, etc are what is needed so that the ā€œProtocol(s)ā€ are what all these heterogeneous platforms are using to interoperate, rather than building on any one common codebase.

In this kind of future, everyone would have 1) their own set of hashes that theyā€™ve ever posted on any service, and 2) a way to view and browse and KNOW later on what those hashes are. Everyone needs their own personal ā€œtreeā€ or data structure to organize their hashes theyā€™ve put out there, and thatā€™s one of the goals of Quanta.

Another potential use for Quanta is that itā€™s just part of the infrastructure of any Pinning Service (like Pinata.cloud), because it has the ability to hold a hierarchy of anything, and be not just your way of seeing your own data, but also the actual platform over which you share it.

You can think of Quanta as a ā€˜meldingā€™ of a Pinning Service and a general wiki or Social Media platform, or the piece of infrastructure that any Pinning Service can provide to unify the experience of people managing their pinned content, across all the different pinning services, so that to ā€œrunā€ or start your own pinning service, you donā€™t start from scratch, but you just use something like Quanta (even if not Quanta itself)

1 Like

BTWā€¦

I could provide some funding via a monthly BAT contribution, if you signup your github as a Brave Creator:

It wonā€™t be muchā€¦ about 5 BAT a month, which currently is worth about $1.25 USD ā€¦ but at least youā€™d have the ability to attract a nearly passive income source. If you could get a few monthly contributors, it starts to add up fast.

This is kind of off topic but related, I am working on a system to help with the IPFS/IPNS naming system. This might help your project since people will have easy to remember ā€œIPFS domain namesā€ and they can then share them out to create webrings, or social networks. I need help thinking through a few items though. Let me know if you are interested and we can discuss.

Cheers

-nb

Hello.

Just want to mention that in galacteek i use decentralized identities with IPID (DID documents stored in IPFS and resolvable with IPNS). Your DID has the form did:ipid:QmhGfhcā€¦ (the last part being an IPNS key ID). Peers exchange DIDs over pubsub and authentify each other through a DIDauth service (using p2p streams). When the DID document changes itā€™s republished to IPNS. One downside is that if a user goes offline for a while and the IPID is not republished, others would most likely not be able to resolve the DID thatā€™s why i use a backup resolve cache only for IPIDs.

Although DID specs are still evolving, i think any decentralized social media application based on IPFS should consider using IPID (or DIDs stored on a blockchain) for verifiable identities backed by crypto.

@wclayf

Thatā€™s an interesting topic. I feel like trying to replicate a centralized database model, what @reload mentioned re blockchains, is sort of outdated and proven not to scale. Thatā€™s why we have NIST P-521 EC social public keys that we share in a quorum based approach with peers in our example app qDesk.org - For verification purposes I feel like itā€™s best to provide a layer for every major external network, but unless itā€™s done by posting a signed message and scraping that, I doubt it really aligns with the values of decentralized computing.

I think a true blockchain as a way of ā€˜decentralizingā€™ Social Media may be too impractical, because of scalability and other reasons, but the Federation model of ActivityPub seems to be working (Pleroma, Mastodon, etc), although itā€™s a bit complicated to implement and test since Mastodon and otherā€™s donā€™t really follow the spec.

What IPFS can potentially bring to ActivityPub is the ability to send only the CIDs from peer to peer, rather than sending actual streams of actual data the way ActivityPub currently works. It would be a fairly straight-forward proposal to the AP Stream spec body that maybe they allow responses to contain just a CID that points to the data.

So for example if I get an AP request for a userā€™s outbox, I can just send back the CID of where that data is stored. This ā€˜seemsā€™ like it would be a good design path, but I itā€™s hard to predict if thatā€™s truly better than plain ordinary AP. I mean, since the server might want to customize the stream on a per-call basis, based on dynamic parameters, so in that kind of case a single CID may not work because IPFS is ultimately links to static pre-generated data.

I know IPFS can publish entire directories but Iā€™m not up to speed on how you could publish a small update to a directory without streaming up the entire content of the new directory. (assuming we were to represent a userā€™s outbox as a directory of files for example) Maybe IPFS chunking does make this practical (similar to how RSYNC sends updates only), but the Social Media scenario is challenging because the user is constantly pushing up new content one sentence at a time essentially, and this constantly changing feed may just be impractical for IPFS? Iā€™m not sure.

Like I said, I think that whole model is flawed, then you donā€™t really end up with a decentralized social network. Youā€™ll have a centralized social network on decentralized architecture.

On qDesk weā€™re using encrypted channels and sharing social profiles and posts that people can search like they do with the embedded search algorithm on Twitter. We donā€™t store any files to IPFS, unless the user attaches a file/image. Weā€™re using IPFS PubSub and people in the network of a certain peer listen to the updates that peer has, one word at a time or when peers are syncing the history also a few words at a time :wink:

Of course it would be cool to have longer messages in a file on IPFS you can just reference by the CID but I think thatā€™s not very well suited for a social network. Especially when EU citizens have a right to mutability. Weā€™re banning users from channels, flagging and blocking users and posts from the timeline search, etc. Thatā€™s pretty smooth when your social network works based on a quorum based approach, where thereā€™s no central directory but just owners and representatives of owners for different bits of data.

1 Like

@StatinedInTheField Iā€™m really glad you posted that, because I didnā€™t know about IPFS PubSub until I read your post, and was really excited to find out about that! I wasnā€™t quite sure yet how I was going to Federate or Decentralize ā€œQuanta.wikiā€ project, other than to use ActivityPub, but if I wanted to connect only instances of Quanta to each other then IPFS PubSub looks really interesting.

However, Iā€™m a believer in ā€œProtocols over Platformsā€, which means I donā€™t want to create a platform and then expect that I can get everyone to use it. I want a platform that can interoperate with a larger network of other platforms, that use a common protocol to communicate as a Social Media network.

RSS is the simple standard for just publishing a simple feed of course (and we can all already publish and consume RSS), but Social Media takes it further where you ā€˜followā€™ people, and consolidate a bunch of feeds from all your friends into a single feed where you can also post to their feeds (i.e. replies)

I mean perhaps even a ā€œWritable RSSā€ spec could even (mostly) solve some of this. If I can publish a feed but others can post ONTO that feedā€¦ that alone would be a game changerā€¦ but it doesnā€™t exist. Thereā€™s no ā€˜standardizedā€™ way to ā€˜updateā€™ an RSS feed afaik.

Obviously thereā€™s a huge network-effect already with the Fediverse (ActivityPub) and Iā€™d love to have Quanta connected to the Fediverse and those millions of users (if I had a full year to implement that!!) but youā€™re right that Federated definitely is not identical to Decentralized. People do want to own their own data, and if they do then itā€™s on them to store it, and doesnā€™t need a blockchain however.

Right now thereā€™s not even a standard file format for how someone would own all their Social Media data, afaik.


Product Plug: Actually in Quanta there is an ability to export all of the data into a ZIP or TAR that can be both browsed offline and imported back in later, so I have that, but what I mean is thereā€™s no ā€œIndustry Standardā€ for what it would even look like for someone to ā€œdownloadā€ all their Social Media data, into a format which could theoretically be ported over to some different platform, etc.

However, Iā€™m a believer in ā€œProtocols over Platformsā€, which means I donā€™t want to create a platform and then expect that I can get everyone to use it. I want a platform that can interoperate with a larger network of other platforms, that use a common protocol to communicate as a Social Media network

Wouldnā€™t we all? When itā€™s supposed to be decentralized itā€™s on us to build such a protocol.

Youā€™re right that Federated definitely is not identical to Decentralized

Yeah and I believe the social network of the future works browser to browser, democratic between the users, without any centralized parties involved. AP doesnā€™t allow that, even requires a user directory.

With our protocol we make sure you donā€™t need that since identities generated for our protocol are inherently unique. Itā€™s things like this that make the cornerstones of decentralized social networks.

Right now thereā€™s not even a standard file format for how someone would own all their Social Media data, afaik.

Iā€™m thinking probably makes more sense to encourage maintainers of social networks to import from other social networks. Like how we are planning to import from Matrix and Discord. Progress will always be faster than we can find standards in this field IMO.

Whenever you find the time look deeper into our Quest Network project Quest Network Documentation - I think we have a lot of the same goals and found solutions to a lot of your questions, we havenā€™t needed any major funding for the past several years, I personally am still working on it at least 6 hours every day (most days a lot more) and weā€™re just a few weeks from offering any feature that you would expect from say Discord (we even support bots and we also already have a few things Discord doesnā€™t have like QR invites) or Twitter (Iā€™m just now testing the timeline feature). Iā€™ve been helping reload to use this in a Python desktop app but with your stack it would be much easier!

You can also use our underlying NodeJS library quest-os for your TypeScript app and use encrypted channels on IPFS and our social functionality like the timelines (0.9.4+) from a super easy API interface. With our bee module, you also have the ability to export your data and ā€œbrowseā€ offline. If you lose the internet connection while talking to someone on chat on your local network, even the web demo will continue to work.

Our primary goal is to offer developers the ability to experiment with protocols and users the ability to enjoy a safe, private, democratic and decentralized social network experience.

What exactly is it that you want to solve with Quanta?

Sounds like you have a lot of cool stuff in qDesk. I still need to look at it better, I only glanced at it, so far.

About browser-to-browser decentralized: I think that level of pure peer to peer is good, but at some point somewhere there needs to be some kind of ā€˜commodity serverā€™ instances (kind of like how Docker commoditized deployments) that can do powerful search and timelines like Quanta does. We donā€™t want any single point of failure, but I think having 100s or 1000s of commodity servers run by volunteers or orginizations (perhaps getting Filecoin miners to run them) is a good plan.

One of the super easy product pivots or spin-offs I could do with Quanta is to make it a Search Engine for IPFS. Practically the only thing Iā€™d have to write would be a crawler, becasue the rest can be done by the power of MongoDB and itā€™s full-text search.

To answer your question of what even is Quanta:
If you scroll down on the main landing page you can find sections of info, and also here are the screencasts of it in action:


And if you want to see the tiny bit of docs Iā€™ve written so far go here:

https://quanta.wiki/f/user-guide

but at some point somewhere there needs to be some kind of ā€˜commodity serverā€™ instances (kind of like how Docker commoditized deployments) that can do powerful search and timelines like Quanta does

That would violate our decentralized design pattern and privacy laws in some countries.

We already have that search on timelines in 0.9.4, works perfectly in the browser.

Like the QR Code feature, we made sure it even works in the browser on my old Nokia. You know, one of our goals is that people in developing countries can use it without the internet, on old smartphones.

In case you are actually using it with millions of people and want magical search results based on your preference beyond regex and logic tricks: Weā€™re adding local machine learning models in 0.9.6 to make the search a little more like what people know from Twitter and other centralized social networks.

If people want to keep it 100% local and centralized around their own infrastructure, they can configure their own star node in the IPFS settings on qDesk.

Docker here: Docker

Manual for a free Heroku setup here: https://suda.pl/free-webrtc-star-heroku/

I didnā€™t ask what Quanta is, I read your wiki, I asked what you are trying to solve with it =)

I might create a pure peer-to-peer variant of Quanta, that does the things that can be done in browsers, like Secure Messaging between groups of friends,

ā€¦but the larger goal of Quanta is to be enterprise-level enterprise-scalable back-end infrastructre, that can Federate similarly to Mastodon/Pleroma, etc., but yet also something anyone running Linux can run a Federated instance of.

For example, one potential pivot would be to create a full-featured ā€œIPFS Pinning Serviceā€ on top of Quanta, because thereā€™s huge overlap in whatā€™s needed for that. This could potentially make Pinning Services" themselves ā€œcommoditizedā€ where everybody just downloads and runs Quanta for their pinning service back end, rather than each pinning service provider writing a proprietary platform from scratch, like they currently do.

Hereā€™s a more direct answer to what problem Quanta solvesā€¦

I might create a pure peer-to-peer variant of Quanta, that does the things that can be done in browsers

Everything can be done in the browser

but the larger goal of Quanta is to be enterprise-level enterprise-scalable back-end infrastructre, that can Federate similarly to Mastodon/Pleroma, etc., but yet also something anyone running Linux can run a Federated instance of.

Yeah thatā€™s the most popular usecase that people are using qDesk for right now. Just peer-to-peer without the need for centralized architecture.

For example, one potential pivot would be to create a full-featured ā€œIPFS Pinning Serviceā€ on top of Quanta, because thereā€™s huge overlap in whatā€™s needed for that. This could potentially make Pinning Services" themselves ā€œcommoditizedā€ where everybody just downloads and runs Quanta for their pinning service back end, rather than each pinning service provider writing a proprietary platform from scratch, like they currently do.

Weā€™re doing this under the hood when your favorites share files - But itā€™s a good idea for Quest Network users to earn money. Especially since weā€™re adding Eth integration in a few weeks. If you can write a good single purpose js library for that, we would use it!

Hereā€™s a more direct answer to what problem Quanta solves ā€¦

Oh so, it doesnā€™t actually do anything that others arenā€™t already doing more professionally. Your editing features are also being widely used unless I misunderstood that.

Weā€™re adding that too btw, since we also need collaborative text editing and especially drawing. If we wonā€™t be able to find a good library we can use for that, Iā€™ll be getting back to you, maybe we can work on a library together that works for our IPFS PubSub protocol and your app.

PS: Also asking myself how you could offer Quest Network channels, timelines, etc since you say you want to integrate many protocols but it seems like you would just be duplicating qDesk since your app relies on centralized paradigms like that there needs to be a single point that updates can be pulled from, which is impossible on decentralized social networks unless you create a centralized gateway, which would remove basically the core of decentralized tech. Am I wrong here or is that how your app works?

I have spent some time thinking about what makes a protocol for social communication diffrerent from a protocol for information ā€œin generalā€ communication like http.

First the philosophical points:
Communicating in a virtual world has some deep philosophical implications. EG : you cannot expect data to disappear (ideas donā€™t vanish), which is well known today (screen captures of deleted tweets), you cannot expect to have a reliable ā€œreal nameā€ policy (because at the end of the day you want to enforce equality between an idea, a name, and a physical entity). More technically when you talk to someone you donā€™t need an adress, just a name. So the content should be linked to the pseudo (nothing prevents you from using your ā€œreal nameā€ as pseudo.

Todayā€™s social network arec fundamentally very similar to one another from a data point of view : whether you are on insta, twitter, facebook, tik tok etc. you just publish data (pictures, text, whatever). Publishing mean that a piece of data contains its producer, timestamp, and maybe acces rights.

What makes a difference is how the feed (what you see) is built from the inner data. You have different AI. Marginally the type of links (symetrical Ć  la facebook) or asymetrical (Ć  la twitter) is of interest (two links in asymetrical is not equivalent to a symetrical one : for instance when you stop following someone is the person informed etc.).

The hard part is : how do you manage rights through time. I think it means you need to have blockchain part (to record associations of pseudo/passwords) together with a clever management of encryption/decryption keys. The idea is that when you publish you generate keys that are possible to build from your ā€œenable persons to read at the timeā€. The problem is that you may change your password (from which a public/private key can be generated say through a secure hash function) across time period. So you need to keep track to be able to decypher old posts. What would be the incentive to work on this blockchain is unclear to me.

So basically Social networks would only be special ways to browse this type of data.

An important barrier is that it looks hard to make it easy to non technologically savvy persons. And it is a hard problem to solve.

I am ready to work on both scientific/practical proposition. Do not hesitate to contact me directly to see if we can make something practical along those ideas.

1 Like

I think ActivityPub is great (Mastodon, Pleroma, etc), but some people will say since itā€™s ā€œfederatedā€ thatā€™s not good enough. The P2P purists. My opinion is that with IPFS used everywhere possible with ActivityPub, then the IPFS provides the kind of ā€œblockchainā€ aspects people are looking for in a new Web3.0 social media app.

Unfortunately I ran into brick walls using AP (trying to implement mine Java) because none of the Federated servers out there truly follow the ā€˜specā€™, so if something fails, unless you personally have a developer on the other end to tell you what went wrong youā€™re left guessing.

So ultimately for me the barrier to entry was language. I only code in Java on the back-end, and thereā€™s not really a Java library that you can use to enable AP. But the platform I developed (https://Quanta.wiki) is huge and powerful, and I am really wanting to get it onto the Fediverse ASAP, and looking for funding to do that.

Iā€™m not really a programmer, so I canā€™t follow a lot of the details in this thread, but I wanted to ask if there are any standards for the data in an IPFS based social media platform that could be used by any/all apps that can serve or pin that sort of content. If someone likes a post, the like button becomes a command to the app to pin that post locally, making it easier for others to find and see. You could subscribe to someoneā€™s feed, which would show posts, comments, pins/likes and shares. Previous posts in a thread could simply be an IPFS reference CID, not a copy of the actual content, meaning that the content could be pulled from the network dynamically.

A social media platform built like this would be unstoppable, uncensorable and perpetual. And if it caught on, other platforms could start to provide content to it, and share the content in it.

Been thinking about this for a while, wondering if there has been any progress.

1 Like

I had made a post earlier, after experimenting with IPFS this weekend. But removed it after reading this forum board much more. And my take on a IPSN, was rather more abstract.

Focused more on a Pokemon GO model of sharing content. While content can certainly be viewed on an IPFS endpoint that could host a userā€™s posts and other posts a user ā€œcollectedā€. I was more fascinated with a concept of verification in this process. Of whether a user has the right to share another personā€™s social content. So the GO model was users would have to meet in some form of direct contact to, so the social creator can transfer a hash encoded image asset to the receiverā€™s version which then authenticates (offline or online) and allows the user to indeed represent this piece in their collection or elsewhere.

Also no account creation. The device is the account. Creates a more ephemeral variant of a social persona that requires consistency and more transparency?

Donā€™t see why most of these existing SN features need to be re-attributed instead of IPFS, potentially repurposing the idea of a social persona as a whole

I resonated with @wclayf 's quote: ā€œProtocols over Platformsā€. .

@chadlupkes " If someone likes a post, the like button becomes a command to the app to pin that post locally, making it easier for others to find and see. " this is also an interesting way for ā€œposts to travelā€ and eventually be seen by someone. Rather than having search or having an explore feed. Then again, I guess it could improve an ā€œecho chambersā€ resilience, which could be a downfall.

@LughSpear ā€œWhat makes a difference is how the feed (what you see) is built from the inner data. You have different AI. Marginally the type of links (symetrical Ć  la facebook) or asymetrical (Ć  la twitter) is of interest (two links in asymetrical is not equivalent to a symetrical one : for instance when you stop following someone is the person informed etc.).ā€

Do you feel these traits truly define a SN? should they be incorporated at all moving forward? I feel the notification system, is flawed and not a healthy additive if something like a IPSN should be developed in the first place.

I am wondering how to join quanta??
Is it an application only available through web gateway, or is it ipfs install to join swarm?

Anyone can try Quanta just by signing up. Itā€™s free. But here are the caveats about IPFS on Quantaā€¦

btw, that user guide URL had a typo, hereā€™s the correct link for it: Quanta