We’ve had AI companies tell us that cloud operations can be more complex and costly than traditional approaches, particularly because there aren’t good tools to scale AI models globally. As a result, some AI companies have to routinely transfer trained models across cloud regions – racking up big ingress and egress costs – to improve reliability, latency, and compliance.
It sounds like a problem that’s just asking for IPFS to solve. Any thoughts?
I’m here because I want to find a way to use/share data from the SatNOGS ( https://satnogs.org/ ) network with an AI. I’ve scraped about 1.2 terabytes of a 20+ terabyte archive, which grows about ~1TB/month as far as I can tell. There are other groups that are also scraping the data for their uses.
I’ve looked into setting up p2p proxy for SatNOGS, but I don’t quite have that working yet. I may dump the files here, but I’m not sure the best way to go about it yet, as it seems everytime I add a file, the DNS record needs updating (?):
A similar issue, one of sharing large indexes, is faced by YaCy. YaCy is a Free Software project which aims to provide distributed, user trained, open-source search engine software. If an individual “trains” their instance by web-crawling a large number of sites, the cache can become large, many Gigabytes, and to be of use to others, it needs to be shared peer-to-peer.
IPFS people are cordially invited to take a look at YaCy here: