Software Repository Mirrors may be a good start for IPFS

Hi, I think hosting public software repository mirrors such as pip-mirror or apt-mirror is a good start for IPFS.
How can we make this possiable?

1 Like

Check this discussion here!

1 Like

I am very interested in a RPM mirror, for distributions that use RPM repositories for software management. Please consider making both a rpm and deb repository for Linux users who want to maintain the up-to-date daemon system-wide.

Both pip, rpm and deb is bad for deduplication.
Git/pijul and IPFS should get objects inside compressed files and other binaries. Similar thing were proposed by @jbnet here: Ubuntu archive on top of IPFS

@kehao95
There is a project for implementing pip packages on IPFS: Dpip - Python Package Index (pypi.org) on IPFS

Thereā€™s also NPM-on-IPFS and GX package manager. But package managers will not work feasibly without deduplication. Thatā€™s why I propose Common Bytes: (draft) Common Bytes - standard for data deduplication

Hello everybody.

I think any Linux package management system could use IPFS.

RPM and DEB will cover most of distributions out there.
With ArchLinux and Alpine being popular as well - there is lot of potential.

Also Docker or Vagrant images could be shared this way.

But obviously we have language packaging systems (Gems, Pip, NPM, Maven repos, ā€¦) and MSI and DMG files as well if we go there.

ā€¦

Question is to have dedicated software for each, write brand new general one as ā€œone meets all demandsā€ (but there is no such thing as silver bullet, is it ?) - or to try extend existing binary repository software and implement everything ā€œone by oneā€ (sounds painful).

So how do we get there? How attract maintainers to pickup any of possible solutions so we can start using benefits if digitally signed p2p distribution - even if itā€™s still alpha?

If I am not mistaken, most of binary repositories support HTTP protocol and it is only question of putting a gateway in front of specific mirror (directory and file) format. That might work for backward compability - which is important - but might not use full benefit of digitally signing and versioning of everything.

ā€¦

Also we want IPFS to gain big traction, we canā€™t forget that most binary repositories supports authentication which is used for paying customers - ie. Oracle Linux and RedHat Enterprise.

I am new to IPFS so I am not sure how to use benefits of this new p2p software which makes by nature everything public? (I hope there might be some specification/concept in work I havenā€™t seen yet that can address this)

ā€¦

One slightly different note - one of features I would love to see would be a client - as companion to IPFS binary repository - that could do file checkout only for requested files on build as in for CI/CD usage. NPM is great example. I guess using symlinks and DNS aliases for IPFS links might be one solution - but it would be better if that could be rolling transition supported directly by mainstream instead of huge rewrite depending on all individual package maintainers.

Anyway ā€¦ thanks for reading.

@RubenKelevra runs a cluster for pacman packages: https://github.com/RubenKelevra/pacman.store/blob/master/README.md#pacmanstore

1 Like

And there work being done to put Nix on IPFS too.

I wonder if ipfs-pack could be used as (temporary ?) repository solution.

It has ā€˜serveā€™ command so it could be used for backward compatibility when used as web server.

And maybe the ā€˜bagā€™ or ā€˜carā€™ command could be used for rsync?

If I understand correctly, bag format could be split to smaller chunks - ie. repositories or maybe even packages? No idea about CAR archive though - need to look up itā€™s specification.

I wonder if there is ipfs import for bag/car archives from ipfs-pack ā€¦ and if that support private IPFS clusters too :thinking: