Any suggestion to make IPFS content searchable/addressable by a user defined tag?

[es_00788224]

@mfan

How exactly to do it to allow many nodes to update the same file/directory name?

Same way, share the IPNS key.

@mfan

Google “vpn bar in china” will show you recent developments about VPN ban in China and Russia.

Russia does not currently have a VPN ban.

@mfan

You need to use shadowsocks or something similar to this My Experience With the Great Firewall of China to camflage the VPN traffic.

The Tor Browser Bundle includes obfsproxy which does the same thing.

@mfan

However, I don’t thinking searchable tag is a horrible idea in IPFS.

It’s not a bad idea per se, but it will be much harder to implement than you’re claiming. How will you solve flooding? This is a non-trivial problem, since you can’t check the correctness of a tag programmatically (you need a human to do it).
So then you need some mechanism for trust, and you’re suddenly looking at a lot more complexity than “add a new IPFS object type”.

@mfan

Guess one important use case for searchable tag is a native key-value store in IPFS.

You already have a key-value store. The database is a folder, the key is a file name, the value is the content.

[mfan]

@es_00788224

Same way, share the IPNS key.

This supports multiple clients. But data loss could happen when many clients update the same IPNS endpoint at the same time.

@es_00788224

How will you solve flooding? This is a non-trivial problem, since you can’t check the correctness of a tag programmatically (you need a human to do it).

Let me explain my idea in more detail. A proposed Tag object has two fields, “Name” is the name of the tag, and “MultiHash” is the multihash of the IPFS object the tag pointing to. Let’s defined H1: multihash(Tag.Name), and H2: multihash(Tag).

H2 is the traditional IPFS content addressed key. It is used as usual to find the Tag object and then the content object the tag pointing to.

H1 is used for tag query, which is different than normal content query. A tag query could return a list of (H2, H2’, H2’', …), and it’s up to the application to decide how to deal with the data.

To support tag query, the routing key for tag is (H1, H2) which has one more dimension than usual content key. Content propagation is driven by H2 and the flooding pattern shall be the same as usual. We might have (H1, H2) and (H1, H2’) cached in the same node and it’s OK.

The IPFS tag query operates at a different plane to support IPFS content annotation. The above design shall serve our goal and make IPFS content searchable/addressable by tag name. Note that when the same tag Name pointing to different IPFS objects, there will be multiple different IPFS Tag objects. A tag query for Name can therefore return multiple results.

For the implementation, we need to augment IPFS routing API, e.g. add new APIs PutTag(), GetTag(), etc. Tag objects are small and can be stored within DHT.

@es_00788224

So then you need some mechanism for trust

I’m afraid this kind of feature shall be supported by Applications. We don’t need to worry about it here.

@es_00788224

You already have a key-value store. The database is a folder, the key is a file name, the value is the content

It’s far from a key-value store. I believe an IPFS directory (folder) is no different than a blob object. Anything changed within the subtree creates a new IPFS object, which requires publish the directory again over IPNS. With shared keys, it supports multiple clients. But it’s not safe and could have data loss.

[es_00788224]

@mfan

This supports multiple clients. But data loss could happen when many clients update the same IPNS endpoint at the same time.

So, a race condition. It’s trivial to implement locking.

@mfan

A tag query could return a list of (H2, H2’, H2’', …), and it’s up to the application to decide how to deal with the data

@mfan

I’m afraid this kind of feature shall be supported by Applications. We don’t need to worry about it here.

This is where you have the problem. What happens when someone inserts some trillion of tags with random keys and values? Is the application supposed to download terabytes of data and “decide how to deal with the data” later? So you need trust BEFORE downloading the tags, and then you have no need for direct indexing anymore.

Here’s a simpler idea: ask web.archive.org to publish their collection to IPFS. They already index most sites, and they’re considered trustworthy. You could then navigate to /ipns/web.archive.org/*/webpage and get redirected to /ipns/web.archive.org/YYYYMMDD/webpage.

[fuckGoogle]

Yep, makes sense, in essence this is what cyber is doing. Building a user defined semantics core using IPFS CIDs, blockchain and a decentralized knowledge graph