Would a VM proxy help me isolate specifc data in regards to file systems/networking? and change/ view my own perhaps in a more functional way? as opposed to only accessing/having direct accress via a single device? What i am curious about is lan internetworking for the purpose of better understanding protocols and syntax of the stack structures being used.
Ah, right.
I shouldāve know that the hash isnāt a signature, my bad.
I see what you mean. IPFS provides the public key (peer id) and the data but not the actual signature!
That does make me wonder what the release notes meant with āThis means that someone only needs your PeerId to verify things youāve signed
ā? As that doesnāt seem to be possible based on your comment. At least not from a IPFS cli point of view. I have a feeling i miss something here as i canāt imagine the release notes to mention something that isnāt possible.
Going the PGP route, as you suggest, is an alternative option indeed.
The annoying part is that it adds a layer.
To give you an example, i really would have liked to do:
// Store some data in IPFS, get the CID of it
echo "some fancy data" | ipfs add
<cid>
// Get the IPFS Peer ID as ed25519
ipfs id
// Send to pubsub
pubsub-send --pubkey=<IPFS peer ID> --data=<cid>
// On one of the servers i would like to do something like
// (note that it would fetch the data behind <cid>):
ipfs key verify --pubkey=<IPFS peer ID> --data=<cid>
Right now the only missing part is getting the signature of that data.
If IPFS would only have a:
ipfs key sign --pubkey=<IPFS peer ID> --data=<cid>
That would be so super awesome to have!
It would prevent the need to roll my own key management.
It would greatly simplify the usage of keys.
It would only require me to store a list of accepted public keys.
It would allow me to verify the data is valid (due to the immutable nature of IPFS) without downloading the data. And on;y download it when all other checks have passed
Easy peasy and super awesome!
ā¦ but weāre not there yet, it seems
For a time I may suggest you create the dedicated DAG branch containing the hash text calculated from the given CID and the peer ID, linked as another CID insige DAG object using ipfs object patch add-link
. Investigating the possibilities I try to use pymultihash
to produce something like digital signature.
Could you elaborate on that please?
It sounds like youāre on to something cool, but i miss details to make much sense of it.
I think I made some progress
Of course. I am on the way, so probably I will share something in a couple of days
I am not sure I understood all youāve said. But if it could help.
Here is a tool to sign and verifiy ed25519 signature
It is used on ācommon directoryā (~/.zen/ipfs_swarm) where every node publish data in a folder (.QmIPFSNODEID) regularly kept in sync between ipfs swarm nodes with this script
NB: We use Scuttlebutt āfriendā relation to add a node to swarm
Sorry for the belated reply, @zicmama.
Yes, that can very definitely help! Thank you for sharing those tools.
I am however hesitant to use it because it means iād have to do key management myself.
Iām still eagerly awaiting what exactly was meant with that line in the release notes:
ā¦ This means that someone only needs your PeerId to verify things youāve signed ā¦
As that kinda triggered my hope for key management in ipfs native. We, as users, can somehow verify that Node X actually created Data Y. Or to be super specific. That IPNS record X is created by node Y.
But HOW?
I just donāt see it without additional steps (like making a signature). And if thatās actually the case here then that specific line in the release notes is misleading at best. Also, that very same line definitely implies (with āthings youāve signedā) that IPFS now has the user facing option to do just that. And i read that from it because doing that yourself is not trivial. Why, you might wonder?
- data signing is, even for technical inclined people like ipfs users, a whole different beast to tame. You can safely expect for the vast majority of IPFS users that this is a step too complicated.
- The release notes mention that you can verify data with the peer id. But it lacks any public facing ways of signing data. Therefore you must do it yourself. And to do it yourself you must:
2.1 Get your private key from IPFS
2.2 Construct a keypair from it
2.3 Create a signature from your data
2.4 Store it somewhere
2.5 Send it to the one that needs to verify your data based on your peer id
2.6 ā¦ the receiving party needs to go though much of the same steps
I honestly just canāt imagine that IPFS would assume that users are able to all of the above. But i also canāt imagine it to be completely false. Surely the writer of those release notes had āsomethingā in mind and knew āsomethingā but didnāt share the intention here yet.
IPFS (or rather libp2p) will do it out of the box. Itās not implemented yet, but itās on the short term road map. There will be customizable behaviour as to whether to sign records and what to do with unsigned records. I expect the default to be signing and accept anything, then signing and reject unsigned in a few releases, for a smooth network upgrade. There is a Github issue somewhere, but I think itās at the spec stage.
It was a lost of man-hours to implement it before switching to ed25519, so they waited until now.
It will be especially useful for gossipsub hardening, as upfs will not only check the record when fetching it, but also when propagating the record, since it doesnāt need to have contacted the publishing node to check its jey and the record integrity.
Iāll try to find the GH issue.
This + the github issue you mentioned. Thatās awesome!
Having that in pubsub is a win imho!
Howeverā¦ Itās still not something a user of ipfs (or even a developer using ipfs) can possible use as itās a quite deep internal implementation detail in libp2p that handles it.
I do hope itās the intention at some point to somehow for pubsub have some way of whitelisting (or blacklisting) peer idās which internally then does signature verification. That allows to create pubsub channels where only selected nodes can understand the data. Of if you go one step further, only selected nodes can join that room. Good stuff
More generic ipfs commands for this would be really useful. Even though there are dedicated security crypto libraries out there, which is always the argument for not doing this. It just saves so much hassle to simply be able to create a signature from the ipfs cli and verify it with a cli command too.
This will likely be something you will have to handle at the app level, by encrypting the messages and share the key with peers deemed relevant by your app.
Iām stuck, sorta.
When you export an ipfs key:
ipfs key gen test
ipfs key export test
you get a libp2p
format binary blob.
As per the documentation:
ipfs key export test
USAGE
ipfs key export <name> - Export a keypair
SYNOPSIS
ipfs key export [--output=<output> | -o] [--] <name>
ARGUMENTS
<name> - name of key to export
OPTIONS
-o, --output string - The path where the output should be stored.
DESCRIPTION
Exports a named libp2p key to disk.
By default, the output will be stored at './<key-name>.key', but an alternate
path can be specified with '--output=<path>' or '-o=<path>'.
I think the documentation for that is here: https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md
I was trying to make a command line tool that would only require openssl 3.0.0 in which you would provide your ipfs private key to sign an ipfs hash (no need to sing the content as the hash represents the content) to get that signature. But from the looks of it i need to go the much more complicated route of making a application that accepts protobuf, parse that key output from ipfs key export test
and the finally be able to singā¦
And iām sure i still missed something that i have yet to discover.
Really guys, make things easier then this.
As I have discovered, you may just to generate the CID for your PeerID string and then link the obtained CID to the graph with the CID of your data using ipfs object patch add-link <data CID> PID <CID generated for PeerID string>
. This approach does not need to use additional software, it is truly IPFS-native.
I donāt get how that makes a third party (that does know the peer id) able verify that the one who added the link is the same one that added the original <cid>
?
Also, what does PID
have to do with this? PID is process identifier. Iām either missing something or itās unrelated.
Could you give a complete example of your discovery, that might be more clear.
Could someone elaborate on the release note please? :
Previously go-ipfs generated 2048 bit RSA keys for new nodes, but it will now use ed25519 keys by default. This will not affect any existing keys, but newly created keys will be ed25519 by default. The main benefit of using ed25519 keys over RSA is that ed25519 keys have an inline public key. This means that someone only needs your PeerId to verify things youāve signed , such as your Peer Records or in the future Signed Provider Records, which means we donāt have to worry about storing bulky RSA public keys.
As (i think) i really like to use whatās there but iām beginning to have my doubts if that paragraph holds any truth. Please just forget what iām trying to do (you can read it in the comments above). I now just simply hope to hear how to do exactly whatās written in the release notes.
Example of how things worked before and now:
Before:
IPFS tries to find /ipns/k2-ipns-key
and receives a record from the DHT that looks essentially like
{
value : "/ipfs/bafy-my-file",
signature : []byte{someBytes},
publicKey: []byte{publicKeyBytes},
}
It then verifies that SHA256(publicKeyBytes) == k2-ipns-key
. If thatās true then it checks if the signature is verified by the public key extracted from publicKeyBytes
. If so then itās a good record and we can return /ipfs/bafy-myfile
, if not then itās a bad record so we cannot return the value.
Now:
IPFS tries to find /ipns/k51-ipns-key
and receives a record from the DHT that looks essentially like
{
value : "/ipfs/bafy-my-file",
signature : []byte{someBytes},
}
Because the public key is so small it could be encoded in the name (i.e. less than 42 bytes per https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#peer-ids) we can extract the public key from the peerID pubKey = peer.ExtractPublicKey(id)
. We then check if the signature is verified by pubKey
. If so then itās a good record and we can return /ipfs/bafy-myfile
, if not then itās a bad record so we cannot return the value.
TLDR: Less data
Not needing to embed an extra RSA public key in every single thing we sign is a really nice bandwidth + storage saver when aggregating over all the provider records, IPNS records, pubsub messages, libp2p signature handshakes, etc. that a node emits and processes. For example, DHT provider records are not currently signed which has some unfortunate ramifications (e.g. I cannot ask a third party to advertise in the DHT that I have some data and the inability to store metadata in provider records). Adding an RSA key into every single provider record we emit would be tremendously costly, but we can now avoid that when we switch to signed provider records.
That is really smart, @adin!
Hats of to the one that came up with that idea!
Thanks a lot for the clear explanation!
Note, the solution for this question too is now in [solved] How to use the IPFS self public and private key in node.js?. I made this topic when i didnāt know the route to get it working yet. The other topic was more to the point where i was in the right direction but missing some details. Your pointer to https://github.com/libp2p/go-libp2p-core/blob/a39b84ea2e340466d57fdb342c7d62f12957d972/peer/peer.go#L92 proved to be one of the missing pieces in this endeavor
Experimenting with new version of IPFS daemon, I discovered that now the signing problem is successfully solved, but the deeper one is provenance that the given PeerID is owned by the person which operates the daemon. Embedding of the 3rd party digital signature into the DAG, which I planned to use at first, only partially solves this problem. This is why I did not publish my last discovery and proceeded with work on it
I donāt see a problem there.
The peer id is the usersā public key.
I trust āthat peer idā.
The user creates whatever data and signs it with itās private key. I can verify that sign with the public key (the peer id) as iāve demonstrated in this post.
I can apply this concept now to IPNS keys (in the 0.7.0 scheme that is) and to any ipfs data where a public key and a sign is known to verify that i want that data. Now how to do that last part is up for debate as iād need to make a format, say like:
{
"pubkey" : "peer id or some other public key mechanism",
"sign": "%$@#TGE#%YHH%#YHYT$R^",
"contentHash": "Qm....."
}
I can then verify that the Qm.....
is in fact signed by that public key (as you need the private key to sign it and only you should have that private key). Still, this only means that i trust whatever that given public key + sign tells me. In this scheme anyone can sign any Qmā¦ hash. Itās just that i trust those that are signed with publickeys that i trust.
With that logic iām able to confirm that <cid>
is valid for me, provided that the user provided me that above json blob to be able to verify it. If the user were to send a IPNS in 0.7.0 format then the public key can be omitted as itās part of the content hash.
Now it would be super awesome if the sign would be part of the content hash itself! Something like:
<cid><sign>
= <newcid>
As it would then contain:
- public key
- sign of the content hash with the private key of that public key
- the content hash
It would solve everything i mentioned here.
It would allow verification with only <newcid>
.
The downside is that this will explode the <newcid>
in length to unusable long.
But i think this mechanism with the json blob above as āmetadataā would be very workable already! Anything more that simplifies it further is just a big bonus
With that, iām making this thread as solved.