[solved] How to use the IPFS self public and private key in node.js?

(posted here as per requested in https://github.com/ipfs/go-ipfs/issues/7715)
Hi,

I’m using IPFS 0.7.0 with the shiny new feature to export keys.

So i do:

ipfs key gen test
ipfs key export test

Which gives me a “test.key” in binary libp2p protobuf format.

What i try to do now is get that key working in nodejs crypto.
This is where i’m hitting some snags.

In node.js i need to import the key as a PEM key to be able to import and use it. This means it needs to be in this exact format:

-----BEGIN PRIVATE KEY-----
<THE KEY>
-----END PRIVATE KEY-----

Begin and end are easy :slight_smile:
The key needs to be in base64.

Now to load this into javascript i did the following:

const fs = require("fs")
const crypto = require('crypto');
const Schema = require("./crypto_pb")

let data = fs.readFileSync("./test.key")
let deserializedData = Schema.PrivateKey.deserializeBinary(data)
let keyBuffer = deserializedData.getData();
console.log(Buffer.from(keyBuffer).toString('base64'))

Note that very last console.log line. There i’m printing the key, which results in:

2qLSIF08WHkcz5f+ca90C+BRe7I8vjNlDarPREXCmkcNUzmzNro7dN0obChI5pQF3aK/owNVgEhw1OMd4dDUiA==

That’s too long.
The keyBuffer object is (i verified that) 64 bytes long. So it does seem to match what the libp2p spec tells.
The base64 result is however 88 characters which doesn’t seem to be right. Importing it as that gives me an error of: header too long.

I’m sure i’m missing a conversion somewhere. But where and from what to what is not clear to me.

Thank a lot!

Cheers,
Mark

Edit 12-10-2020
I changed the title from How to convert an exported key to base64 in nodejs? to [solved] How to use the IPFS self public and private key in node.js? because i essentially want to use the IPFS self key in node.js to add functionality. I want to sign a hash with the node’s private key which anyone else can then verify if they have the node’s peer id (which is also it’s public key as of ipfs 0.7.0).

@jacobheun Thank you for your response on that github issue!

You mentioned i should be looking into: https://github.com/libp2p/js-libp2p-crypto#cryptokeysimportencryptedkey-password

2 questions here.

  1. How was i even supposed to know that i need that? Literally nothing in the IPFS documentation gives me any clue about that. ipfs key export --help at the most generous gives me: Exports a named libp2p key to disk. makes me think that the libp2p key is dumped to a readable format on disk, which is totally not the case.
  2. The link you mentioned for crypto.keys.import wants a password as second argument. What is that supposed to be? I never even had the option in IPFS to create a private key with a password. So how do i import it then?

Just to remind you here.
I want to use the IPFS private key to generate a signature of the hash of content it had added.
I then want to let another party receive both that signature and my peer id to verify that i had actually added the content to IPFS.

I do not want to add a new key. The whole point is re-using the existing key to create that signature on one end and verify it on the other. Something that, in my opinion, really should be put into IPFS natively. As not having it natively requires me to either go through these hoops to do it safely or to do my own key management.

@markg85 let’s take a step back, can you explain why you’re exporting the key from Go into JS? I’m not clear on what you’re trying to accomplish with this. I understand you want to sign and verify content with your ID, but I’m not clear on why you’re exporting/importing the key for this.

Signature and verification support is possible with your existing ID, and it’s used for things like pubsub message verification and signed peer records. How are you currently trying to generate the signatures?

Right, let’s take a step back indeed :slight_smile:

I add content to ipfs, say like: ipfs add somefile
That gives me a Qm… hash

I merely need to have a way for anyone to confirm that I added something to IPFS if that user knows my peer id and the Qm… hash that needs confirmation. I’ll not go into details about why i need this as then this discussion is likely going to derail to discuss that.

What you said with “Signature and verification support is possible with your existing ID” is what i keep reading in places (the release notes of 0.7.0 among them) but nobody anywhere explains how to get that done (and i definitely asked multiple times and in multiple places!). How?

I just saw the other post you had at How to verify that <cid> is signed by <peerid>? talking about this, so let me try to clarify.

You cannot currently do signing/verification via the CLI, you need to use IPFS programmatically to achieve this. This will give you access to the sign/verify methods on they key https://github.com/libp2p/go-libp2p-core/blob/v0.7.0/crypto/ed25519.go. The release notes are specifically referring to the ability to programmatically unmarshal the public key from the ed25519 ID, which is not possible with RSA keys, for signature verification.

So that means i cannot sign any data with any exported key from ipfs, not even with the (js-)libp2p library, correct?

Is there a plan to add this to IPFS CLI?
Usually this request is immediately shot down with arguments like “but there are dedicated libraries for this, use those”… which is impossible, as I’ve demonstrated.

Having said that, it is possible with other libraries but only if you do all key stuff in there. Not if you want to take the private key from IPFS and do something with it.

So that means i cannot sign any data with any exported key from ipfs, not even with the (js-)libp2p library, correct?

You can, you just have to unmarshal the bytes of the exported key. Both Go and JS this would involve reading the file in and then using the unmarshal API for that language, (Go) https://github.com/libp2p/go-libp2p-core/blob/v0.7.0/crypto/key.go#L338, and then you could sign with the returned private key.

Is there a plan to add this to IPFS CLI?

There’s nothing on the immediate roadmap for this (there are also no closed/open issues requesting this). We’re looking at adding better support for using go-ipfs as an API, but in the interim you could look at creating a daemon plugin, https://github.com/ipfs/go-ipfs/blob/master/docs/plugins.md#daemon, to add support for generating signatures for a given CID.

Awesome, this brought me further :slight_smile:
This code can sign and verify!

const fs = require("fs");
const crypto = require('libp2p-crypto');

function str2ab(text) {
    return new TextEncoder().encode(text);
}

function ab2str(buf) {
    return new TextDecoder().decode(buf);
}

(async function() {
    try {
        let data = fs.readFileSync("./test.key")
        let key = await crypto.keys.unmarshalPrivateKey(data);
        let sign = await key.sign(str2ab("QmReEZnKLvXvfric19kv6crts7ijK3xPicNEMkHFqBWfEq"));
        let hexSign = Buffer.from(sign).toString("hex");
        
        // At this point you have a readable (hex format) sign buffer.
        console.log("Sign: " + hexSign)

        // This public key should be created from the data you received
        let publicKey = key.public;

        // Verify the hash is signed by the private key
        let verify = await publicKey.verify(str2ab("QmReEZnKLvXvfric19kv6crts7ijK3xPicNEMkHFqBWfEq"), Buffer.from(hexSign, "hex"))

        // Should print true if verified
        console.log(verify)
    } catch (error) {
        console.log(error)
    }
}());

Now i’m stuck with the next issue… It never stops :wink: (well, nearly there)
I want to give “the other party”:

  • the signature
  • my peer id
  • the Qm… it needs to verify with that sign

The way (js-)libp2p seems to be working is by accepting the protobuf key object to unmarshal it. That is fine for the signing side, as there you need to have the private key. It should not be shared so having that in a protobuf as a binary is OK.

However, i would like to send the receiving side a string version of my peer id (which should be a public key). But that won’t work because the peer id is in a different format (not in the protobuf format).

How would i tackle that one?

Also, another issue is that you apparently can’t export the self key. Is there a way to get that key in the protobuf format? In this case it is an option to add a new key, as long as it’s all in ipfs

I’m curious what your thoughts would be for this one and how you’d solve it?

Edit
Got it!

        const multihashes = require('multihashes')
        let peerId = "<your peer id>"
        let multiHashPeerID = multihashes.decode(multihashes.fromB58String(peerId))
        let peerPubKey = await crypto.keys.unmarshalPublicKey(multiHashPeerID.digest)

That gives a proper public key i think :slight_smile:
Took a bit of “reverse logic” fiddling from the go code that extracts a the public key: https://github.com/libp2p/go-libp2p-core/blob/a39b84ea2e340466d57fdb342c7d62f12957d972/peer/peer.go#L92

Last thing i now need to know is how to get the self private key from IPFS?

Edit 2
For whoever ends up here with google. The IPFS private key (the self one that can’t be exported) is in the ipfs config file under the name PrivKey. It’s base64 encoded.

To be complete, loading your private key in (js-)libp2p is done like so:

        let data = Buffer.from("<your private key>", "base64")
        let key = await crypto.keys.unmarshalPrivateKey(data);
        // ... the same as my other example in this post

Got it all working now :smiley:
This took quite a while!
Your help, @jacobheun, was really valuable here! Thank you very much!

1 Like