Ipfs-cluster-service daemon --bootstrap failing "failed to negotiate security protocol: peer IDs don't match"

Help
1

I’m attempting to boot strap a cluster between two nodes.

Node 1 is running in a freebsd jail with a NAT’d interface on 192.168.43.2. The system has a public IP address and ports 4001, 9094-9096 forwarding to the jail. IPFS on this node has the public IP address set in .ipfs/config --> Announce set to “/ip4//tcp/4001”.

Node 2 is a system running on my home network NAT’d behind my router with an external IP. Ports 4001,9094-9096 forwards from the router to the internal system.

Using "./bin/ipfs dht findpeer " on each node, each node gets information for the other node.

When I attempt to do an ipfs-cluster-service daemon --bootstrap from either node, I get the same error (with corresponding opposite node info)

2020-07-02T05:52:43.189Z DEBUG pstoremgr pstoremgr/pstoremgr.go:328 failed to dial QmauwGtdDwrAf16hEYWKYTLeSpFcetXqkDAVZ21sajZwCe: all dials failed

  • [/ip4/174.136.99.242/tcp/9096] failed to negotiate security protocol: peer IDs don’t match

I can’t find any information on how to resolve this issue.

Any thoughts?

Thanks
-jim

2

The message says that QmauwGtdDwrAf16hEYWKYTLeSpFcetXqkDAVZ21sajZwCe is not the peer ID listening on /ip4/174.136.99.242/tcp/9096

3

From the peer config file on that system:

{

“Identity”: {
“PeerID”: “QmauwGtdDwrAf16hEYWKYTLeSpFcetXqkDAVZ21sajZwCe”,
“PrivKey”: “…”
},
“Datastore”: {
“StorageMax”: “10GB”,
“StorageGCWatermark”: 90,
“GCPeriod”: “1h”,
“Spec”: {
“mounts”: [
{
“child”: {
“path”: “blocks”,
“shardFunc”: “/repo/flatfs/shard/v1/next-to-last/2”,
“sync”: true,
“type”: “flatfs”
},
“mountpoint”: “/blocks”,
“prefix”: “flatfs.datastore”,
“type”: “measure”
},
{
“child”: {
“compression”: “none”,
“path”: “datastore”,
“type”: “levelds”
},
“mountpoint”: “/”,
“prefix”: “leveldb.datastore”,
“type”: “measure”
}
],
“type”: “mount”
},
“HashOnRead”: false,
“BloomFilterSize”: 0
},
“Addresses”: {
“Swarm”: [
“/ip4/0.0.0.0/tcp/4001”,
“/ip6/::/tcp/4001”,
“/ip4/0.0.0.0/udp/4001/quic”,
“/ip6/::/udp/4001/quic”
],
“Announce”: [
“/ip4/174.136.99.242/tcp/4001”,
“/ip4/192.168.42.3/tcp/4001”
],

4

That’s the IPFS peer ID. Cluster peers have their own peer IDs (in .ipfs-cluster/identity.json).

5

So when the docs reference a peer id, it should be documented as “cluster peer id” in the file you mention?

6

I guess when the cluster documentation mentions a peer id it is the cluster peer id, yes. Do you have a pointer to a concrete place where this can be improved? The overview states:

IPFS Cluster is a fully distributed application. ipfs-cluster-service runs a Cluster peer and all peers are equal. Cluster peers form an separate, isolated libp2p [private] network, which uses the cluster_secret (a 32-bit hex-encoded passphrase present in the configuration of every peer).

This network does not interact with the main IPFS network, nor with other private IPFS networks and is solely used so that Cluster peers can communicate and operate. The network uses a number of blocks also used by IPFS (DHT, PubSub, Bitswap…) but, unlike IPFS, does not enjoy public bootstrappers.