Has anyone seen this error? cannot unmarshal object into Go struct field

Help IPFS Cluster
1

I have a four peer cluster. When I run ipfs-cluster-ctl peers ls, it looks like all the peers can see each other:

$ ipfs-cluster-ctl peers ls
QmPszzw8UkJkbfxft3FR5t41hiX6EEHbC85cu8ykfhFsNW | Dharwin Node D | Sees 3 other peers
  > Addresses:
    - /ip4/127.0.0.1/tcp/9096/ipfs/QmPszzw8UkJkbfxft3FR5t41hiX6EEHbC85cu8ykfhFsNW
    - /ip4/207.148.86.28/tcp/9096/ipfs/QmPszzw8UkJkbfxft3FR5t41hiX6EEHbC85cu8ykfhFsNW
    - /p2p-circuit/ipfs/QmPszzw8UkJkbfxft3FR5t41hiX6EEHbC85cu8ykfhFsNW
  > IPFS: QmZkbzXbg6NQEUUn9cuyYPEEokhBotUwoe3Qh4j1nBspmG
    - /ip4/127.0.0.1/tcp/4001/ipfs/QmZkbzXbg6NQEUUn9cuyYPEEokhBotUwoe3Qh4j1nBspmG
    - /ip4/207.148.86.28/tcp/4001/ipfs/QmZkbzXbg6NQEUUn9cuyYPEEokhBotUwoe3Qh4j1nBspmG
    - /ip6/::1/tcp/4001/ipfs/QmZkbzXbg6NQEUUn9cuyYPEEokhBotUwoe3Qh4j1nBspmG
QmUCXkEZ6FWH1ZhfP66aT1ZdteE9fUQh9PziN9XgFi8Svg | Dharwin Node A | Sees 3 other peers
  > Addresses:
    - /ip4/127.0.0.1/tcp/9096/ipfs/QmUCXkEZ6FWH1ZhfP66aT1ZdteE9fUQh9PziN9XgFi8Svg
    - /ip4/45.32.232.5/tcp/9096/ipfs/QmUCXkEZ6FWH1ZhfP66aT1ZdteE9fUQh9PziN9XgFi8Svg
    - /p2p-circuit/ipfs/QmUCXkEZ6FWH1ZhfP66aT1ZdteE9fUQh9PziN9XgFi8Svg
  > IPFS: QmTkxoWXYy4brGtj3XGpTMsZYY3GHzgHryWdgFQnuWZeqM
    - /ip4/127.0.0.1/tcp/4001/ipfs/QmTkxoWXYy4brGtj3XGpTMsZYY3GHzgHryWdgFQnuWZeqM
    - /ip4/45.32.232.5/tcp/4001/ipfs/QmTkxoWXYy4brGtj3XGpTMsZYY3GHzgHryWdgFQnuWZeqM
    - /ip6/2001:19f0:5001:2f30:5400:1ff:fedf:aed2/tcp/4001/ipfs/QmTkxoWXYy4brGtj3XGpTMsZYY3GHzgHryWdgFQnuWZeqM
    - /ip6/::1/tcp/4001/ipfs/QmTkxoWXYy4brGtj3XGpTMsZYY3GHzgHryWdgFQnuWZeqM
QmYQuJrttL3P2yx8u6SVhRXDjwa9Aqq4suY46oyxxo4t9i | Dharwin Node C | Sees 3 other peers
  > Addresses:
    - /ip4/127.0.0.1/tcp/9096/ipfs/QmYQuJrttL3P2yx8u6SVhRXDjwa9Aqq4suY46oyxxo4t9i
    - /ip4/155.138.132.101/tcp/9096/ipfs/QmYQuJrttL3P2yx8u6SVhRXDjwa9Aqq4suY46oyxxo4t9i
    - /p2p-circuit/ipfs/QmYQuJrttL3P2yx8u6SVhRXDjwa9Aqq4suY46oyxxo4t9i
  > IPFS: QmdVDKX2ZcsfFi444xijhYV4cJmMJ3n32uH2GHXuL6NT35
    - /ip4/127.0.0.1/tcp/4001/ipfs/QmdVDKX2ZcsfFi444xijhYV4cJmMJ3n32uH2GHXuL6NT35
    - /ip4/155.138.132.101/tcp/4001/ipfs/QmdVDKX2ZcsfFi444xijhYV4cJmMJ3n32uH2GHXuL6NT35
    - /ip6/::1/tcp/4001/ipfs/QmdVDKX2ZcsfFi444xijhYV4cJmMJ3n32uH2GHXuL6NT35
Qmdui4L3ktDRwQUoPRo6Gy4GjTYE9J7zHXsecNmn9vvzqA | Dharwin Node B | Sees 3 other peers
  > Addresses:
    - /ip4/127.0.0.1/tcp/9096/ipfs/Qmdui4L3ktDRwQUoPRo6Gy4GjTYE9J7zHXsecNmn9vvzqA
    - /ip4/139.180.139.156/tcp/9096/ipfs/Qmdui4L3ktDRwQUoPRo6Gy4GjTYE9J7zHXsecNmn9vvzqA
    - /p2p-circuit/ipfs/Qmdui4L3ktDRwQUoPRo6Gy4GjTYE9J7zHXsecNmn9vvzqA
  > IPFS: QmaQuLNbvXgFyXpzwpNmRwU95Yfj4jsaVvQ59rDBLq3FBK
    - /ip4/127.0.0.1/tcp/4001/ipfs/QmaQuLNbvXgFyXpzwpNmRwU95Yfj4jsaVvQ59rDBLq3FBK
    - /ip4/139.180.139.156/tcp/4001/ipfs/QmaQuLNbvXgFyXpzwpNmRwU95Yfj4jsaVvQ59rDBLq3FBK
    - /ip6/::1/tcp/4001/ipfs/QmaQuLNbvXgFyXpzwpNmRwU95Yfj4jsaVvQ59rDBLq3FBK

Then, when I pinned a file on one peer, I got what looked like a good result:

$ ipfs-cluster-ctl pin add QmaJok6smSMc7rAY3JeE1fDo7Q9VA2xC9WUj9LgCo1aD6S

    QmaJok6smSMc7rAY3JeE1fDo7Q9VA2xC9WUj9LgCo1aD6S :
        > Dharwin Node D  : PINNED | 2019-03-10T18:19:45.599849261Z
        > Dharwin Node A  : PINNED | 2019-03-10T18:19:44.927490977Z
        > Dharwin Node C  : PINNED | 2019-03-10T18:19:44.975235772Z
        > Dharwin Node B  : PINNED | 2019-03-10T18:19:45.047596369Z

On three of my peers, if I check the status of pinned files, I get this:

$ ipfs-cluster-ctl pin ls
QmaJok6smSMc7rAY3JeE1fDo7Q9VA2xC9WUj9LgCo1aD6S |  | PIN | Repl. Factor: -1 | Allocations: [everywhere] | Recursive

BUT, when I check on one peer, I get this cryptic message:

$ ipfs-cluster-ctl pin ls
An error occurred:
  Code: 200
  Message: json: cannot unmarshal object into Go struct field PinSerial.cid of type string

Does anyone know what this means?

2

Pretty sure that means you have mismatching version numbers. I would probably hazard a guess and say your daemons are running 0.10.0 while ipfs-cluster-ctl 0.9.0.

The recent update (0.10.0) changed the response format and I got that error when using older clients on updated daemons.

3

My ifs-cluster-service is 0.10.0, but IPFS-cluster-ctl is 0.9.0, so I’ll try updating that. Thanks!

4

I made that change, but now I am getting this error from all peers, anyone know what this one means?

$ ipfs-cluster-ctl peers ls
An error occurred:
  Code: 200
  Message: input isn't valid multihash

Thanks!

5

hey @dharwin, sorry for these problems. Can you triple check that ipfs-cluster-service --version and ipfs-cluster-ctl --version are the same everywhere?

Can you check the logs of the daemon for errors otherwise?

It is weird that all peers stopped working

6

Hmm I managed to reproduce it when shutting down a peer. It seems one of your peers is offline. We will fix this asap.

7

Tracking here https://github.com/ipfs/ipfs-cluster/issues/715

1 Like
8

For what it’s worth in my own testing i’ve encountered the following:

I don’t seem to be getting this error on a single node cluster. It seems to only occur when running a multi-peer cluster, and having one of the peers go offline.

9

Just confirming that yes, all peers’ ipfs-cluster-service and ipfs-cluster-ctl are the same version, at 0.10.0.

10

Thank you for creating that!

11

Also confirming that after restarting the peer that was down, the cluster seems to be working.