Hi! I'm on the go-ipfs team and work on the data transfer subsystem Bitswap (though fairly recently). I'd be happy to provide some assistance on the slowness vis a vis downloading. I'd love to get some specifics about your setup and then maybe we can go ahead and file a Github issue if there if it's a bug in the code.
Just curious are you guys both running IPFS 0.4.18 (the most recent released version)? Are either of you building from source? (i.e. master on the IPFS github repo)?
I tried replicating on my machine but right now I have the same connection issues as you started off with and can't diagnose the slowness as a result. (i.e. @john1deer I can't ping your machine - I can ping the relay you gave but it won't let me swarm connect to your peer).