IPFS Daemon slows regular browsing to a crawl

Help
21

I have not explicitly found the issue in terms of a sure fire way to reproduce it and then solve it.

However, I have apparently been running the IPFS daemon on my local machine for over a week and I haven’t noticed.

One thing that I can think of that has changed is that I’m now using systemctl to manage the daemon instead of running it in a background tmux session.

I did write up notes on how to install the deamon as a service:

    # https://gist.github.com/pstehlik/9efffa800bd1ddec26f48d37ce67a59f
    # https://www.maxlaumeister.com/u/run-ipfs-on-boot-ubuntu-debian/
    # https://linuxconfig.org/how-to-create-systemd-service-unit-in-linux#:~:text=There%20are%20basically%20two%20places,%2Fetc%2Fsystemd%2Fsystem%20.
    SERVICE_DPATH=/etc/systemd/system
    SERVICE_FPATH=$SERVICE_DPATH/ipfs.service
    IPFS_EXE=$(which ipfs)

    # TODO: This will depend on how you initialized IPFS
    #IPFS_PATH=/data/ipfs
    #IPFS_PATH=$HOME/.ipfs

    echo "IPFS_EXE = $IPFS_EXE"
    echo "SERVICE_FPATH = $SERVICE_FPATH"
    echo "IPFS_PATH = $IPFS_PATH"

    sudo_writeto $SERVICE_FPATH "
        [Unit]
        Description=IPFS daemon
        After=network.target
        [Service]
        Environment=\"IPFS_PATH=$IPFS_PATH\"
        User=$USER
        ExecStart=${IPFS_EXE} daemon
        [Install]
        WantedBy=multiuser.target
        "
    #sudo systemctl daemon-reload
    sudo systemctl start ipfs
    sudo systemctl status ipfs

as part of my setup ipfs script: local/setup_ipfs.sh at main · Erotemic/local · GitHub

22

At first glance, that make absolutely no sense as a fix. (however I’ve been wrong many time saying that)
And we have cases where the deamon do the same thing using systemd so … (it’s not that)

AFAIK it’s a bitswap issue and we will be working on that for go-ipfs0.15~0.18 (hopefully, soonTM).

23

Well this is so cool! Running as a service is much stable I think. Will try it later. Thanks for sharing.

24

I think my observation on 2022-05-22 was a coincidence. The problem is not resolved.

I’ve been noticing my browsing running very sluglishly again, and using speedtest-cli I measured a very healthy up / down speed of:

Retrieving speedtest.net configuration...
Testing from <ISP> ...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by WebNX Inc (New York, NY) [218.48 km]: 52.959 ms
Testing download speed................................................................................
Download: 364.57 Mbit/s
Testing upload speed......................................................................................................
Upload: 19.65 Mbit/s

But my browsing was still very slow. Then I remembered I had started up my IPFS daemon via systemctl a week ago to pin some files, and lo and behold I never had shut it down: systemctl status ipfs.service returned:

● ipfs.service - IPFS daemon
     Loaded: loaded (/etc/systemd/system/ipfs.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2023-07-15 16:14:14 EDT; 6 days ago
   Main PID: 3143148 (ipfs)
      Tasks: 42 (limit: 154293)
     Memory: 745.6M
        CPU: 8h 2min 27.312s
     CGroup: /system.slice/ipfs.service
             └─3143148 /home/joncrall/.local/bin/ipfs daemon

After stopping the service with sudo systemctl stop ipfs.service, all of the pages that were struggling to load immediately loaded all of their content (twitter, reddit, and an airflow server). I reran speedtest-cli and got comparable results:

Retrieving speedtest.net configuration...
Testing from <ISP>...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by WebNX Inc (New York, NY) [218.48 km]: 29.759 ms
Testing download speed................................................................................
Download: 369.61 Mbit/s
Testing upload speed......................................................................................................
Upload: 19.67 Mbit/s

My advertised speeds from my ISP are 500Mbit/s down and 20Mbit/s up, but I have multiple machines on my network, so not hitting the full 500Mbit/s is normal.

Anyway, this issue is very annoying. I would really like to just leave the IPFS daemon running and just have it exist in the background, but it really interferes with day-to-day productivity.

Any suggestions for debugging or quantitatively identifying what could be causing this would be very helpful to both me and I think the larger IPFS userbase.

25

Are you using the accelerated DHT client and are you noticing the problem during the hourly DHT scan?

If the answer is yes, your problem is likely to be that the router’s state table isn’t large enough to handle the rate at which the daemon opens connections during the scan.

A few months ago, I moved to a different state and had to change ISPs as a result. The ISP provided router caused the same issue for me as well. I replaced that router with a pfsense router where I have a lot more control (and which has a very large state table), and the problem went away.

Of course, not using the accelerated DHT client also solves the issue, but causes other problems if you are hosting a lot of blocks.

26

I haven’t enabled anything outside of the defaults. Running ipfs config show | grep DHT I get:

    "AcceleratedDHTClient": false,

So I don’t think that’s the issue. The particular node (my main machine) that is sluggish is also not open to the WAN (not sure if that makes a difference). I do have a node that is on the WAN, but I don’t notice anything unless the main machine has its ipfs daemon running.