I used helia to run local ipfs node using electron, How to persist nodeId

smakintel · March 27, 2024, 3:33am

Every time I start and Stop my electron app, Helia creates new Node with new peerId (NodeId) is there any way to use the same peerId.

I use local storage (windows path) , blockstore , datastore.

also is there any way to get total Size of Pinned content and How many peers accessed pinned content.

@SgtPooki please help thanks

Appreciate your reply.

danieln · March 27, 2024, 1:36pm

Can you share the code where you instantiate Helia and the data store?

There are two sizes:

Size of all the blocks of the pinned content (slightly more than the raw files due to the unixfs metadata)
Size of the pinned files

You can enumerate all your pinned blocks with Pins.ls and call the stat on each which returns both relevant sizes: helia/packages/unixfs/src/index.ts at main · ipfs/helia · GitHub

There might be some nuance around recursive pins and underlying blocks, but that should be enough for you to calculate the size.

That’s a tricky one, given that this isn’t tracked by Helia. There might be a clever way to hook into block requests via Bitswap and track this, but I don’t have enough knowledge of the js-bitswap implementation to answer this.

smakintel · March 27, 2024, 1:58pm

Thanks for your prompt reply. this is my code.

/* eslint-disable no-console */

import { noise } from '@chainsafe/libp2p-noise' // for encrypting connections 
import { yamux } from '@chainsafe/libp2p-yamux'
import { unixfs } from '@helia/unixfs'
import { bootstrap } from '@libp2p/bootstrap'
import { identify } from '@libp2p/identify'
import { tcp } from '@libp2p/tcp' // Required to enable TCP connection for peer to peer communication
import { FsBlockstore } from 'blockstore-fs'
import { FsDatastore } from 'datastore-fs'
import { createHelia } from 'helia'
import { createLibp2p } from 'libp2p'



// the blockstore is where we store the blocks that make up files
const blockstore = new FsBlockstore('block-store')

// application-specific data lives in the datastore
const datastore = new FsDatastore('data-store')

// libp2p is the networking layer that underpins Helia
const libp2p = await createLibp2p({
  datastore,
  blockstore,
  addresses: {
    listen: [
      // add a listen address (localhost) to accept TCP connections on a random port
      '/ip4/127.0.0.1/tcp/0'
    ]
  },
  transports: [
    tcp()
  ],
  connectionEncryption: [
    noise()
  ],
  streamMuxers: [
    yamux()
  ],
  peerDiscovery: [
    bootstrap({
      list: [
          '/ip4/********/tcp/4001/p2p/12D3KooWR6PtoaHyoQZ6tJmjEdGHCNA4LMXctrvnph4DNYYLPprb'
          ]
    })
  ],
  services: {
    identify: identify()
  }
})

console.log(`Node started with id ${libp2p.peerId.toString()}`)
// Save the generated peer ID
//await savePeerId(libp2p.peerId.toString());

// print out listening addresses
console.log('listening on addresses:')
libp2p.getMultiaddrs().forEach((addr) => {
  console.log(addr.toString())
})

// Listen for new connections to peers
libp2p.addEventListener("peer:connect", (evt) => {
  const connection = evt.detail;
  console.log(`Connected to ${connection.toString()}`);
  
});

libp2p.addEventListener('peer:discovery', (peerId) => {
  // No need to dial, autoDial is on
      console.log('Discovered:', peerId.toString())
  })

// Listen for peers disconnecting
libp2p.addEventListener("peer:disconnect", (evt) => {
  const connection = evt.detail;
  console.log(`Disconnected from ${connection.toCID().toString()}`);
});


const helia = await createHelia({
  libp2p,
  blockstore,
  datastore

 
  
})


//console.log(await helia)


// create a filesystem on top of Helia, in this case it's UnixFS
const fs = unixfs(helia)

// we will use this TextEncoder to turn strings into Uint8Arrays
const encoder = new TextEncoder()

// add the bytes to your node and receive a unique content identifier
const cid = await fs.addBytes(encoder.encode('Hello Kava'))

console.log('Added file:', cid.toString())

smakintel · April 4, 2024, 2:22am

Hi @danieln kindly check when you have time

danieln · April 4, 2024, 5:23am

It’s hard to know without knowing the exact versions of dependencies. If you could create a reproduction repo, that would help me help you

smakintel · April 8, 2024, 3:01pm

Hi,

thank you for the response , this is the content of my package.json

{
  "name": "helia-101",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "description": "Getting started with Helia",
  "license": "MIT",
  "dependencies": {
    "@chainsafe/libp2p-noise": "^15.0.0",
    "@chainsafe/libp2p-yamux": "^6.0.1",
    "@helia/unixfs": "^3.0.0",
    "@libp2p/bootstrap": "^10.0.12",
    "@libp2p/identify": "^1.0.14",
    "@libp2p/tcp": "^9.0.12",
    "blockstore-core": "^4.1.0",
    "blockstore-fs": "^1.1.10",
    "datastore-core": "^9.1.1",
    "datastore-fs": "^9.1.8",
    "helia": "^4.0.1",
    "libp2p": "^1.2.0"
  },
  "devDependencies": {
    "test-ipfs-example": "^1.0.0"
  }
}

smakintel · April 10, 2024, 12:15am

I don’t mind using any simple example . Which can demonstrate example on how to persist nodeId in helia.

This is blocker for my project. I almost done with 90% of my project. Hope to launch soon after fixing this blocker.

danieln · April 10, 2024, 10:03am

Thanks for sharing.

It can be quite time consuming to create a reproduction for such errors. Given this, we ask for a reproduction repo to make it easier for us to debug the problem and find a resolution if needed.

smakintel · April 10, 2024, 1:15pm

Hi I have created this public repo with required source code.
This has inde.js file and package.json file.

Output of script.

hector · April 10, 2024, 1:43pm

Have you tried doing what the docs say? js-libp2p/doc/CONFIGURATION.md at main · libp2p/js-libp2p · GitHub

SgtPooki · April 11, 2024, 6:01pm

using an fsDataStore like you are should be saving the peerId created on first instantiation of libp2p and then re-loading that on subsequent loads.

using electron makes me think that somehow electron is removing those files when started/stopped?

I would need to test to be sure, but another way you can explicitly persist the peerId is by passing it directly in the libp2p config:

{
  // ...
  peerId: savedPeerId ?? await createEd25519PeerId()
  // ...
}

if you were to create the peerId explicitly yourself, and save it to a known location, and reload it, that should solve the problem.

with all that said, using a persistent datastore used to do this for you automatically (and still does AFAIK), but you can look at js-libp2p/doc/migrations/v0.46-v1.0.0.md at c5003d40207bc3be15645b5af93f8dec1fd5d55b · libp2p/js-libp2p · GitHub and links within for further details

Here is an extremely tiny example showing that libp2p and helia will persist a peerId if it’s saved to the datastore: https://codepen.io/SgtPooki/pen/VwNdWBQ?editors=0011

SgtPooki · April 11, 2024, 6:16pm

Thanks so much. this made it really easy to see what’s happening.

I think because you’re constructing libp2p yourself, some of the required loading of peerId is not being handled (talked about in the link I posted in my comment above)

I pushed up a PR to your repo that fixes the issue, fix: persistent peer ids by SgtPooki · Pull Request #1 · smakintel/ipfs-helia · GitHub.

basically the main fix here is that we construct the libp2p instance with helia, and we get the same peerid:

╭─    ~/code/testing/smakintel/ipfs-helia    fix/persistent-peerid ?1 ────────────────────────────────────────────────────────────────────────── 23s   ▼  21.7.0   11:12:16  ─╮
╰─ ✔ ❯ node index.js
Added file: bafkreih7eug2oqxx7ft427q4fcce6lnl73q6kmtzaxeldllx266mxr6os4
Node started with id 12D3KooWE4FgCf3o52J9qRSNftRukp9X9buzpPSmznbP4EieUMfM
Added file contents: Hello World 201

╭─    ~/code/testing/smakintel/ipfs-helia    fix/persistent-peerid !1 ?1 ─────────────────────────────────────────────────────────────────────────────── ▼  21.7.0   11:15:45  ─╮
╰─ ✔ ❯ node index.js
Added file: bafkreih7eug2oqxx7ft427q4fcce6lnl73q6kmtzaxeldllx266mxr6os4
Node started with id 12D3KooWE4FgCf3o52J9qRSNftRukp9X9buzpPSmznbP4EieUMfM
Added file contents: Hello World 201

╭─    ~/code/testing/smakintel/ipfs-helia    fix/persistent-peerid !1 ?1 ─────────────────────────────────────────────────────────────────────────────── ▼  21.7.0   11:15:49  ─╮
╰─ ✔ ❯ node index.js
Added file: bafkreih7eug2oqxx7ft427q4fcce6lnl73q6kmtzaxeldllx266mxr6os4
Node started with id 12D3KooWE4FgCf3o52J9qRSNftRukp9X9buzpPSmznbP4EieUMfM
Added file contents: Hello World 201

SgtPooki · April 11, 2024, 6:19pm

the keychain logic you would need to do yourself is being handled by helia in helia/packages/helia/src/utils/libp2p.ts at b67ac5f16eca1df5534c985045250bdb334a85cf · ipfs/helia · GitHub

smakintel · April 26, 2024, 2:40pm

Thanks @SgtPooki It was really helpful. I will try these recommendations.

smakintel · April 26, 2024, 2:41pm

Appreciate your time and reply, This helped me to solve peer id persistent issue. Thank you so much

smakintel · April 26, 2024, 2:42pm

Hi Hector

Appreciate your reply, Sorry I couldn’t reply on time.

Below example provided by @SgtPooki helped me solve the peer Id persistent issue
Here is an extremely tiny example showing that libp2p and helia will persist a peerId if it’s saved to the datastore: https://codepen.io/SgtPooki/pen/VwNdWBQ?editors=0011

Topic		Replies	Views
How to retrieve content uploaded via Helia using the IPFS gateway? Helia	16	3959	October 28, 2023
Total newbir and not afraid to admit it	0	287	November 27, 2020
Check if nodes the same Help js-ipfs	1	439	April 3, 2018
Asking other nodes to pin stuff and the state of pubsub Help go-ipfs , ipfs-cluster	1	457	December 18, 2018
Pinning vs. adding content to IPFS Help go-ipfs , files	2	376	July 10, 2021

I used helia to run local ipfs node using electron, How to persist nodeId

Related Topics