Today’s research.
I’ll write again bold for received messages, italic for transmitted.
I’ll omit in the first part the preceeding binary byte of message length for clarity. Also at the end it’s always \n which I’ll also omit for the same reason. I’m using still --disable-transport-encryption.
So the handshake insofar goes like this:
/multistream/1.0.0
/multistream/1.0.0
/plaintext/1.0.0
/plaintext/1.0.0
/multistream/1.0.0
/multistream/1.0.0
/mplex/6.7.0
/mplex/6.7.0
After you execute the last line you get a bunch of data which is in a different format now. The format is found here: https://github.com/libp2p/specs/tree/master/mplex
Mplex in short is a protocol for stream multiplexing, which in simple terms is how to group a bunch of requests into a single connection.
First a bit of intro:
The format is such, that first you get an extra header byte, then the rest is like before. Note that there are then two data length bytes one after the other, because of “nesting” of protocols i.e. one in the other. One comes from mplex the other from multistream.
- 1st byte is the header byte.
- 2nd byte is the data length byte.
- The data after that is data length bytes long.
Or in graphical form:
[header] [strlen(data) in binary] [data]
The header is in a special binary format where the 3 righter-most bits are type (or flag) of the header and lefter-most 5 bits are the ID of the stream.
Visually this would be like this, where D stands for ID, T stands for type:
D D D D D T T T
The left part, ID, is easy to understand. The 5 bits just represent a number, which is used to identify a stream. They are used for identifying streams. 5 bits gives us 2^5 = 32 possible streams.
A few real examples, where I’ve replaced D with actual values. T aren’t important for now.
0 0 0 0 0 T T T is for example stream ID 0
0 0 0 0 1 T T T is for example stream ID 1
0 0 0 1 0 T T T is for example stream ID 2
When you get the header byte, the most easy way to extract the stream ID is to use bitwise shifting operators, which shift bits by X places. In this case we need to shift by 3 places, that’s why we’ll use header >> 3.
That gives us for the first 3 examples:
0 0 0 0 0 0 0 0 for stream ID 0
0 0 0 0 0 0 0 1 for stream ID 1
0 0 0 0 0 0 1 0 for stream ID 2
Now we’ll focus on the Ts, i.e. type or flag of the header.
There are 3 bits, that gives us 2^3 = 8 possible flags, but just 7 are defined in the protocol.
D D D D D 0 0 0 (0) NewStream initiating a new stream, that one that sends this becomes the initiator
D D D D D 0 0 1 (1) MessageReceiver data sent by the one who didn’t initiate the stream
D D D D D 0 1 0 (2) MessageInitiator data sent by the one who initiated the stream
D D D D D 0 1 1 (3) CloseReceiver closing the stream by the one who didn’t initiate the stream
D D D D D 1 0 0 (4) CloseInitiator closing the stream by the one who initiated the stream
D D D D D 1 0 1 (5) ResetReceiverreseting the stream by the one who didn’t initiate the stream
D D D D D 1 1 0 (6) ResetInitiator reseting the stream by the one who did initiate the stream
We can see a pattern where the rightermost bit indicates 0 for the initiator and 1 for the receiver. This also manifests in the decimal numbers so that initiator has always even numbers and receiver odd ones.
To get the type (flag) from the header we use bitwise and of the number 7, which is 111: header & 0x07
So what did I get?
IPFS, after doing the /mplex/6.7.0 handshake, returns 3 streams. This is manifested with getting the following things all at once:
initiate stream 0
initiate stream 1
initiate stream 2
stream0: /multistream/1.0.0
stream1: /multistream/1.0.0
stream2: /multistream/1.0.0
I’ll present it here also in raw, but I’ll present it like this:
1st byte in binary (header)
2nd byte in hexadecimal (length)
3rd byte in ASCII character (stream name)
initiate stream 0: 00000000 0x01 0
initiate stream 1: 00001000 0x01 1
initiate stream 2: 00010000 0x01 2
Now I’ll do it the same as before, just that the 3rd part will be length byte of the nested protocol + ASCII data (stream data). \n stands for new line character.
stream 0: 00000010 0x14 0x13 /multistream/1.0.0\n
stream 1: 00001010 0x14 0x13 /multistream/1.0.0\n
stream 2: 00010010 0x14 0x13 /multistream/1.0.0\n
As we can see we first got initiate stream type headers, i.e. type 0 with stream IDs 0, 1, 2. Then next we got message initiator type headers, i.e. type 2 with IDs 0, 1, 2.
We see we have first 0x14 which is 20 i.e. strlen([data byte] + /multistream/1.0.0\n) and then the next byte is 0x13 which is 19 i.e. strlen(/multistream/1.0.0\n).