RoCEv2 CNP Packet Format Example

https://community.mellanox.com/s/article/rocev2-cnp-packet-format-example

 

2018-12-5•Knowledge Article

 

rocev2-cnp-packet-format-example

This post describes the RoCE CNP packet fields, with examples.

 

References

• Understanding RoCEv2 Congestion Management
• HowTo Configure RoCE Congestion Control for Windows 2012
• Annex 17 RoCEv2
• Download Wireshark with RoCEv2 Support

 

 

RoCEv2 CNP Headers

Here is the RoCEv2 CNP Packet format as described in the standard Annex 17.9.3 RoCEv2

 

 

Here is a wireshark example of this packet:

 

Note: there is an option to Download Wireshark with RoCEv2 Support, in this case the parsing will be easier.

 

The CNP data starts in the UDP payload which is the BTH InfiniBand header.

 

Here are the fields of the BTH header aligned with the example above in hexa 0x81 00 ff ff 40 00 00 d2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e4 2d ad 81:

 

Field	Size (in bits)	Description	Example (as above)
OpCode	8	Indicates the IBA packet type. The OpCode also specifies which extension headers follow the Base Transport Header	0x81 = 10000001b   As described above, the CNP packet will have this OpCode.
Solicited Event (SE)	1	Indicates that an event should be generated by the responder	0
MigReq (M)	1	Used to communicate migration state	0
Pad Count (PadCnt)	2	Indicates how many extra bytes are added to the payload to align to a 4 byte boundary	0
Transport Header Version (TVer)	4	Indicates the version of the IBA Transport Headers.	0
Partition Key (P_KEY)	16	Indicates which logical Partition is associated with this packet	0xffff
F/Res1* (F/R)	1	F (FECN): 0 indicates that a FECN indication was not received 1 indicates that the packet went through a point of congestion Res1*: Transmitted as 0, ignored on receive. This field is not included in the invariant CRC.	0
B/Res1* (B/R)	1	B (BECN): 0 the packet did not go through a point of congestion or went through a point of congestion but was not marked 1 indicates that the packet indicated by this header was subject to forward congestion. The B bit is set in an ACK or CN BTH Res1*: Transmitted as 0, ignored on receive. This field is not included in the invariant CRC. see 7.8 CRCs on page 207 for details.	1   Note: this is part of the 0x40 octet (01000000b).
Reserved (variant)	6	Transmitted as 0, ignored on receive. This field is not included in the invariant CRC.	0
Destination QP (DestQP)	24	Indicates the Work Queue Pair (QP) Number at the destination.	0x0000d2 = 210
Acknowledge Request (A)	1	Used to indicate that an acknowledge (for this packet) should be scheduled by the responder.	0
Reserved	7	Transmitted as 0, ignored on receive. This field is included in the invariant CRC.	0
Packet Sequence Number (PSN)	24	Used to detect a missing or duplicate Packet.	0
Padding	16	All zeros	0
iCRC	4	iCRC checksum	0xe42dad81