【verbs】ibv_query_qp()
原文:http://www.polarhome.com/service/man/AIX/libs/ofed/ofed_ibv_create_qp.htm
ibv_query_qp()
Contents [hide]
5.00 平均评分 (98% score) - 2 votes
|
描述
ibv_query_qp() 返回QP的属性和当前值。 .
struct ibv_qp_attr 描述QP的属性。
|
struct ibv_qp_attr: 完整的描述
qp_state | QP状态。它可以是下列枚举值之一: IBV_QPS_RESET-重置状态 IBV_QPS_INIT-初始化状态 IBV_QPS_RTR-准备接收状态 IBV_QPS_RTS-准备发送状态 IBV_QPS_SQD-发送队列耗尽状态 IBV_QPS_SQE-发送队列错误状态 IBV_QPS_ERR-错误状态 |
cur_qp_state | Assume that this is the current QP state. This is useful if it is known to the application that the QP state is different from the assumed state by the low-level driver. It can be one of the enumerated values as qp_state. Not relevant for ibv_query_qp() 假设这是当前的QP状态。如果应用程序知道QP状态与底层驱动程序假定的状态不同,这将很有用。它可以是qp_state枚举值之一。与ibv_query_qp()不相关 |
path_mtu | 路径MTU(path MTU)(最大传输单位),即可以在路径中传输的数据包的最大有效负载大小。它可以是下列枚举值之一:
对于UC和RC QP,在需要时,RDMA设备将自动将消息分段为该大小的数据包。 |
path_mig_state | The state of the QP's path migration state machine if supported by the device (IBV_DEVICE_AUTO_PATH_MIG is set in dev_cap.device_cap_flags). It can be one of the following enumerated values: 如果设备支持QP的路径迁移状态机的状态(IBV_DEVICE_AUTO_PATH_MIG在dev_cap.device_cap_flags中设置)。它可以是下列枚举值之一:
|
qkey | 将检查传入消息的Q_Key并可能将其用作传出的Q_Key(如果已设置“发送请求”中q_key的MSB)。仅与UD QP相关 |
rq_psn | RC和UC模式的 QP收到的数据包的包序号(24bit的值) |
sq_psn | 发送包的包序号。(24bit的值) |
dest_qp_num | RC and UC模式的远端QP的number。(24bit的值) 当发送就是发往这个number的QP,接收数据也只接收从这个number的QP发来的数据。 |
qp_access_flags | Allowed access flags of the remote operations for incoming packets of RC and UC QPs. It is either 0 or the bitwise OR of one or more of the following flags: RC和UC QP传入数据包的远程操作的允许访问标志。它是0或以下一个或多个标志的按位或:
|
cap | QP中的Wore Requests数的属性,如下表所述 |
ah_attr | Address vector of the primary path which describes the path information to the remote QP as described in the table below 主路径的地址向量,描述到远程QP的路径的信息,具体描述见下表。 |
alt_ah_attr | Address vector of the alternate path which describes the path information to the remote QP as described in the table below. Can be used only if supported by the device (IBV_DEVICE_AUTO_PATH_MIG is set in dev_cap.device_cap_flags) 替代路径的地址向量,描述了到远程QP的路径信息,如下表所述。仅在设备支持的情况下才能使用(在dev_cap.device_cap_flags中设置了IBV_DEVICE_AUTO_PATH_MIG) |
pkey_index | Primary P_Key index. The value of the entry in the P_Key table that outgoing packets from this QPwill be sent with and incoming packets to this QP will be verified within the Primary path 主P_Key索引。 |
alt_pkey_index | Alternate P_Key index. The value of the entry in the P_Key table that outgoing packets from this QP will be sent with and incoming packets to this QP will be verified within the Alternate path 备用P_Key索引。 |
en_sqd_async_notify | If non-zero, generate the affiliated asynchronous event IBV_EVENT_SQ_DRAINED when the QP state becomes SQD.drained, i.e. the Send Queue is drained. Not relevant for ibv_query_qp() 如果非零,则在QP状态变为SQD.drained(即发送队列被耗尽)时生成关联的异步事件IBV_EVENT_SQ_DRAINED。与ibv_query_qp()不相关 |
sq_draining | 如果已设置,则表明正在进行发送队列排空(draining )。仅当QP处于SQD状态时相关 |
max_rd_atomic | The number of RDMA Reads & atomic operations outstanding at any time that can be handled by this QP as an initiator. Relevant only for RC QPs QP作为启动程序,在任何时候可以处理的RDMA读取和原子操作的数量。仅与RC QP相关 |
max_dest_rd_atomic | The number of RDMA Reads & atomic operations outstanding at any time that can be handled by this QP as a destination. Relevant only for RC QPs |
min_rnr_timer | Minimum RNR NAK Timer Field Value. When an incoming message to this QP should consume a Work Request from the Receive Queue, but not Work Request is outstanding on that Queue, the QP will send an RNR NAK packet to the initiator. It does not affect RNR NAKs sent for other reasons. The value can be one of the following numeric values since those values aren’t enumerated:
最小RNR NAK计时器字段值。当传入消息到达此QP,将消费接收队列中的一个WR (Work Request)。但该队列上未完成工作请求时,QP将向发起方发送RNR NAK数据包。它不会影响由于其他原因发送的RNR NAK。该值可以是以下数字值之一(这些值未被枚举): 仅与RC QP相关。
|
port_num | Primary physical port number associated with this QP |
timeout | The minimum timeout that a QP waits for ACK/NACK from remote QP before retransmitting the packet. The value zero is special value which means wait an infinite time for the ACK/NACK (useful for debugging). For any other value of timeout, the time calculation is: usec. For your convenience, here is the summary of each value and its timeout:
Relevant only to RC QPs |
retry_cnt | A 3 bits value of the total number of times that the QP will try to resend the packets before reporting an error because the remote side doesn't answer in the primary path |
rnr_retry | A 3 bits value of the total number of times that the QP will try to resend the packets when an RNR NACK was sent by the remote QP before reporting an error. The value 7 is special and specify to retry infinite times in case of RNR |
alt_port_num | Alternate physical port number associated with this QP |
alt_timeout | The total number of times that the QP will try to resend the packets before reporting an error because the remote side doesn't answer in the alternate path |
A few caveats about some of the specific values in the QP attributes:
- The value 0 in attr.timeout means waiting infinite time for the ACK or NACK. This means that if any packet in a message is being lost and no ACK or NACK is being sent, no retry will ever occur and the QP will just stop sending data
- The value 7 in attr.rnr_retry means to retry infinite number of times to send the message when RNR Nack is being sent by remote side
The struct ibv_qp_init_attr describes the attributes of the QP's Queues:
|
Here is the full description of struct ibv_qp_init_attr:
qp_context | (optional) User defined value which will be available in qp->qp_context | ||||||
send_cq | A Completion Queue, that was returned from ibv_create_cq(), to be associated with the Send Queue | ||||||
recv_cq | A Completion Queue, that was returned from ibv_create_cq(), to be associated with the Receive Queue | ||||||
srq | (optional) A Shared Receive Queue, that was returned from ibv_create_srq(), that this Queue Pair will be associated with. Otherwise, NULL | ||||||
cap | Attributes of the Queue Pair size, as described in the table below. Upon a successful Queue Pair creation, this structure will hold the actual Queue Pair attributes | ||||||
qp_type | Transport Service Type for requested this QP:
| ||||||
sq_sig_all | The Signaling level of Work Requests that will be posted to the Send Queue in this QP.
|
struct ibv_qp_cap describes the size of the Queue Pair (for both Send and Receive Queues).
|
Here is the full description of struct ibv_qp_cap:
max_send_wr | The maximum number of outstanding Work Requests that can be posted to the Send Queue in that Queue Pair. Value can be [1..dev_cap.max_qp_wr] |
max_recv_wr | The maximum number of outstanding Work Requests that can be posted to the Receive Queue in that Queue Pair. Value can be [1..dev_cap.max_qp_wr]. This value is ignored if the Queue Pair is associated with an SRQ |
max_send_sge | The maximum number of scatter/gather elements in any Work Request that can be posted to the Send Queue in that Queue Pair. Value can be [1..dev_cap.max_sge] |
max_recv_sge | The maximum number of scatter/gather elements in any Work Request that can be posted to the Receive Queue in that Queue Pair. Value can be [1..dev_cap.max_sge]. This value is ignored if the Queue Pair is associated with an SRQ |
max_inline_data | The maximum message size (in bytes) that can be posted inline to the Send Queue. 0, if no inline message is requested |
struct ibv_ah_attr describes the Address Vector of the QP.
|
Here is the full description of struct ibv_ah_attr:
grh | Attributes of the Global Routing Headers (GRH), as described in the table below. This is useful when sending packets to another subnet |
dlid | If the destination is in same subnet, the LID of the port to which the subnet delivers the packets to. If the destination is in another subnet, the LID of the Router |
sl | 4 bits. The Service Level to be used |
src_path_bits | The used Source Path Bits. This is useful when LMC is used in the port, i.e. each port covers a range of LIDs. The packets are being sent with the port's base LID, bitwised ORed with the value of the source path bits. The value 0 indicates the port's base LID is used |
static_rate | A value which limits the rate of packets that being sent to the subnet. This can be useful if the rate of the packet origin is higher than the rate of the destination |
is_global | If this value contains any value other than zero, then GRH information exists in this AH, thus the field grh if valid |
port_num | The local physical port that the packets will be sent from |
struct ibv_global_route describes the values to be used in the GRH of the packets that will be sent when using this AH.
|
这是struct ibv_global_route的完整描述:
dgid | 用于标识数据包目标端口的GID |
flow_label | 20 bits. If this value is set to a non-zero value, it gives a hint for switches and routers with multiple outbound paths that these sequence of packets must be delivered in order, those staying on the same path, so that they won't be reordered. |
sgid_index | An index in the port's GID table that will be used to identify the originator of the packet |
hop_limit | The number of hops (i.e. the number of routers) that the packet is permitted to take before being discarded. This ensures that a packet will not loop indefinitely between routers if a routing loop occur. Each router decrement by one this value at the packet and when this value reaches 0, this packet is discarded. Setting the value to 0 or 1 will ensure that the packet won't leave the local subnet. |
traffic_class | Using this value, the originator of the packets specifies the required delivery priority for handling them by the routers |
attr_mask为RDMA设备的底层驱动程序提供了一个提示,该QP的属性应该被查询。 ibv_query_qp()可能会填充未请求的其他属性。它是0或以下一个或多个标志的按位或:
(attr_mask provides a hint to the low-level driver of the RDMA device which QP attributes should be queried. It is possible that extra attributes, which weren't requested, will be filled by ibv_query_qp(). It is either 0 or the bitwise OR of one or more of the following flags:)
IBV_QP_STATE | Fill the value in attr->qp_state |
IBV_QP_CUR_STATE | Fill the value in attr->cur_qp_state |
IBV_QP_EN_SQD_ASYNC_NOTIFY | Fill the value in attr->en_sqd_async_notify |
IBV_QP_ACCESS_FLAGS | Fill the value in attr->qp_access_flags |
IBV_QP_PKEY_INDEX | Fill the value in attr->pkey_index |
IBV_QP_PORT | Fill the value in attr->port_num |
IBV_QP_QKEY | Fill the value in attr->qkey |
IBV_QP_AV | Fill the value in attr->ah_attr |
IBV_QP_PATH_MTU | Fill the value in attr->path_mtu |
IBV_QP_TIMEOUT | Fill the value in attr->timeout |
IBV_QP_RETRY_CNT | Fill the value in attr->retry_cnt |
IBV_QP_RNR_RETRY | Fill the value in attr->rnr_retry |
IBV_QP_RQ_PSN | Fill the value in attr->rq_psn |
IBV_QP_MAX_QP_RD_ATOMIC | Fill the value in attr->max_rd_atomic |
IBV_QP_ALT_PATH | Fill the value in attr->alt_ah_attr, attr->alt_pkey_index, attr->alt_port_num, attr->alt_timeout |
IBV_QP_MIN_RNR_TIMER | Fill the value in attr->min_rnr_timer |
IBV_QP_SQ_PSN | Fill the value in attr->sq_psn |
IBV_QP_MAX_DEST_RD_ATOMIC | Fill the value in attr->max_dest_rd_atomic |
IBV_QP_PATH_MIG_STATE | Fill the value in attr->path_mig_state |
IBV_QP_CAP | Fill the value in attr->cap |
IBV_QP_DEST_QPN | Fill the value in attr->dest_qp_num |
下表列出在每种状态下服务类型为IBV_QPT_UD 的QP的有效(valid )属性:
State | Valid Attributes |
---|---|
RESET | IBV_QP_STATE |
INIT | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_QKEY |
RTR | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_QKEY |
RTS | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_QKEY, IBV_QP_SQ_PSN |
SQD | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_QKEY, IBV_QP_SQ_PSN |
SQE | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_QKEY, IBV_QP_SQ_PSN |
ERR | IBV_QP_STATE |
下表列出在每种状态下服务类型为IBV_QPT_UC 的QP的有效(valid )属性:
State | Valid Attributes |
---|---|
RESET | IBV_QP_STATE |
INIT | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS |
RTR | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_ALT_PATH |
RTS | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_ALT_PATH, IBV_QP_SQ_PSN, IBV_QP_PATH_MIG_STATE |
SQD | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_ALT_PATH, IBV_QP_SQ_PSN, IBV_QP_PATH_MIG_STATE |
SQE | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_ALT_PATH, IBV_QP_SQ_PSN, IBV_QP_PATH_MIG_STATE |
ERR | IBV_QP_STATE |
下表列出在每种状态下服务类型为IBV_QPT_RC的QP的有效(valid )属性:
State | Valid Attributes |
---|---|
RESET | IBV_QP_STATE |
INIT | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS |
RTR | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_MAX_DEST_RD_ATOMIC, IBV_QP_MIN_RNR_TIMER, IBV_QP_ALT_PATH |
RTS | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_MAX_DEST_RD_ATOMIC, IBV_QP_MIN_RNR_TIMER, IBV_QP_ALT_PATH, IBV_QP_SQ_PSN, IBV_QP_TIMEOUT, IBV_QP_RETRY_CNT, IBV_QP_RNR_RETRY, IBV_QP_MAX_QP_RD_ATOMIC, IBV_QP_PATH_MIG_STATE |
SQD | IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_ACCESS_FLAGS, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN, IBV_QP_MAX_DEST_RD_ATOMIC, IBV_QP_MIN_RNR_TIMER, IBV_QP_ALT_PATH, IBV_QP_SQ_PSN, IBV_QP_TIMEOUT, IBV_QP_RETRY_CNT, IBV_QP_RNR_RETRY, IBV_QP_MAX_QP_RD_ATOMIC, IBV_QP_PATH_MIG_STATE |
ERR | IBV_QP_STATE |
参数
Name | Direction | Description |
---|---|---|
qp | in | 从ibv_create_qp()返回的QP |
attr | out | 将被QP的当前属性填充 |
attr_mask | in | 要查询的QP属性的掩码 |
init_attr | out | 将被QP的队列(QP's)属性填充 |
返回值
Value | Description |
---|---|
0 | 成功 |
errno | 失败 |
ENOMEM | 没有足够的资源来完成此操作 |
例子
查询QP以获取其状态:
|
常见问题
Calling every time to ibv_query_qp() when I need a QP attribute takes time, can I cache some of the attributes?
Actually, yes. Most of the attributes are constants unless they were changed using ibv_modify_qp(). The following fields in the QP attribute structure may change: qp_state, path_mig_state, sq_draining, ah_attr, pkey_index, port_num, timeout.
Can I specify exactly which attributes will be filled by ibv_query_qp()?
No. The parameter attr_mask behaves as a hint, and the low-lever driver of the RDMA device may (and most of the time, will) fill many more attributes than the ones that were requested in attr_mask.
Do all of the QP attributes are valid?
No. The valid QP attributes depends on the QP state.
How to get the QP state?
One should call ibv_query_qp() with, at least, the flag IBV_QP_STATE set.