iperf3在OpenShift 4上的网络测试
有账号的同学可以去访问一下这个
https://access.redhat.com/articles/5233541
话不多说先构建一个iperf3的镜像，话说在rhel下真是麻烦，repolist一个没有，然后直接切换到fedora了搞定了。
[lab-user@bastion ~]$ cat Dockerfile FROM fedora RUN yum update -y RUN yum install -y iperf3 ENTRYPOINT trap : TERM INT; sleep infinity & wait # Listen for kill signals and exit quickly.
$ podman build -t quay.io/jnie/iperf . $ podman push quay.io/jnie/iperf 对了还需要先登录一下 podman login -u="jnie" -p="8TIlF4K0Zjo2ds1fHHU+Bn9q+YLVy1404EXmdUa1c+wDz4yjZ/cJ6xxxxxxx/tt" quay.io
Pod与Pod间的带宽测试
部署两个Pod然后进行测试
oc adm policy add-scc-to-user -z default privileged
oc label --overwrite ns myproject \ pod-security.kubernetes.io/enforce=baseline \ pod-security.kubernetes.io/enforce-version=v1.24
[lab-user@bastion ~]$ cat iperf1.yaml apiVersion: v1 kind: Pod metadata: name: iperf-server labels: role: server spec: containers: - name: server image: quay.io/jnie/iperf --- apiVersion: v1 kind: Pod metadata: name: iperf-client spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: role operator: In values: - server topologyKey: kubernetes.io/hostname containers: - name: client image: quay.io/jnie/iperf
[lab-user@bastion ~]$ oc get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES iperf-client 1/1 Running 0 15s 10.130.0.25 ip-10-0-162-179.us-east-2.compute.internal <none> <none> iperf-server 1/1 Running 0 15s 10.131.0.35 ip-10-0-139-112.us-east-2.compute.internal <none> <none>
[lab-user@bastion ~]$ oc exec -it iperf-server -- iperf3 -i 5 -s ----------------------------------------------------------- Server listening on 5201 (test #1) ----------------------------------------------------------- Accepted connection from 10.130.0.25, port 35024 [ 5] local 10.131.0.35 port 5201 connected to 10.130.0.25 port 35040 [ ID] Interval Transfer Bitrate [ 5] 0.00-5.00 sec 2.42 GBytes 4.16 Gbits/sec [ 5] 5.00-10.01 sec 2.44 GBytes 4.19 Gbits/sec [ 5] 10.01-15.01 sec 2.49 GBytes 4.28 Gbits/sec [ 5] 15.01-20.01 sec 2.51 GBytes 4.32 Gbits/sec [ 5] 20.01-25.01 sec 2.51 GBytes 4.32 Gbits/sec [ 5] 25.01-30.01 sec 2.53 GBytes 4.35 Gbits/sec [ 5] 30.01-35.01 sec 2.53 GBytes 4.35 Gbits/sec [ 5] 35.01-40.01 sec 2.55 GBytes 4.38 Gbits/sec [ 5] 40.01-45.01 sec 2.52 GBytes 4.32 Gbits/sec [ 5] 45.01-50.01 sec 2.51 GBytes 4.31 Gbits/sec [ 5] 50.01-55.01 sec 2.52 GBytes 4.34 Gbits/sec [ 5] 55.01-60.01 sec 2.54 GBytes 4.36 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-60.01 sec 30.1 GBytes 4.31 Gbits/sec receiver ----------------------------------------------------------- Server listening on 5201 (test #2) ----------------------------------------------------------- [lab-user@bastion ~]$ oc exec -it iperf-client -- iperf3 -i 5 -t 60 -c 10.131.0.35 Connecting to host 10.131.0.35, port 5201 [ 5] local 10.130.0.25 port 35040 connected to 10.131.0.35 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-5.00 sec 2.42 GBytes 4.16 Gbits/sec 59 1.06 MBytes [ 5] 5.00-10.00 sec 2.44 GBytes 4.20 Gbits/sec 0 1.51 MBytes [ 5] 10.00-15.00 sec 2.49 GBytes 4.28 Gbits/sec 9 3.05 MBytes [ 5] 15.00-20.00 sec 2.51 GBytes 4.32 Gbits/sec 19 3.05 MBytes [ 5] 20.00-25.00 sec 2.51 GBytes 4.32 Gbits/sec 19 3.05 MBytes [ 5] 25.00-30.00 sec 2.53 GBytes 4.35 Gbits/sec 12 3.05 MBytes [ 5] 30.00-35.00 sec 2.53 GBytes 4.35 Gbits/sec 25 3.05 MBytes [ 5] 35.00-40.00 sec 2.55 GBytes 4.38 Gbits/sec 16 3.05 MBytes [ 5] 40.00-45.00 sec 2.52 GBytes 4.32 Gbits/sec 27 3.05 MBytes [ 5] 45.00-50.00 sec 2.51 GBytes 4.31 Gbits/sec 23 3.05 MBytes [ 5] 50.00-55.00 sec 2.52 GBytes 4.34 Gbits/sec 17 3.05 MBytes [ 5] 55.00-60.00 sec 2.54 GBytes 4.36 Gbits/sec 15 3.05 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-60.00 sec 30.1 GBytes 4.31 Gbits/sec 241 sender [ 5] 0.00-60.01 sec 30.1 GBytes 4.31 Gbits/sec receiver iperf Done.
换个参数，P=4
[lab-user@bastion ~]$ oc exec -it iperf-client -- iperf3 -i 5 -t 60 -c 10.131.0.35 -f Mbits -P 4 Connecting to host 10.131.0.35, port 5201 [ 5] local 10.130.0.25 port 51318 connected to 10.131.0.35 port 5201 [ 7] local 10.130.0.25 port 51328 connected to 10.131.0.35 port 5201 [ 9] local 10.130.0.25 port 51342 connected to 10.131.0.35 port 5201 [ 11] local 10.130.0.25 port 51346 connected to 10.131.0.35 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-5.00 sec 841 MBytes 168 MBytes/sec 5 838 KBytes [ 7] 0.00-5.00 sec 819 MBytes 164 MBytes/sec 39 717 KBytes [ 9] 0.00-5.00 sec 831 MBytes 166 MBytes/sec 39 916 KBytes [ 11] 0.00-5.00 sec 2.28 GBytes 468 MBytes/sec 125 1002 KBytes [SUM] 0.00-5.00 sec 4.72 GBytes 966 MBytes/sec 208 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 5.00-10.00 sec 848 MBytes 169 MBytes/sec 40 1.34 MBytes [ 7] 5.00-10.00 sec 822 MBytes 164 MBytes/sec 46 951 KBytes [ 9] 5.00-10.00 sec 831 MBytes 166 MBytes/sec 6 1.14 MBytes [ 11] 5.00-10.00 sec 2.25 GBytes 460 MBytes/sec 70 951 KBytes [SUM] 5.00-10.00 sec 4.69 GBytes 960 MBytes/sec 162 - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 10.00-15.00 sec 834 MBytes 167 MBytes/sec 23 1.16 MBytes [ 7] 10.00-15.00 sec 823 MBytes 165 MBytes/sec 21 1.16 MBytes [ 9] 10.00-15.00 sec 842 MBytes 169 MBytes/sec 23 1.87 MBytes [ 11] 10.00-15.00 sec 2.27 GBytes 464 MBytes/sec 21 1.24 MBytes [SUM] 10.00-15.00 sec 4.71 GBytes 964 MBytes/sec 88 ==============Server=============== Server listening on 5201 (test #2) ----------------------------------------------------------- Accepted connection from 10.130.0.25, port 51308 [ 5] local 10.131.0.35 port 5201 connected to 10.130.0.25 port 51318 [ 8] local 10.131.0.35 port 5201 connected to 10.130.0.25 port 51328 [ 10] local 10.131.0.35 port 5201 connected to 10.130.0.25 port 51342 [ 12] local 10.131.0.35 port 5201 connected to 10.130.0.25 port 51346 [ ID] Interval Transfer Bitrate [ 5] 0.00-5.00 sec 838 MBytes 1.41 Gbits/sec [ 8] 0.00-5.00 sec 816 MBytes 1.37 Gbits/sec [ 10] 0.00-5.00 sec 828 MBytes 1.39 Gbits/sec [ 12] 0.00-5.00 sec 2.28 GBytes 3.92 Gbits/sec [SUM] 0.00-5.00 sec 4.70 GBytes 8.08 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 5.00-10.00 sec 846 MBytes 1.42 Gbits/sec [ 8] 5.00-10.00 sec 821 MBytes 1.38 Gbits/sec [ 10] 5.00-10.00 sec 830 MBytes 1.39 Gbits/sec [ 12] 5.00-10.00 sec 2.25 GBytes 3.86 Gbits/sec [SUM] 5.00-10.00 sec 4.68 GBytes 8.05 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 10.00-15.00 sec 834 MBytes 1.40 Gbits/sec [ 8] 10.00-15.00 sec 823 MBytes 1.38 Gbits/sec [ 10] 10.00-15.00 sec 843 MBytes 1.41 Gbits/sec [ 12] 10.00-15.00 sec 2.26 GBytes 3.89 Gbits/sec [SUM] 10.00-15.00 sec 4.71 GBytes 8.08 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 10.00-15.00 sec 834 MBytes 1.40 Gbits/sec [ 8] 10.00-15.00 sec 823 MBytes 1.38 Gbits/sec [ 10] 10.00-15.00 sec 843 MBytes 1.41 Gbits/sec [ 12] 10.00-15.00 sec 2.26 GBytes 3.89 Gbits/sec [SUM] 10.00-15.00 sec 4.71 GBytes 8.08 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-15.00 sec 3.05 GBytes 1.75 Gbits/sec receiver [ 8] 0.00-15.00 sec 2.99 GBytes 1.71 Gbits/sec receiver [ 10] 0.00-15.00 sec 3.05 GBytes 1.75 Gbits/sec receiver [ 12] 0.00-15.00 sec 8.43 GBytes 4.83 Gbits/sec receiver [SUM] 0.00-15.00 sec 17.5 GBytes 10.0 Gbits/sec receiver iperf3: the client has terminated
Node与Node之间的带宽测试
节点与节点之前的通讯，因为是rhcos，所以创建两个pod, 然后通过hostnetwork测试
oc label --overwrite ns myproject \ pod-security.kubernetes.io/enforce=privileged \ pod-security.kubernetes.io/enforce-version=v1.24
oc adm policy add-scc-to-user -z default privileged oc adm policy add-scc-to-user hostnetwork -z default
[lab-user@bastion ~]$ cat iperf-hostnetwork.yaml apiVersion: v1 kind: Pod metadata: name: iperf-server labels: role: server spec: hostNetwork: true containers: - name: server image: quay.io/jnie/iperf --- apiVersion: v1 kind: Pod metadata: name: iperf-client spec: hostNetwork: true affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: role operator: In values: - server topologyKey: kubernetes.io/hostname containers: - name: client image: quay.io/jnie/iperf
[lab-user@bastion ~]$ oc create -f iperf-hostnetwork.yaml pod/iperf-server created pod/iperf-client created [lab-user@bastion ~]$ oc get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES iperf-client 1/1 Running 0 19s 10.0.162.179 ip-10-0-162-179.us-east-2.compute.internal <none> <none> iperf-server 1/1 Running 0 19s 10.0.139.112 ip-10-0-139-112.us-east-2.compute.internal <none> <none>
测试的时候因为5201端口问题，没有联通，切换端口到9010测试，数据如下
[lab-user@bastion ~]$ oc exec -it iperf-server -- iperf3 -i 5 -s -p 9010 ----------------------------------------------------------- Server listening on 9010 (test #1) ----------------------------------------------------------- Accepted connection from 10.0.158.220, port 46388 [ 5] local 10.0.149.135 port 9010 connected to 10.0.158.220 port 46396 [ ID] Interval Transfer Bitrate [ 5] 0.00-5.02 sec 3.85 GBytes 6.60 Gbits/sec [ 5] 5.02-10.02 sec 3.72 GBytes 6.39 Gbits/sec [ 5] 10.02-15.02 sec 3.79 GBytes 6.52 Gbits/sec [ 5] 15.02-20.02 sec 3.69 GBytes 6.34 Gbits/sec [ 5] 20.02-25.02 sec 3.61 GBytes 6.21 Gbits/sec [ 5] 25.02-30.02 sec 3.90 GBytes 6.70 Gbits/sec [ 5] 30.02-35.02 sec 3.94 GBytes 6.77 Gbits/sec [ 5] 35.02-40.00 sec 3.94 GBytes 6.79 Gbits/sec [ 5] 40.00-45.02 sec 3.90 GBytes 6.68 Gbits/sec [ 5] 45.02-50.02 sec 4.02 GBytes 6.90 Gbits/sec [ 5] 50.02-55.02 sec 4.15 GBytes 7.12 Gbits/sec [ 5] 55.02-60.02 sec 3.85 GBytes 6.62 Gbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-60.02 sec 46.4 GBytes 6.64 Gbits/sec receiver [lab-user@bastion ~]$ oc exec -it iperf-client -- iperf3 -i 5 -t 60 -c 10.0.149.135 -p 9010 Connecting to host 10.0.149.135, port 9010 [ 5] local 10.0.158.220 port 46396 connected to 10.0.149.135 port 9010 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-5.00 sec 3.86 GBytes 6.63 Gbits/sec 14 3.04 MBytes [ 5] 5.00-10.00 sec 3.72 GBytes 6.39 Gbits/sec 13 2.90 MBytes [ 5] 10.00-15.00 sec 3.80 GBytes 6.52 Gbits/sec 23 2.50 MBytes [ 5] 15.00-20.00 sec 3.69 GBytes 6.34 Gbits/sec 0 2.53 MBytes [ 5] 20.00-25.00 sec 3.61 GBytes 6.21 Gbits/sec 1 2.91 MBytes [ 5] 25.00-30.00 sec 3.90 GBytes 6.70 Gbits/sec 28 2.89 MBytes [ 5] 30.00-35.00 sec 3.94 GBytes 6.77 Gbits/sec 17 3.01 MBytes [ 5] 35.00-40.00 sec 3.94 GBytes 6.77 Gbits/sec 20 3.01 MBytes [ 5] 40.00-45.00 sec 3.90 GBytes 6.70 Gbits/sec 15 3.01 MBytes [ 5] 45.00-50.00 sec 4.02 GBytes 6.90 Gbits/sec 18 3.03 MBytes [ 5] 50.00-55.00 sec 4.15 GBytes 7.12 Gbits/sec 17 3.03 MBytes [ 5] 55.00-60.00 sec 3.85 GBytes 6.62 Gbits/sec 22 3.03 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-60.00 sec 46.4 GBytes 6.64 Gbits/sec 188 sender [ 5] 0.00-60.02 sec 46.4 GBytes 6.64 Gbits/sec receiver iperf Done.
