fio

一、安装
http://freshmeat.sourceforge.net/projects/fio上下载fio-2.1.10.tar.gz，版本包见附件
解压
tar  -xvf  fio-3.3.tar.gz
cd  fio-3.3

执行命令：
./configure
make
make install

安装libaio-devel
yum install libaio-devel

重新执行：
#./configure
#make
#make install

直接执行fio测试。
#fio -ioengine=libaio -filename=fiofile -bs=20k -direct=1 -iodepth=4 -thread -size=300g -rw=randwrite -name=20k_randwrite -numjobs=2 -group_reporting
如果有报错：是因为没装libaio-devel

检查libaio和rbd是否存在，存在表示已经安装了libaio和rdb，下面可以看到所有引擎
[root@ceph1 home]# fio --enghelp
Available IO engines:
cpuio
mmap
sync
psync
vsync
pvsync
null
net
netsplice
libaio
posixaio
falloc
e4defrag
splice
rbd
sg
binject
二、fio引擎介绍
参考网址：
http://blog.yufeng.info/archives/677
ioengine=str
Defines how the job issues I/O. The following types are defined:
sync
Basic read(2) or write(2) I/O. fseek(2) is used to position the I/O location.
psync
Basic pread(2) or pwrite(2) I/O.
vsync
Basic readv(2) or writev(2) I/O. Will emulate queuing by coalescing adjacents IOs into a single submission.
libaio
Linux native asynchronous I/O. This ioengine defines engine specific options.
posixaio
POSIX asynchronous I/O using aio_read(3) and aio_write(3).
solarisaio
Solaris native asynchronous I/O.
windowsaio
Windows native asynchronous I/O.
mmap
File is memory mapped with mmap(2) and data copied using memcpy(3).
splice
splice(2) is used to transfer the data and vmsplice(2) to transfer data from user-space to the kernel.
syslet-rw
Use the syslet system calls to make regular read/write asynchronous.
sg
SCSI generic sg v3 I/O. May be either synchronous using the SG_IO ioctl, or if the target is an sg character device, we use read(2) and write(2) for asynchronous I/O.
null
Doesn’t transfer any data, just pretends to. Mainly used to exercise fio itself and for debugging and testing purposes.
net
Transfer over the network. The protocol to be used can be defined with the protocol parameter. Depending on the protocol, filename, hostname, port, or listen must be specified. This ioengine defines engine specific options.
netsplice
Like net, but uses splice(2) and vmsplice(2) to map data and send/receive. This ioengine defines engine specific options.
cpuio
Doesn’t transfer any data, but burns CPU cycles according to cpuload and cpucycles parameters.
guasi
The GUASI I/O engine is the Generic Userspace Asynchronous Syscall Interface approach to asycnronous I/O.

个别中文解释：
默认值是sync同步阻塞I/O，
libaio是Linux的native异步I/O
io引擎使用pync方式
sync:采用read,write,使用fseek定位读写位置。
psync:采用pread、pwrite进行文件读写操作
vsync:采用readv(2) orwritev(2)进行文件读写操作
三、各引擎使用方法
1、fio-vsync：
vsync:采用readv(2) orwritev(2)进行文件读写操作
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=vsync -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

2、sync:采用read,write,使用fseek定位读写位置。同步阻塞I/O，
fio -filename=/mnt/data/111.log -ioengine=sync -direct=1 -rw=randwrite -bs=8k -size=1G -numjobs=8 -runtime=10-group_reporting -name=fio_test

3、psync对磁盘进行读写（lsblk），如下查出来就是/dev/vdb
[root@host111 data]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
vda 252:0 0 75G 0 disk
├─vda1 252:1 0 200M 0 part /boot
└─vda2 252:2 0 74.8G 0 part
├─systemvg-swaplv (dm-0) 253:0 0 10G 0 lvm [SWAP]
├─systemvg-rootlv (dm-1) 253:1 0 8G 0 lvm /
├─systemvg-homelv (dm-2) 253:2 0 2G 0 lvm /home
├─systemvg-varlv (dm-3) 253:3 0 6G 0 lvm /var
├─systemvg-tmplv (dm-4) 253:4 0 2G 0 lvm /tmp
├─systemvg-optlv (dm-5) 253:5 0 30G 0 lvm /opt
└─systemvg-usrlv (dm-6) 253:6 0 10G 0 lvm /usr
vdb 252:16 0 100G 0 disk /mysql
vdc 252:32 0 500G 0 disk /mnt/data

或者对一个文件进行写：
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

4、posixaio：glibc POSIX asynchronous I/O using aio_read(3) and aio_write(3).
fio -name=/mnt/data/11.log -direct=0 -iodepth=96 -rw=randread -ioengine=posixaio -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting

5、libaio：Linux native asynchronous I/O.Linux的native异步I/O
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

6、mmap：File is memory mapped with mmap(2) and data copied using memcpy(3).
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=mmap -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

7、splice： splice(2) is used to transfer the data and vmsplice(2) to transfer data from user-space to the kernel.
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=splice -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

8、falloc
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=falloc -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

9、pvsync：
fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=pvsync -bs=4k -size=1G -numjobs=64 -runtime=10 -group_reporting -name=file

引擎参照：没有举例的可能执行有问题，或者是针对块存储的
binject:只针对block devices
sg:针对块
splice
e4defrag：Option inplace: missing long option name
falloc
posixaio
libaio
net：fio: network IO can't be random
netsplice
null
sync
psync
vsync
pvsync
mmap
cpuio
syslet-rw

使用配置文件执行：
参考：https://segmentfault.com/a/1190000003880571
如下为rdb测试的配置文件：执行：fio 256k-randread.fio
[root@host111 fio_cfg]# vi 256k-randread.fio
[randread-256k]
description="randread test with block size of 256k-test-poolimages1"
ioengine=rbd
clientname=admin
pool=test-pool
rbdname=test-poolimages1
iodepth=8
runtime=300
rw=randread
bs=256k
numjobs=1
size=5g
#write_iops_log=write_iops
#log_avg_msec=1000
#filename=/data/osd.0/1.txt
write_bw_log=rw
write_lat_log=rw
write_iops_log=rw

[randread-256k]
description="randread test with block size of 256k-test-poolimages2"
ioengine=rbd
clientname=admin
pool=test-pool
rbdname=test-poolimages2
iodepth=8
runtime=300
rw=randread
bs=256k
numjobs=1
size=5g
#write_iops_log=write_iops
#log_avg_msec=1000
#filename=/data/osd.0/1.txt
write_bw_log=rw
write_lat_log=rw
write_iops_log=rw
四、补充
1、安装插件：
yum install gnuplot

2.输出bw，lat和iops数据并画图
fio安装完后自带有一个高级脚本fio_generate_plots能够根据fio输出的数据进行画图。操作流程如下：
fio的输出日志主要包含三种：bw，lat和iops，设置这三种的参数如下：
write_bw_log=rw
w
write_lat_log=rw
w
write_iops_log=rw

如：fio -filename=/mnt/data/112.log -direct=1 -iodepth 1 -thread -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=10 -group_reporting -name=file -write_bw_log=rww -write_lat_log=rww -write_iops_log=rw
最后生成：
[root@host111 fio_cfg]# ls
psync randrw.fio read.fio rw_iops.1.log rww_clat.1.log rww_slat.1.log
randread.fio randwrite.fio rw.fio rww_bw.1.log rww_lat.1.log write.fio
需要将$1.log改为*.log
[root@host111 fio_cfg]# mv rw_iops.1.log rw_iops.log
[root@host111 fio_cfg]# mv rww_bw.1.log rww_bw.log
[root@host111 fio_cfg]# mv rww_clat.1.log rww_clat.log
[root@host111 fio_cfg]# mv rww_lat.1.log rww_lat.log
[root@host111 fio_cfg]# mv rww_slat.1.log rww_slat.log
[root@host111 fio_cfg]# ls
psync randrw.fio read.fio rw_iops.log rww_clat.log rww_slat.log
randread.fio randwrite.fio rw.fio rww_bw.log rww_lat.log write.fio

或者使用下面脚本进行修改：
这里需要强调的一点是，后面接的参数rw，是输出日志文件名的prefix，如最终会生成的日志文件如下：
rw_iops.log

rw_clat.log

rw_slat.log

rw_lat.log

rw_bw.log

这个参数在后面画图的时候也要用到。
for i in clat lat slat bw iops;do mv rbd_$i.1.log rbd_$i.log;done

3、画图
前提是还需要安装好gnuplot，然后使用下面的命令即可自动画图：
root@ubuntu:/tmp> fio_generate_plots bw

发现没有，fio_generate_plots接受的唯一参数就是这个日志文件名的prefix。
本例中生成的图片文件有：
bw-bw.svg 
bw-clat.svg 
bw-iops.svg 
bw-lat.svg 
bw-slat.svg

下载到本地，需要用谷歌浏览器才能打开，图片软件打不开