fio测试硬盘速度

使用方法

–readonly 确保只读,这个选项在生产系统上很有用,不会生成临时文件,读的对象只能是已经存在的,例如设备
io=执行了多少M的IO
bw=平均IO带宽
iops=IOPS
lat=响应时间
bw=带宽
cpu=利用率
IO depths=io队列
IO submit=单个IO提交要提交的IO数
IO complete=Like the above submit number, but for completions instead.
IO issued=The number of read/write requests issued, and how many of them were short.
IO latencies=IO完延迟的分布

io=总共执行了多少size的IO
aggrb=group总带宽
minb=最小.平均带宽.
maxb=最大平均带宽.
mint=group中线程的最短运行时间.
maxt=group中线程的最长运行时间.

ios=所有group总共执行的IO数.
merge=总共发生的IO合并数.
ticks=Number of ticks we kept the disk busy.
io_queue=花费在队列上的总共时间.
util=磁盘利用率
runt=线程运行时间 run time
slat=提交延迟 single latency
clat=完成延迟 compliet latency

7个ssd盘raid 0 随机写22.5k

配置

$ sudo mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Thu Dec 19 13:51:21 2019
        Raid Level : raid0
        Array Size : 13669884928 (13036.62 GiB 13997.96 GB)
      Raid Devices : 7
     Total Devices : 7
       Persistence : Superblock is persistent

       Update Time : Thu Dec 19 13:51:21 2019
             State : clean 
    Active Devices : 7
   Working Devices : 7
    Failed Devices : 0
     Spare Devices : 0

        Chunk Size : 512K

Consistency Policy : none

              Name : localhost.localdomain:0  (local to host localhost.localdomain)
              UUID : 10231753:f9330671:5661c0f8:3f953bbe
            Events : 0

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
       2       8       48        2      active sync   /dev/sdd
       3       8       64        3      active sync   /dev/sde
       4       8       80        4      active sync   /dev/sdf
       5       8       96        5      active sync   /dev/sdg
       6       8      112        6      active sync   /dev/sdh
[oracle@localhost admin]$ 

随机写22.5K

[oracle@localhost orcl]$ fio -filename=fio_test  -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=psync -bs=8k -size=10G -numjobs=8 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=randwrite, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.7
Starting 8 threads
Jobs: 4 (f=4): [w(1),_(2),w(2),_(2),w(1)][99.6%][r=0KiB/s,w=220MiB/s][r=0,w=28.2k IOPS][eta 00m:02s]
w_8k: (groupid=0, jobs=8): err= 0: pid=80165: Mon Mar 30 11:41:51 2020
  write: IOPS=22.5k, BW=176MiB/s (184MB/s)(80.0GiB/465690msec)
    clat (usec): min=50, max=21739, avg=352.52, stdev=503.74
     lat (usec): min=50, max=21740, avg=352.77, stdev=503.74
    clat percentiles (usec):
     |  1.00th=[  137],  5.00th=[  157], 10.00th=[  163], 20.00th=[  169],
     | 30.00th=[  174], 40.00th=[  176], 50.00th=[  180], 60.00th=[  182],
     | 70.00th=[  188], 80.00th=[  219], 90.00th=[ 1090], 95.00th=[ 1729],
     | 99.00th=[ 1860], 99.50th=[ 3032], 99.90th=[ 3261], 99.95th=[ 3294],
     | 99.99th=[16188]
   bw (  KiB/s): min=11424, max=59552, per=12.51%, avg=22538.23, stdev=1913.92, samples=7434
   iops        : min= 1428, max= 7444, avg=2817.26, stdev=239.24, samples=7434
  lat (usec)   : 100=0.21%, 250=81.40%, 500=3.74%, 750=3.95%, 1000=0.03%
  lat (msec)   : 2=9.90%, 4=0.75%, 10=0.01%, 20=0.01%, 50=0.01%
  cpu          : usr=1.11%, sys=6.05%, ctx=12927814, majf=0, minf=23
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,10485760,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=176MiB/s (184MB/s), 176MiB/s-176MiB/s (184MB/s-184MB/s), io=80.0GiB (85.9GB), run=465690-465690msec

Disk stats (read/write):
    md0: ios=518/10836645, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=74/1672864, aggrmerge=0/9, aggrticks=38/518237, aggrin_queue=518132, aggrutil=73.46%
  sdb: ios=2/1672325, merge=0/6, ticks=1/504646, in_queue=504538, util=68.17%
  sdc: ios=456/1672393, merge=0/8, ticks=241/503666, in_queue=503765, util=68.00%
  sdd: ios=17/1673043, merge=0/11, ticks=4/504728, in_queue=504679, util=68.20%
  sde: ios=2/1672969, merge=0/9, ticks=1/502417, in_queue=502271, util=67.98%
  sdf: ios=24/1673190, merge=0/12, ticks=9/504294, in_queue=504117, util=68.08%
  sdg: ios=6/1673045, merge=0/10, ticks=4/502743, in_queue=502512, util=68.02%
  sdh: ios=11/1673083, merge=0/10, ticks=6/605170, in_queue=605045, util=73.46%
[oracle@localhost orcl]$ 

本地SATA硬盘读IOPS 384 121M/s 写IOPS 273 109M/S

随机读

scutech@Yao:~/tmp$ fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=randread -ioengine=psync -bs=8k -size=100M -numjobs=8 -runtime=600 -group_reporting -name=r_8k
r_8k: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 1 (f=1): [_(3),E(1),_(3),r(1)][96.7%][r=6328KiB/s,w=0KiB/s][r=791,w=0 IOPS][eta 00m:09s]
r_8k: (groupid=0, jobs=8): err= 0: pid=10982: Tue Mar 24 15:15:14 2020
   read: IOPS=384, BW=3073KiB/s (3146kB/s)(800MiB/266613msec)
    clat (usec): min=86, max=963999, avg=20656.61, stdev=27429.32
     lat (usec): min=88, max=964001, avg=20658.32, stdev=27429.31
    clat percentiles (usec):
     |  1.00th=[   167],  5.00th=[   249], 10.00th=[   453], 20.00th=[  1696],
     | 30.00th=[  4883], 40.00th=[  7439], 50.00th=[ 10421], 60.00th=[ 15533],
     | 70.00th=[ 22676], 80.00th=[ 33817], 90.00th=[ 53740], 95.00th=[ 76022],
     | 99.00th=[125305], 99.50th=[145753], 99.90th=[200279], 99.95th=[225444],
     | 99.99th=[291505]
   bw (  KiB/s): min=   80, max= 3392, per=12.55%, avg=385.59, stdev=149.43, samples=4226
   iops        : min=   10, max=  424, avg=48.20, stdev=18.68, samples=4226
  lat (usec)   : 100=0.02%, 250=5.03%, 500=5.58%, 750=2.23%, 1000=2.88%
  lat (msec)   : 2=4.53%, 4=6.13%, 10=22.45%, 20=18.23%, 50=21.34%
  lat (msec)   : 100=9.30%, 250=2.24%, 500=0.02%, 1000=0.01%
  cpu          : usr=0.07%, sys=0.21%, ctx=102437, majf=0, minf=16
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=102400,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=3073KiB/s (3146kB/s), 3073KiB/s-3073KiB/s (3146kB/s-3146kB/s), io=800MiB (839MB), run=266613-266613msec

Disk stats (read/write):
  sdb: ios=102172/303, merge=0/185, ticks=2129364/7072268, in_queue=9888028, util=100.00%
scutech@Yao:~/tmp$ 

顺序读

scutech@Yao:~/tmp$ fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=read -ioengine=psync -bs=512k -size=100M -numjobs=8 -runtime=600 -group_reporting -name=r_512k
r_512k: (g=0): rw=read, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 6 (f=6): [R(6),_(2)][75.0%][r=103MiB/s,w=0KiB/s][r=206,w=0 IOPS][eta 00m:02s]     
r_512k: (groupid=0, jobs=8): err= 0: pid=11121: Tue Mar 24 15:18:07 2020
   read: IOPS=241, BW=121MiB/s (127MB/s)(800MiB/6613msec)
    clat (usec): min=1959, max=523812, avg=30078.61, stdev=63209.97
     lat (usec): min=1960, max=523813, avg=30080.28, stdev=63209.96
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    4], 20.00th=[    4],
     | 30.00th=[    4], 40.00th=[    5], 50.00th=[    9], 60.00th=[   13],
     | 70.00th=[   21], 80.00th=[   29], 90.00th=[   63], 95.00th=[  165],
     | 99.00th=[  347], 99.50th=[  422], 99.90th=[  506], 99.95th=[  523],
     | 99.99th=[  523]
   bw (  KiB/s): min= 1024, max=41984, per=12.59%, avg=15601.70, stdev=8175.37, samples=90
   iops        : min=    2, max=   82, avg=30.41, stdev=16.00, samples=90
  lat (msec)   : 2=0.19%, 4=37.50%, 10=13.75%, 20=18.69%, 50=18.12%
  lat (msec)   : 100=3.88%, 250=5.81%, 500=1.94%, 750=0.12%
  cpu          : usr=0.02%, sys=0.23%, ctx=1633, majf=0, minf=1024
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=1600,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=121MiB/s (127MB/s), 121MiB/s-121MiB/s (127MB/s-127MB/s), io=800MiB (839MB), run=6613-6613msec

Disk stats (read/write):
  sdb: ios=1512/17, merge=7/16, ticks=46696/3628, in_queue=50568, util=98.49%
scutech@Yao:~/tmp$ 

随机写273

scutech@Yao:~/tmp$ fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=psync -bs=8k -size=100M -numjobs=8 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=randwrite, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 3 (f=3): [_(1),w(1),_(1),w(1),_(3),w(1)][98.7%][r=0KiB/s,w=3323KiB/s][r=0,w=415 IOPS][eta 00m:05s]
w_8k: (groupid=0, jobs=8): err= 0: pid=11173: Tue Mar 24 15:25:35 2020
  write: IOPS=273, BW=2184KiB/s (2237kB/s)(800MiB/375014msec)
    clat (usec): min=150, max=1110.1k, avg=29157.72, stdev=79389.54
     lat (usec): min=151, max=1110.1k, avg=29159.62, stdev=79389.54
    clat percentiles (usec):
     |  1.00th=[   198],  5.00th=[   241], 10.00th=[   289], 20.00th=[  1516],
     | 30.00th=[  2278], 40.00th=[  2507], 50.00th=[  2638], 60.00th=[  2802],
     | 70.00th=[  3228], 80.00th=[ 20317], 90.00th=[ 95945], 95.00th=[164627],
     | 99.00th=[446694], 99.50th=[549454], 99.90th=[734004], 99.95th=[801113],
     | 99.99th=[952108]
   bw (  KiB/s): min=   16, max= 3392, per=12.92%, avg=282.26, stdev=172.01, samples=5790
   iops        : min=    2, max=  424, avg=35.28, stdev=21.50, samples=5790
  lat (usec)   : 250=7.52%, 500=6.43%, 750=0.54%, 1000=0.80%
  lat (msec)   : 2=10.77%, 4=46.56%, 10=4.39%, 20=2.85%, 50=5.34%
  lat (msec)   : 100=5.11%, 250=7.38%, 500=1.59%, 750=0.63%, 1000=0.08%
  lat (msec)   : 2000=0.01%
  cpu          : usr=0.04%, sys=0.25%, ctx=205155, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,102400,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=2184KiB/s (2237kB/s), 2184KiB/s-2184KiB/s (2237kB/s-2237kB/s), io=800MiB (839MB), run=375014-375014msec

Disk stats (read/write):
  sdb: ios=2645/110080, merge=0/1788, ticks=18500/2015160, in_queue=2033552, util=98.69%
scutech@Yao:~/tmp$ 

顺序写109M/S

scutech@Yao:~/tmp$ fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=write -ioengine=psync -bs=512k -size=500M -numjobs=8 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=write, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 3 (f=3): [_(2),W(1),_(2),W(2),_(1)][94.7%][r=0KiB/s,w=120MiB/s][r=0,w=239 IOPS][eta 00m:02s]
w_8k: (groupid=0, jobs=8): err= 0: pid=11398: Tue Mar 24 15:30:43 2020
  write: IOPS=217, BW=109MiB/s (114MB/s)(4000MiB/36800msec)
    clat (usec): min=1974, max=827643, avg=33994.75, stdev=78367.48
     lat (usec): min=1981, max=827658, avg=34006.11, stdev=78368.51
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    3], 40.00th=[    3], 50.00th=[    3], 60.00th=[    3],
     | 70.00th=[    3], 80.00th=[   43], 90.00th=[  121], 95.00th=[  213],
     | 99.00th=[  372], 99.50th=[  456], 99.90th=[  617], 99.95th=[  701],
     | 99.99th=[  827]
   bw (  KiB/s): min= 1024, max=118784, per=13.28%, avg=14783.35, stdev=13347.91, samples=530
   iops        : min=    2, max=  232, avg=28.83, stdev=26.04, samples=530
  lat (msec)   : 2=0.30%, 4=76.38%, 10=0.74%, 20=0.51%, 50=2.95%
  lat (msec)   : 100=7.38%, 250=8.01%, 500=3.56%, 750=0.16%, 1000=0.01%
  cpu          : usr=0.07%, sys=0.26%, ctx=16008, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,8000,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=109MiB/s (114MB/s), 109MiB/s-109MiB/s (114MB/s-114MB/s), io=4000MiB (4194MB), run=36800-36800msec

Disk stats (read/write):
  sdb: ios=6/8093, merge=0/111, ticks=352/41808, in_queue=42400, util=97.35%
scutech@Yao:~/tmp$ 

本地SSD硬盘读IOPS 36K 340M/S 写IOPS 12.7K 407M/S

随机读36K

scutech@Yao:~$ sudo fio -filename=/tmp/test1  -direct=1 -iodepth 32 -thread -rw=randread -ioengine=psync -bs=8k -size=1G -numjobs=8 -runtime=600 -group_reporting -name=r_8k
r_8k: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
r_8k: Laying out IO file (1 file / 1024MiB)
Jobs: 8 (f=8): [r(8)][100.0%][r=257MiB/s,w=0KiB/s][r=32.9k,w=0 IOPS][eta 00m:00s]
r_8k: (groupid=0, jobs=8): err= 0: pid=10784: Tue Mar 24 15:05:13 2020
   read: IOPS=35.7k, BW=279MiB/s (293MB/s)(8192MiB/29364msec)
    clat (usec): min=132, max=2497, avg=214.00, stdev=44.84
     lat (usec): min=133, max=2499, avg=215.83, stdev=45.34
    clat percentiles (usec):
     |  1.00th=[  151],  5.00th=[  161], 10.00th=[  169], 20.00th=[  180],
     | 30.00th=[  190], 40.00th=[  198], 50.00th=[  206], 60.00th=[  217],
     | 70.00th=[  227], 80.00th=[  243], 90.00th=[  269], 95.00th=[  297],
     | 99.00th=[  359], 99.50th=[  383], 99.90th=[  465], 99.95th=[  537],
     | 99.99th=[  922]
   bw (  KiB/s): min=29856, max=36848, per=12.52%, avg=35767.12, stdev=1238.81, samples=464
   iops        : min= 3732, max= 4606, avg=4470.85, stdev=154.90, samples=464
  lat (usec)   : 250=83.92%, 500=16.01%, 750=0.05%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%
  cpu          : usr=2.71%, sys=23.37%, ctx=1050419, majf=0, minf=16
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=1048576,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=279MiB/s (293MB/s), 279MiB/s-279MiB/s (293MB/s-293MB/s), io=8192MiB (8590MB), run=29364-29364msec

Disk stats (read/write):
  sda: ios=1047501/48, merge=0/13, ticks=203252/34732, in_queue=236016, util=99.49%
scutech@Yao:~$ 

顺序读340M/S

scutech@Yao:~$ sudo fio -filename=/tmp/test1  -direct=1 -iodepth 32 -thread -rw=read -ioengine=psync -bs=512k -size=1G -numjobs=8 -runtime=600 -group_reporting -name=r_512k
r_512k: (g=0): rw=read, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 8 (f=8): [R(8)][100.0%][r=361MiB/s,w=0KiB/s][r=722,w=0 IOPS][eta 00m:00s]
r_512k: (groupid=0, jobs=8): err= 0: pid=10839: Tue Mar 24 15:06:58 2020
   read: IOPS=720, BW=360MiB/s (378MB/s)(8192MiB/22735msec)
    clat (usec): min=1860, max=17237, avg=11079.77, stdev=645.43
     lat (usec): min=1862, max=17238, avg=11081.42, stdev=645.42
    clat percentiles (usec):
     |  1.00th=[ 9765],  5.00th=[10159], 10.00th=[10290], 20.00th=[10552],
     | 30.00th=[10814], 40.00th=[10945], 50.00th=[11076], 60.00th=[11207],
     | 70.00th=[11338], 80.00th=[11600], 90.00th=[11863], 95.00th=[12125],
     | 99.00th=[12518], 99.50th=[12780], 99.90th=[13698], 99.95th=[13960],
     | 99.99th=[14746]
   bw (  KiB/s): min=43008, max=50176, per=12.49%, avg=46102.67, stdev=1830.97, samples=360
   iops        : min=   84, max=   98, avg=90.03, stdev= 3.57, samples=360
  lat (msec)   : 2=0.01%, 4=0.01%, 10=2.26%, 20=97.72%
  cpu          : usr=0.08%, sys=0.73%, ctx=16446, majf=0, minf=1024
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=16384,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=360MiB/s (378MB/s), 360MiB/s-360MiB/s (378MB/s-378MB/s), io=8192MiB (8590MB), run=22735-22735msec

Disk stats (read/write):
  sda: ios=16305/53, merge=4/30, ticks=179696/12728, in_queue=192436, util=99.52%

随机写IOPS 12.7K

scutech@Yao:/tmp$ sudo fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=psync -bs=8k -size=300M -numjobs=8 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=randwrite, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 7 (f=7): [w(4),_(1),w(3)][100.0%][r=0KiB/s,w=100MiB/s][r=0,w=12.8k IOPS][eta 00m:00s]
w_8k: (groupid=0, jobs=8): err= 0: pid=11513: Tue Mar 24 15:34:09 2020
  write: IOPS=12.7k, BW=99.4MiB/s (104MB/s)(2400MiB/24136msec)
    clat (usec): min=53, max=37815, avg=618.62, stdev=739.68
     lat (usec): min=54, max=37816, avg=620.15, stdev=739.68
    clat percentiles (usec):
     |  1.00th=[   69],  5.00th=[  570], 10.00th=[  578], 20.00th=[  586],
     | 30.00th=[  594], 40.00th=[  594], 50.00th=[  603], 60.00th=[  603],
     | 70.00th=[  611], 80.00th=[  635], 90.00th=[  693], 95.00th=[  725],
     | 99.00th=[  865], 99.50th=[  922], 99.90th=[ 1074], 99.95th=[12780],
     | 99.99th=[36963]
   bw (  KiB/s): min= 9648, max=18224, per=12.52%, avg=12746.38, stdev=758.95, samples=382
   iops        : min= 1206, max= 2278, avg=1593.28, stdev=94.85, samples=382
  lat (usec)   : 100=2.91%, 250=0.09%, 500=0.77%, 750=92.18%, 1000=3.86%
  lat (msec)   : 2=0.12%, 4=0.01%, 20=0.02%, 50=0.04%
  cpu          : usr=0.95%, sys=8.58%, ctx=614502, majf=0, minf=0
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,307200,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=99.4MiB/s (104MB/s), 99.4MiB/s-99.4MiB/s (104MB/s-104MB/s), io=2400MiB (2517MB), run=24136-24136msec

Disk stats (read/write):
  sda: ios=0/303917, merge=0/15, ticks=0/15068, in_queue=14932, util=62.27%
scutech@Yao:/tmp$ 

顺序写407M/S

scutech@Yao:/tmp$ sudo fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=write -ioengine=psync -bs=512k -size=1000M -numjobs=8 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=write, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=32
...
fio-3.1
Starting 8 threads
Jobs: 4 (f=4): [_(2),W(2),_(1),W(2),_(1)][86.4%][r=0KiB/s,w=412MiB/s][r=0,w=824 IOPS][eta 00m:03s]
w_8k: (groupid=0, jobs=8): err= 0: pid=11568: Tue Mar 24 15:35:21 2020
  write: IOPS=813, BW=407MiB/s (427MB/s)(8000MiB/19665msec)
    clat (usec): min=1015, max=306546, avg=9406.80, stdev=24225.47
     lat (usec): min=1021, max=306556, avg=9417.60, stdev=24225.94
    clat percentiles (usec):
     |  1.00th=[  1029],  5.00th=[  1057], 10.00th=[  1074], 20.00th=[  1090],
     | 30.00th=[  1106], 40.00th=[  1106], 50.00th=[  1123], 60.00th=[  1139],
     | 70.00th=[  1139], 80.00th=[  8717], 90.00th=[ 30278], 95.00th=[ 58983],
     | 99.00th=[117965], 99.50th=[152044], 99.90th=[208667], 99.95th=[229639],
     | 99.99th=[291505]
   bw (  KiB/s): min=12288, max=168745, per=12.64%, avg=52657.90, stdev=23245.82, samples=297
   iops        : min=   24, max=  329, avg=102.78, stdev=45.38, samples=297
  lat (msec)   : 2=79.23%, 4=0.21%, 10=3.91%, 20=4.67%, 50=5.67%
  lat (msec)   : 100=4.76%, 250=1.51%, 500=0.04%
  cpu          : usr=0.24%, sys=0.96%, ctx=31846, majf=0, minf=1
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,16000,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=407MiB/s (427MB/s), 407MiB/s-407MiB/s (427MB/s-427MB/s), io=8000MiB (8389MB), run=19665-19665msec

Disk stats (read/write):
  sda: ios=0/15998, merge=0/21, ticks=0/18136, in_queue=18120, util=91.70%
scutech@Yao:/tmp$ 

virtualbox上的虚拟机读iops 1596 940M/S 顺序写IOPS 354 24M/S

随机读iops1596

[root@localhost fio-2.1.10]# ./fio -filename=/dev/sda1  -direct=1 -iodepth 32 -thread -rw=randread -ioengine=psync -bs=8k -size=100M -numjobs=8 -runtime=600 -group_reporting --readonly -name=r_8k
r_8k: (g=0): rw=randread, bs=8K-8K/8K-8K/8K-8K, ioengine=psync, iodepth=32
...
fio-2.1.10
Starting 8 threads
Jobs: 3 (f=3): [r__r___r] [97.0% done] [42249KB/0KB/0KB /s] [5281/0/0 iops] [eta 00m:02s]
r_8k: (groupid=0, jobs=8): err= 0: pid=2317: Tue Mar 24 02:45:28 2020
  read : io=819200KB, bw=12770KB/s, iops=1596, runt= 64152msec
    clat (usec): min=69, max=205390, avg=4994.91, stdev=12572.80
     lat (usec): min=70, max=205391, avg=4995.41, stdev=12572.80
    clat percentiles (usec):
     |  1.00th=[   94],  5.00th=[  117], 10.00th=[  137], 20.00th=[  159],
     | 30.00th=[  179], 40.00th=[  213], 50.00th=[  262], 60.00th=[  326],
     | 70.00th=[ 1640], 80.00th=[ 5088], 90.00th=[15168], 95.00th=[28800],
     | 99.00th=[63232], 99.50th=[77312], 99.90th=[112128], 99.95th=[128512],
     | 99.99th=[173056]
    bw (KB  /s): min=  483, max= 7864, per=12.29%, avg=1569.03, stdev=614.51
    lat (usec) : 100=2.02%, 250=45.45%, 500=16.95%, 750=1.36%, 1000=1.23%
    lat (msec) : 2=4.43%, 4=6.17%, 10=8.48%, 20=6.17%, 50=5.83%
    lat (msec) : 100=1.72%, 250=0.18%
  cpu          : usr=0.14%, sys=1.70%, ctx=102490, majf=0, minf=23
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=102400/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=819200KB, aggrb=12769KB/s, minb=12769KB/s, maxb=12769KB/s, mint=64152msec, maxt=64152msec

Disk stats (read/write):
  sda: ios=101415/19, merge=0/8, ticks=505285/11353, in_queue=516618, util=99.98%
[root@localhost fio-2.1.10]# 

顺序读940M/s

[root@localhost fio-2.1.10]# ./fio -filename=/dev/sda1  -direct=1 -iodepth 32 -thread -rw=read -ioengine=psync -bs=512k -size=10G -numjobs=8 -runtime=600 -group_reporting --readonly -name=r_512k
r_512k: (g=0): rw=read, bs=512K-512K/512K-512K/512K-512K, ioengine=psync, iodepth=32
...
fio-2.1.10
Starting 8 threads
Jobs: 5 (f=5): [_RRR_R_R] [3.4% done] [1532MB/0KB/0KB /s] [3063/0/0 iops] [eta 02m:22s]
r_512k: (groupid=0, jobs=8): err= 0: pid=2303: Tue Mar 24 02:42:56 2020
  read : io=4000.0MB, bw=948807KB/s, iops=1853, runt=  4317msec
    clat (usec): min=223, max=393856, avg=3938.25, stdev=22191.18
     lat (usec): min=224, max=393856, avg=3938.60, stdev=22191.21
    clat percentiles (usec):
     |  1.00th=[  237],  5.00th=[  239], 10.00th=[  239], 20.00th=[  243],
     | 30.00th=[  249], 40.00th=[  270], 50.00th=[  290], 60.00th=[  318],
     | 70.00th=[  346], 80.00th=[  438], 90.00th=[ 2544], 95.00th=[11456],
     | 99.00th=[118272], 99.50th=[181248], 99.90th=[272384], 99.95th=[305152],
     | 99.99th=[395264]
    bw (KB  /s): min= 6059, max=413696, per=11.29%, avg=107112.93, stdev=104276.42
    lat (usec) : 250=33.00%, 500=49.46%, 750=3.84%, 1000=1.58%
    lat (msec) : 2=1.31%, 4=3.18%, 10=2.27%, 20=2.08%, 50=1.39%
    lat (msec) : 100=0.70%, 250=0.99%, 500=0.21%
  cpu          : usr=0.00%, sys=2.24%, ctx=7973, majf=0, minf=650
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=8000/w=0/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: io=4000.0MB, aggrb=948807KB/s, minb=948807KB/s, maxb=948807KB/s, mint=4317msec, maxt=4317msec

Disk stats (read/write):
  sda: ios=7318/0, merge=0/0, ticks=30337/0, in_queue=30485, util=97.69%

随机写 354

[root@localhost fio-2.1.10]# ./fio -filename=/tmp/test1  -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=psync -bs=8k -size=100M -numjobs=1 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=randwrite, bs=8K-8K/8K-8K/8K-8K, ioengine=psync, iodepth=32
fio-2.1.10
Starting 1 thread
w_8k: Laying out IO file(s) (1 file(s) / 100MB)
Jobs: 1 (f=1): [w] [100.0% done] [0KB/3956KB/0KB /s] [0/494/0 iops] [eta 00m:00s]
w_8k: (groupid=0, jobs=1): err= 0: pid=2523: Tue Mar 24 03:16:54 2020
  write: io=102400KB, bw=2834.3KB/s, iops=354, runt= 36129msec
    clat (usec): min=105, max=1953.4K, avg=2818.74, stdev=57852.36
     lat (usec): min=105, max=1953.4K, avg=2819.51, stdev=57852.37
    clat percentiles (usec):
     |  1.00th=[  109],  5.00th=[  110], 10.00th=[  111], 20.00th=[  112],
     | 30.00th=[  113], 40.00th=[  114], 50.00th=[  115], 60.00th=[  121],
     | 70.00th=[  127], 80.00th=[  137], 90.00th=[  175], 95.00th=[  217],
     | 99.00th=[  366], 99.50th=[  532], 99.90th=[1056768], 99.95th=[1630208],
     | 99.99th=[1744896]
    bw (KB  /s): min=    7, max= 7258, per=100.00%, avg=2986.90, stdev=1973.61
    lat (usec) : 250=96.52%, 500=2.95%, 750=0.15%, 1000=0.05%
    lat (msec) : 2=0.05%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 250=0.01%, 500=0.02%, 750=0.05%, 1000=0.05%, 2000=0.12%
  cpu          : usr=0.68%, sys=1.35%, ctx=12806, majf=0, minf=6
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=12800/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=102400KB, aggrb=2834KB/s, minb=2834KB/s, maxb=2834KB/s, mint=36129msec, maxt=36129msec

Disk stats (read/write):
    dm-0: ios=0/12715, merge=0/0, ticks=0/79659, in_queue=80600, util=98.25%, aggrios=0/12828, aggrmerge=0/198, aggrticks=0/42827, aggrin_queue=42827, aggrutil=98.17%
  sda: ios=0/12828, merge=0/198, ticks=0/42827, in_queue=42827, util=98.17%
[root@localhost fio-2.1.10]# 

顺序写 24M/S

[root@localhost fio-2.1.10]# ./fio -filename=/tmp/test1  -direct=1 -iodepth 32 -thread -rw=write -ioengine=psync -bs=512k -size=1000M -numjobs=1 -runtime=600 -group_reporting -name=w_512k
w_512k: (g=0): rw=write, bs=512K-512K/512K-512K/512K-512K, ioengine=psync, iodepth=32
fio-2.1.10
Starting 1 thread
w_512k: Laying out IO file(s) (1 file(s) / 1000MB)
Jobs: 1 (f=1): [W] [97.7% done] [0KB/18925KB/0KB /s] [0/36/0 iops] [eta 00m:01s]
w_512k: (groupid=0, jobs=1): err= 0: pid=2571: Tue Mar 24 03:25:04 2020
  write: io=1000.0MB, bw=24607KB/s, iops=48, runt= 41615msec
    clat (usec): min=299, max=2715.1K, avg=20745.19, stdev=98346.83
     lat (usec): min=344, max=2715.1K, avg=20803.25, stdev=98347.77
    clat percentiles (usec):
     |  1.00th=[  326],  5.00th=[  342], 10.00th=[  350], 20.00th=[  386],
     | 30.00th=[  446], 40.00th=[  490], 50.00th=[  532], 60.00th=[  572],
     | 70.00th=[  636], 80.00th=[  796], 90.00th=[38656], 95.00th=[79360],
     | 99.00th=[403456], 99.50th=[593920], 99.90th=[921600], 99.95th=[1019904],
     | 99.99th=[2703360]
    bw (KB  /s): min= 1129, max=48188, per=100.00%, avg=27122.26, stdev=9890.06
    lat (usec) : 500=41.80%, 750=36.90%, 1000=4.25%
    lat (msec) : 2=0.50%, 4=0.05%, 10=0.35%, 50=8.70%, 100=3.70%
    lat (msec) : 250=1.10%, 500=2.05%, 750=0.35%, 1000=0.15%, 2000=0.05%
    lat (msec) : >=2000=0.05%
  cpu          : usr=0.28%, sys=0.56%, ctx=2029, majf=0, minf=6
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=0/w=2000/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=1000.0MB, aggrb=24606KB/s, minb=24606KB/s, maxb=24606KB/s, mint=41615msec, maxt=41615msec

Disk stats (read/write):
    dm-0: ios=0/2054, merge=0/0, ticks=0/46482, in_queue=46487, util=99.79%, aggrios=0/2040, aggrmerge=0/34, aggrticks=0/44831, aggrin_queue=44830, aggrutil=99.75%
  sda: ios=0/2040, merge=0/34, ticks=0/44831, in_queue=44830, util=99.75%
[root@localhost fio-2.1.10]# 

dd读942M/s写523m/s

[root@localhost fio-2.1.10]# dd if=/dev/zero of=test bs=512k count=16k conv=fsync
16384+0 records in
16384+0 records out
8589934592 bytes (8.6 GB) copied, 16.4324 s, 523 MB/s
[root@localhost fio-2.1.10]# dd of=/dev/zero if=test bs=512k count=16k conv=fsync
dd: fsync failed for `/dev/zero': Invalid argument
16384+0 records in
16384+0 records out
8589934592 bytes (8.6 GB) copied, 9.11447 s, 942 MB/s

nvme一个盘IOPS 70.9K 4G/S写IOPS108K 3197MiB/s

随机读70.9K

$  fio -filename=/test-nvme/oradata/orcl/cmcc1.dbf  -direct=1 -iodepth 32 -thread -rw=randread -ioengine=psync -bs=8k -size=10000M -numjobs=8 -runtime=600 -group_reporting --readonly -name=r_8k
r_8k: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.7
Starting 8 threads
Jobs: 6 (f=6): [_(2),r(6)][100.0%][r=424MiB/s,w=0KiB/s][r=54.3k,w=0 IOPS][eta 00m:00s]
r_8k: (groupid=0, jobs=8): err= 0: pid=49997: Tue Mar 24 14:31:12 2020
   read: IOPS=70.9k, BW=554MiB/s (581MB/s)(78.1GiB/144491msec)
    clat (usec): min=67, max=17217, avg=110.68, stdev=25.47
     lat (usec): min=67, max=17217, avg=110.80, stdev=25.47
    clat percentiles (usec):
     |  1.00th=[   79],  5.00th=[   82], 10.00th=[   83], 20.00th=[   86],
     | 30.00th=[   96], 40.00th=[  100], 50.00th=[  103], 60.00th=[  114],
     | 70.00th=[  130], 80.00th=[  133], 90.00th=[  139], 95.00th=[  147],
     | 99.00th=[  188], 99.50th=[  204], 99.90th=[  227], 99.95th=[  239],
     | 99.99th=[  273]
   bw (  KiB/s): min=68544, max=77152, per=12.56%, avg=71185.67, stdev=1084.06, samples=2298
   iops        : min= 8568, max= 9644, avg=8898.19, stdev=135.50, samples=2298
  lat (usec)   : 100=42.29%, 250=57.68%, 500=0.03%, 750=0.01%
  lat (msec)   : 2=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=2.53%, sys=9.60%, ctx=10280710, majf=0, minf=2676
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=10240000,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=554MiB/s (581MB/s), 554MiB/s-554MiB/s (581MB/s-581MB/s), io=78.1GiB (83.9GB), run=144491-144491msec

Disk stats (read/write):
  nvme0n1: ios=10238892/265, merge=0/14, ticks=1026378/32, in_queue=1034162, util=100.00%
[oracle@base-test-01 orcl]$ 

顺序读

$ fio -filename=/test-nvme/oradata/orcl/cmcc1.dbf  -direct=1 -iodepth 32 -thread -rw=read -ioengine=psync -bs=512k -size=10G -numjobs=8 -runtime=600 -group_reporting --readonly -name=r_512k
r_512k: (g=0): rw=read, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=32
...
fio-3.7
Starting 8 threads
Jobs: 6 (f=6): [R(2),_(2),R(4)][100.0%][r=3752MiB/s,w=0KiB/s][r=7503,w=0 IOPS][eta 00m:00s]
r_512k: (groupid=0, jobs=8): err= 0: pid=49927: Tue Mar 24 14:28:28 2020
   read: IOPS=8020, BW=4010MiB/s (4205MB/s)(80.0GiB/20427msec)
    clat (usec): min=308, max=46522, avg=967.65, stdev=276.06
     lat (usec): min=308, max=46522, avg=967.81, stdev=276.07
    clat percentiles (usec):
     |  1.00th=[  469],  5.00th=[  586], 10.00th=[  660], 20.00th=[  766],
     | 30.00th=[  840], 40.00th=[  914], 50.00th=[  971], 60.00th=[ 1029],
     | 70.00th=[ 1090], 80.00th=[ 1156], 90.00th=[ 1237], 95.00th=[ 1319],
     | 99.00th=[ 1582], 99.50th=[ 1696], 99.90th=[ 2024], 99.95th=[ 2245],
     | 99.99th=[ 3195]
   bw (  KiB/s): min=441344, max=720896, per=12.81%, avg=526075.96, stdev=42439.37, samples=313
   iops        : min=  862, max= 1408, avg=1027.47, stdev=82.88, samples=313
  lat (usec)   : 500=1.78%, 750=16.68%, 1000=36.27%
  lat (msec)   : 2=45.16%, 4=0.11%, 10=0.01%, 50=0.01%
  cpu          : usr=0.24%, sys=13.58%, ctx=172485, majf=0, minf=1551
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=163840,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=4010MiB/s (4205MB/s), 4010MiB/s-4010MiB/s (4205MB/s-4205MB/s), io=80.0GiB (85.9GB), run=20427-20427msec

Disk stats (read/write):
  nvme0n1: ios=662629/3537, merge=0/545, ticks=467846/990, in_queue=470679, util=99.84%

dd读 4.2 G/s写989M/S

[oracle@base-test-01 bigfile1]$ sudo dd if=/dev/zero of=test bs=512k count=16k conv=fsync
16384+0 records in
16384+0 records out
8589934592 bytes (8.6 GB) copied, 8.68725 s, 989 MB/s
[oracle@base-test-01 bigfile1]$ sudo dd of=/dev/zero if=test bs=512k count=16k conv=fsync
dd: fsync failed for ‘/dev/zero’: Invalid argument
16384+0 records in
16384+0 records out
8589934592 bytes (8.6 GB) copied, 2.02678 s, 4.2 GB/s
[oracle@base-test-01 bigfile1]$ 

随机写108K

[oracle@base-test-01 bigfile1]$ sudo  fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=randwrite -ioengine=psync -bs=8k -size=300M -numjobs=8 -runtime=600 -group_reporting -name=w_8k
w_8k: (g=0): rw=randwrite, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=32
...
fio-3.7
Starting 8 threads
w_8k: Laying out IO file (1 file / 300MiB)
Jobs: 8 (f=8)
w_8k: (groupid=0, jobs=8): err= 0: pid=52042: Tue Mar 24 15:41:42 2020
  write: IOPS=108k, BW=841MiB/s (881MB/s)(2400MiB/2855msec)
    clat (usec): min=19, max=1667, avg=72.30, stdev=43.59
     lat (usec): min=19, max=1667, avg=72.51, stdev=43.60
    clat percentiles (usec):
     |  1.00th=[   26],  5.00th=[   31], 10.00th=[   35], 20.00th=[   43],
     | 30.00th=[   50], 40.00th=[   56], 50.00th=[   61], 60.00th=[   69],
     | 70.00th=[   79], 80.00th=[   92], 90.00th=[  122], 95.00th=[  155],
     | 99.00th=[  235], 99.50th=[  269], 99.90th=[  334], 99.95th=[  429],
     | 99.99th=[  676]
   bw (  KiB/s): min=50752, max=139680, per=11.99%, avg=103227.60, stdev=32935.02, samples=40
   iops        : min= 6344, max=17460, avg=12903.45, stdev=4116.88, samples=40
  lat (usec)   : 20=0.01%, 50=30.94%, 100=53.07%, 250=15.27%, 500=0.68%
  lat (usec)   : 750=0.03%, 1000=0.01%
  lat (msec)   : 2=0.01%
  cpu          : usr=4.29%, sys=42.16%, ctx=719801, majf=0, minf=82
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,307200,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=841MiB/s (881MB/s), 841MiB/s-841MiB/s (881MB/s-881MB/s), io=2400MiB (2517MB), run=2855-2855msec

Disk stats (read/write):
  nvme0n1: ios=0/277947, merge=0/1, ticks=0/4945, in_queue=4998, util=60.95%
[oracle@base-test-01 bigfile1]$ 

顺序写3197MiB/s

[oracle@base-test-01 bigfile1]$ sudo  fio -filename=test1  -direct=1 -iodepth 32 -thread -rw=write -ioengine=psync -bs=512k -size=10G -numjobs=8 -runtime=600 -group_reporting -name=w_512k
w_512k: (g=0): rw=write, bs=(R) 512KiB-512KiB, (W) 512KiB-512KiB, (T) 512KiB-512KiB, ioengine=psync, iodepth=32
...
fio-3.7
Starting 8 threads
w_512k: Laying out IO file (1 file / 10240MiB)
Jobs: 8 (f=8): [W(8)][96.2%][r=0KiB/s,w=3117MiB/s][r=0,w=6233 IOPS][eta 00m:01s]
w_512k: (groupid=0, jobs=8): err= 0: pid=52162: Tue Mar 24 15:43:28 2020
  write: IOPS=6393, BW=3197MiB/s (3352MB/s)(80.0GiB/25627msec)
    clat (usec): min=141, max=46116, avg=1206.94, stdev=1445.30
     lat (usec): min=156, max=46174, avg=1246.44, stdev=1445.10
    clat percentiles (usec):
     |  1.00th=[  388],  5.00th=[  603], 10.00th=[  717], 20.00th=[  840],
     | 30.00th=[  930], 40.00th=[  996], 50.00th=[ 1057], 60.00th=[ 1139],
     | 70.00th=[ 1205], 80.00th=[ 1319], 90.00th=[ 1500], 95.00th=[ 1713],
     | 99.00th=[ 3032], 99.50th=[14353], 99.90th=[20579], 99.95th=[21103],
     | 99.99th=[26608]
   bw (  KiB/s): min=24576, max=502784, per=12.49%, avg=408752.81, stdev=117091.90, samples=408
   iops        : min=   48, max=  982, avg=798.32, stdev=228.72, samples=408
  lat (usec)   : 250=0.13%, 500=2.37%, 750=9.68%, 1000=28.11%
  lat (msec)   : 2=57.35%, 4=1.48%, 10=0.29%, 20=0.37%, 50=0.22%
  cpu          : usr=3.53%, sys=5.54%, ctx=174412, majf=0, minf=7504
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,163840,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: bw=3197MiB/s (3352MB/s), 3197MiB/s-3197MiB/s (3352MB/s-3352MB/s), io=80.0GiB (85.9GB), run=25627-25627msec

Disk stats (read/write):
  nvme0n1: ios=0/717923, merge=0/0, ticks=0/2412828, in_queue=2413712, util=99.70%
[oracle@base-test-01 bigfile1]$ 

姚远ACE CSDN认证博客专家 ACE 华为云 MVP
Oracle ACE,华为云 MVP,Oracle10g,12c OCM; MySQL 5.6,5.7,8.0 OCP;CCNA; EMC Certified; IBM P Certified; RHCE; SQLServer 764; DB2 Certified; TOEIC 890;获得过两次国家部级科技进步奖;发明过两项计算机专利。微信:yaoyuanace 邮箱:yaoyuanace(at)qq.com
已标记关键词 清除标记
相关推荐
©️2020 CSDN 皮肤主题: Age of Ai 设计师:meimeiellie 返回首页