Skip to content

Instantly share code, notes, and snippets.

@brianmichel
Last active August 22, 2025 20:24
Show Gist options
  • Select an option

  • Save brianmichel/056245a30bda5157df8a50ae6e2600a2 to your computer and use it in GitHub Desktop.

Select an option

Save brianmichel/056245a30bda5157df8a50ae6e2600a2 to your computer and use it in GitHub Desktop.

Revisions

  1. brianmichel revised this gist Aug 22, 2025. 3 changed files with 0 additions and 373 deletions.
    124 changes: 0 additions & 124 deletions bare-metal-host-simple-suite.txt
    Original file line number Diff line number Diff line change
    @@ -1,124 +0,0 @@
    Testing with async I/O engine: posixaio
    seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [R(1)][100.0%][r=15.3GiB/s][r=15.7k IOPS][eta 00m:00s]
    seq-read: (groupid=0, jobs=1): err= 0: pid=42586: Fri Aug 22 15:15:53 2025
    read: IOPS=15.7k, BW=15.3GiB/s (16.4GB/s)(459GiB/30001msec)
    clat (usec): min=60, max=222, avg=63.66, stdev= 2.44
    lat (usec): min=60, max=222, avg=63.68, stdev= 2.44
    clat percentiles (usec):
    | 1.00th=[ 62], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 63],
    | 30.00th=[ 64], 40.00th=[ 64], 50.00th=[ 64], 60.00th=[ 64],
    | 70.00th=[ 65], 80.00th=[ 65], 90.00th=[ 67], 95.00th=[ 68],
    | 99.00th=[ 73], 99.50th=[ 76], 99.90th=[ 95], 99.95th=[ 100],
    | 99.99th=[ 108]
    bw ( MiB/s): min=15337, max=15787, per=100.00%, avg=15682.03, stdev=80.43, samples=59
    iops : min=15337, max=15787, avg=15681.61, stdev=80.34, samples=59
    lat (usec) : 100=99.94%, 250=0.06%
    cpu : usr=0.51%, sys=99.48%, ctx=12, majf=0, minf=74
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=470286,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    READ: bw=15.3GiB/s (16.4GB/s), 15.3GiB/s-15.3GiB/s (16.4GB/s-16.4GB/s), io=459GiB (493GB), run=30001-30001msec
    seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [f(1)][100.0%][w=11.2GiB/s][w=11.5k IOPS][eta 00m:00s]
    seq-write: (groupid=0, jobs=1): err= 0: pid=42606: Fri Aug 22 15:16:25 2025
    write: IOPS=12.0k, BW=11.7GiB/s (12.6GB/s)(352GiB/30001msec); 0 zone resets
    clat (usec): min=70, max=115807, avg=81.20, stdev=259.16
    lat (usec): min=72, max=115809, avg=83.05, stdev=259.17
    clat percentiles (usec):
    | 1.00th=[ 74], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 76],
    | 30.00th=[ 77], 40.00th=[ 77], 50.00th=[ 77], 60.00th=[ 78],
    | 70.00th=[ 79], 80.00th=[ 80], 90.00th=[ 81], 95.00th=[ 83],
    | 99.00th=[ 92], 99.50th=[ 115], 99.90th=[ 873], 99.95th=[ 1811],
    | 99.99th=[ 6325]
    bw ( MiB/s): min= 832, max=12543, per=98.93%, avg=11896.32, stdev=2208.11, samples=60
    iops : min= 832, max=12543, avg=11895.90, stdev=2208.04, samples=60
    lat (usec) : 100=99.26%, 250=0.43%, 500=0.07%, 750=0.05%, 1000=0.10%
    lat (msec) : 2=0.06%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 250=0.01%
    cpu : usr=2.60%, sys=94.13%, ctx=8044, majf=0, minf=11
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,360774,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    WRITE: bw=11.7GiB/s (12.6GB/s), 11.7GiB/s-11.7GiB/s (12.6GB/s-12.6GB/s), io=352GiB (378GB), run=30001-30001msec
    rand-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [r(4)][100.0%][r=176MiB/s][r=45.1k IOPS][eta 00m:00s]
    rand-read: (groupid=0, jobs=4): err= 0: pid=42629: Fri Aug 22 15:16:59 2025
    read: IOPS=42.0k, BW=164MiB/s (172MB/s)(4917MiB/30002msec)
    slat (nsec): min=0, max=44000, avg=359.97, stdev=657.69
    clat (usec): min=858, max=55603, avg=1524.40, stdev=571.86
    lat (usec): min=858, max=55604, avg=1524.76, stdev=571.85
    clat percentiles (usec):
    | 1.00th=[ 1172], 5.00th=[ 1237], 10.00th=[ 1287], 20.00th=[ 1336],
    | 30.00th=[ 1369], 40.00th=[ 1401], 50.00th=[ 1434], 60.00th=[ 1467],
    | 70.00th=[ 1516], 80.00th=[ 1582], 90.00th=[ 1729], 95.00th=[ 1926],
    | 99.00th=[ 3687], 99.50th=[ 3982], 99.90th=[ 5080], 99.95th=[ 6325],
    | 99.99th=[29754]
    bw ( KiB/s): min=141032, max=185331, per=100.00%, avg=167860.66, stdev=1750.16, samples=236
    iops : min=35258, max=46332, avg=41963.85, stdev=437.53, samples=236
    lat (usec) : 1000=0.01%
    lat (msec) : 2=95.96%, 4=3.55%, 10=0.47%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%
    cpu : usr=2.55%, sys=2.57%, ctx=1083301, majf=0, minf=39
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=48.7%, 16=51.3%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.1%, 8=0.9%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=1258702,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    READ: bw=164MiB/s (172MB/s), 164MiB/s-164MiB/s (172MB/s-172MB/s), io=4917MiB (5156MB), run=30002-30002msec
    rand-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [w(4)][100.0%][w=108MiB/s][w=27.6k IOPS][eta 00m:00s]
    rand-write: (groupid=0, jobs=4): err= 0: pid=42747: Fri Aug 22 15:17:34 2025
    write: IOPS=26.7k, BW=104MiB/s (109MB/s)(3129MiB/30002msec); 0 zone resets
    slat (nsec): min=0, max=46000, avg=377.97, stdev=721.84
    clat (usec): min=706, max=46304, avg=2395.90, stdev=4005.77
    lat (usec): min=707, max=46305, avg=2396.28, stdev=4005.83
    clat percentiles (usec):
    | 1.00th=[ 1090], 5.00th=[ 1172], 10.00th=[ 1221], 20.00th=[ 1270],
    | 30.00th=[ 1319], 40.00th=[ 1352], 50.00th=[ 1401], 60.00th=[ 1434],
    | 70.00th=[ 1483], 80.00th=[ 1582], 90.00th=[ 1827], 95.00th=[10814],
    | 99.00th=[22152], 99.50th=[23462], 99.90th=[27657], 99.95th=[29230],
    | 99.99th=[33817]
    bw ( KiB/s): min=10806, max=183091, per=99.08%, avg=105816.82, stdev=6054.33, samples=240
    iops : min= 2700, max=45772, avg=26452.88, stdev=1513.63, samples=240
    lat (usec) : 750=0.01%, 1000=0.13%
    lat (msec) : 2=91.64%, 4=1.41%, 10=1.68%, 20=2.99%, 50=2.15%
    cpu : usr=1.66%, sys=1.69%, ctx=695409, majf=0, minf=38
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=48.4%, 16=51.6%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.1%, 8=0.9%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,801036,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    WRITE: bw=104MiB/s (109MB/s), 104MiB/s-104MiB/s (109MB/s-109MB/s), io=3129MiB (3281MB), run=30002-30002msec
    124 changes: 0 additions & 124 deletions tart-vm-no-options-simple-suite.txt
    Original file line number Diff line number Diff line change
    @@ -1,124 +0,0 @@
    Testing with async I/O engine: posixaio
    seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [R(1)][100.0%][r=14.5GiB/s][r=14.9k IOPS][eta 00m:00s]
    seq-read: (groupid=0, jobs=1): err= 0: pid=696: Fri Aug 22 19:28:22 2025
    read: IOPS=16.4k, BW=16.0GiB/s (17.2GB/s)(480GiB/30001msec)
    clat (usec): min=52, max=8159, avg=60.73, stdev=38.24
    lat (usec): min=52, max=8160, avg=60.78, stdev=38.98
    clat percentiles (usec):
    | 1.00th=[ 55], 5.00th=[ 56], 10.00th=[ 57], 20.00th=[ 57],
    | 30.00th=[ 58], 40.00th=[ 59], 50.00th=[ 59], 60.00th=[ 60],
    | 70.00th=[ 61], 80.00th=[ 62], 90.00th=[ 65], 95.00th=[ 71],
    | 99.00th=[ 102], 99.50th=[ 123], 99.90th=[ 200], 99.95th=[ 343],
    | 99.99th=[ 1565]
    bw ( MiB/s): min=11592, max=17158, per=100.00%, avg=16415.70, stdev=1090.23, samples=59
    iops : min=11592, max=17158, avg=16415.20, stdev=1090.19, samples=59
    lat (usec) : 100=98.89%, 250=1.04%, 500=0.03%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%
    cpu : usr=0.91%, sys=99.09%, ctx=18, majf=0, minf=74
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=491969,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    READ: bw=16.0GiB/s (17.2GB/s), 16.0GiB/s-16.0GiB/s (17.2GB/s-17.2GB/s), io=480GiB (516GB), run=30001-30001msec
    seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [W(1)][100.0%][w=12.0GiB/s][w=12.3k IOPS][eta 00m:00s]
    seq-write: (groupid=0, jobs=1): err= 0: pid=704: Fri Aug 22 19:28:54 2025
    write: IOPS=11.7k, BW=11.4GiB/s (12.2GB/s)(342GiB/30001msec); 0 zone resets
    clat (usec): min=63, max=17462, avg=83.35, stdev=90.22
    lat (usec): min=65, max=17464, avg=85.50, stdev=90.37
    clat percentiles (usec):
    | 1.00th=[ 71], 5.00th=[ 72], 10.00th=[ 73], 20.00th=[ 74],
    | 30.00th=[ 74], 40.00th=[ 75], 50.00th=[ 76], 60.00th=[ 77],
    | 70.00th=[ 79], 80.00th=[ 81], 90.00th=[ 86], 95.00th=[ 97],
    | 99.00th=[ 239], 99.50th=[ 441], 99.90th=[ 955], 99.95th=[ 1369],
    | 99.99th=[ 3326]
    bw ( MiB/s): min= 1431, max=12802, per=100.00%, avg=11669.95, stdev=2141.32, samples=59
    iops : min= 1431, max=12802, avg=11669.44, stdev=2141.25, samples=59
    lat (usec) : 100=95.45%, 250=3.56%, 500=0.64%, 750=0.10%, 1000=0.15%
    lat (msec) : 2=0.05%, 4=0.02%, 10=0.01%, 20=0.01%
    cpu : usr=3.23%, sys=96.26%, ctx=770, majf=0, minf=11
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,349861,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    WRITE: bw=11.4GiB/s (12.2GB/s), 11.4GiB/s-11.4GiB/s (12.2GB/s-12.2GB/s), io=342GiB (367GB), run=30001-30001msec
    rand-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [r(4)][100.0%][r=186MiB/s][r=47.7k IOPS][eta 00m:00s]
    rand-read: (groupid=0, jobs=4): err= 0: pid=714: Fri Aug 22 19:29:30 2025
    read: IOPS=48.9k, BW=191MiB/s (200MB/s)(5726MiB/30002msec)
    slat (nsec): min=0, max=699000, avg=710.78, stdev=1047.48
    clat (usec): min=102, max=17113, avg=1306.96, stdev=380.52
    lat (usec): min=148, max=17114, avg=1307.67, stdev=380.52
    clat percentiles (usec):
    | 1.00th=[ 766], 5.00th=[ 898], 10.00th=[ 979], 20.00th=[ 1074],
    | 30.00th=[ 1139], 40.00th=[ 1205], 50.00th=[ 1270], 60.00th=[ 1336],
    | 70.00th=[ 1418], 80.00th=[ 1500], 90.00th=[ 1647], 95.00th=[ 1778],
    | 99.00th=[ 2180], 99.50th=[ 2540], 99.90th=[ 4228], 99.95th=[ 5735],
    | 99.99th=[14222]
    bw ( KiB/s): min=155388, max=221817, per=100.00%, avg=195604.73, stdev=3433.39, samples=236
    iops : min=38846, max=55452, avg=48899.83, stdev=858.34, samples=236
    lat (usec) : 250=0.01%, 500=0.01%, 750=0.75%, 1000=11.34%
    lat (msec) : 2=86.13%, 4=1.65%, 10=0.09%, 20=0.03%
    cpu : usr=4.42%, sys=4.90%, ctx=756208, majf=0, minf=38
    IO depths : 1=0.1%, 2=0.1%, 4=0.5%, 8=58.3%, 16=41.2%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=97.7%, 8=2.1%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=1465974,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    READ: bw=191MiB/s (200MB/s), 191MiB/s-191MiB/s (200MB/s-200MB/s), io=5726MiB (6005MB), run=30002-30002msec
    rand-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [w(4)][100.0%][w=27.9MiB/s][w=7131 IOPS][eta 00m:00s]
    rand-write: (groupid=0, jobs=4): err= 0: pid=727: Fri Aug 22 19:30:05 2025
    write: IOPS=17.4k, BW=68.1MiB/s (71.4MB/s)(2043MiB/30007msec); 0 zone resets
    slat (nsec): min=0, max=4641.0k, avg=795.02, stdev=9453.01
    clat (usec): min=87, max=245131, avg=3668.14, stdev=4107.40
    lat (usec): min=137, max=245132, avg=3668.94, stdev=4107.58
    clat percentiles (usec):
    | 1.00th=[ 1926], 5.00th=[ 2147], 10.00th=[ 2278], 20.00th=[ 2442],
    | 30.00th=[ 2606], 40.00th=[ 2737], 50.00th=[ 2868], 60.00th=[ 2999],
    | 70.00th=[ 3195], 80.00th=[ 3556], 90.00th=[ 4948], 95.00th=[ 8029],
    | 99.00th=[ 18744], 99.50th=[ 21627], 99.90th=[ 40633], 99.95th=[ 57410],
    | 99.99th=[235930]
    bw ( KiB/s): min=19714, max=94453, per=99.37%, avg=69291.70, stdev=4785.17, samples=240
    iops : min= 4926, max=23612, avg=17321.73, stdev=1196.28, samples=240
    lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01%
    lat (msec) : 2=1.91%, 4=83.03%, 10=11.74%, 20=2.58%, 50=0.67%
    lat (msec) : 100=0.05%, 250=0.01%
    cpu : usr=2.05%, sys=2.62%, ctx=390457, majf=0, minf=39
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.4%, 16=49.6%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.0%, 8=1.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,523119,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    WRITE: bw=68.1MiB/s (71.4MB/s), 68.1MiB/s-68.1MiB/s (71.4MB/s-71.4MB/s), io=2043MiB (2143MB), run=30007-30007msec
    125 changes: 0 additions & 125 deletions tart-vm-sync-none-simple-suite.txt
    Original file line number Diff line number Diff line change
    @@ -1,125 +0,0 @@
    Testing with async I/O engine: posixaio
    seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [R(1)][100.0%][r=14.1GiB/s][r=14.4k IOPS][eta 00m:00s]
    seq-read: (groupid=0, jobs=1): err= 0: pid=728: Fri Aug 22 19:36:00 2025
    read: IOPS=13.8k, BW=13.5GiB/s (14.5GB/s)(405GiB/30001msec)
    clat (usec): min=60, max=42925, avg=72.10, stdev=79.60
    lat (usec): min=60, max=42925, avg=72.14, stdev=79.61
    clat percentiles (usec):
    | 1.00th=[ 64], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 67],
    | 30.00th=[ 68], 40.00th=[ 68], 50.00th=[ 69], 60.00th=[ 70],
    | 70.00th=[ 71], 80.00th=[ 74], 90.00th=[ 80], 95.00th=[ 90],
    | 99.00th=[ 129], 99.50th=[ 157], 99.90th=[ 293], 99.95th=[ 424],
    | 99.99th=[ 1221]
    bw ( MiB/s): min= 8617, max=14838, per=100.00%, avg=13830.91, stdev=878.86, samples=59
    iops : min= 8617, max=14838, avg=13830.44, stdev=878.88, samples=59
    lat (usec) : 100=97.17%, 250=2.69%, 500=0.10%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    cpu : usr=0.86%, sys=99.13%, ctx=15, majf=0, minf=75
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=414541,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    READ: bw=13.5GiB/s (14.5GB/s), 13.5GiB/s-13.5GiB/s (14.5GB/s-14.5GB/s), io=405GiB (435GB), run=30001-30001msec
    seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [f(1)][100.0%][w=10.7GiB/s][w=11.0k IOPS][eta 00m:00s]
    seq-write: (groupid=0, jobs=1): err= 0: pid=736: Fri Aug 22 19:36:31 2025
    write: IOPS=11.2k, BW=10.9GiB/s (11.7GB/s)(327GiB/30001msec); 0 zone resets
    clat (usec): min=65, max=12770, avg=87.03, stdev=74.06
    lat (usec): min=67, max=19127, avg=89.26, stdev=80.53
    clat percentiles (usec):
    | 1.00th=[ 71], 5.00th=[ 73], 10.00th=[ 74], 20.00th=[ 75],
    | 30.00th=[ 77], 40.00th=[ 79], 50.00th=[ 80], 60.00th=[ 82],
    | 70.00th=[ 83], 80.00th=[ 85], 90.00th=[ 90], 95.00th=[ 102],
    | 99.00th=[ 326], 99.50th=[ 519], 99.90th=[ 922], 99.95th=[ 1004],
    | 99.99th=[ 2114]
    bw ( MiB/s): min= 1677, max=12218, per=100.00%, avg=11177.83, stdev=2152.19, samples=59
    iops : min= 1677, max=12218, avg=11177.36, stdev=2152.14, samples=59
    lat (usec) : 100=94.33%, 250=4.39%, 500=0.70%, 750=0.37%, 1000=0.17%
    lat (msec) : 2=0.04%, 4=0.01%, 10=0.01%, 20=0.01%
    cpu : usr=3.13%, sys=95.61%, ctx=2488, majf=0, minf=11
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,335168,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    WRITE: bw=10.9GiB/s (11.7GB/s), 10.9GiB/s-10.9GiB/s (11.7GB/s-11.7GB/s), io=327GiB (351GB), run=30001-30001msec
    rand-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [r(4)][100.0%][r=204MiB/s][r=52.2k IOPS][eta 00m:00s]
    rand-read: (groupid=0, jobs=4): err= 0: pid=751: Fri Aug 22 19:37:10 2025
    read: IOPS=41.4k, BW=162MiB/s (170MB/s)(4854MiB/30002msec)
    slat (nsec): min=0, max=3849.0k, avg=704.57, stdev=4341.73
    clat (usec): min=19, max=103071, avg=1542.57, stdev=1326.84
    lat (usec): min=41, max=103072, avg=1543.27, stdev=1326.96
    clat percentiles (usec):
    | 1.00th=[ 807], 5.00th=[ 963], 10.00th=[ 1057], 20.00th=[ 1172],
    | 30.00th=[ 1254], 40.00th=[ 1336], 50.00th=[ 1418], 60.00th=[ 1500],
    | 70.00th=[ 1598], 80.00th=[ 1729], 90.00th=[ 1942], 95.00th=[ 2180],
    | 99.00th=[ 3752], 99.50th=[ 5669], 99.90th=[17433], 99.95th=[28967],
    | 99.99th=[50594]
    bw ( KiB/s): min=96931, max=211336, per=99.88%, avg=165460.69, stdev=6196.21, samples=236
    iops : min=24232, max=52834, avg=41363.85, stdev=1549.04, samples=236
    lat (usec) : 20=0.01%, 50=0.01%, 100=0.01%, 250=0.01%, 500=0.01%
    lat (usec) : 750=0.44%, 1000=6.40%
    lat (msec) : 2=84.97%, 4=7.28%, 10=0.66%, 20=0.15%, 50=0.07%
    lat (msec) : 100=0.01%, 250=0.01%
    cpu : usr=3.97%, sys=4.38%, ctx=643923, majf=0, minf=39
    IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=54.3%, 16=45.4%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=98.2%, 8=1.7%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=1242543,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    READ: bw=162MiB/s (170MB/s), 162MiB/s-162MiB/s (170MB/s-170MB/s), io=4854MiB (5089MB), run=30002-30002msec
    rand-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [f(1),E(3)][100.0%][w=3549KiB/s][w=887 IOPS][eta 00m:00s]
    rand-write: (groupid=0, jobs=4): err= 0: pid=772: Fri Aug 22 19:37:46 2025
    write: IOPS=20.4k, BW=79.8MiB/s (83.7MB/s)(2395MiB/30003msec); 0 zone resets
    slat (nsec): min=0, max=1880.0k, avg=709.88, stdev=2826.83
    clat (usec): min=1185, max=123495, avg=3128.70, stdev=2583.57
    lat (usec): min=1185, max=123496, avg=3129.41, stdev=2583.69
    clat percentiles (usec):
    | 1.00th=[ 1778], 5.00th=[ 1975], 10.00th=[ 2089], 20.00th=[ 2245],
    | 30.00th=[ 2376], 40.00th=[ 2507], 50.00th=[ 2606], 60.00th=[ 2769],
    | 70.00th=[ 2933], 80.00th=[ 3228], 90.00th=[ 3785], 95.00th=[ 5473],
    | 99.00th=[15139], 99.50th=[17695], 99.90th=[24773], 99.95th=[34866],
    | 99.99th=[74974]
    bw ( KiB/s): min=39056, max=104244, per=99.28%, avg=81170.23, stdev=3927.76, samples=240
    iops : min= 9762, max=26058, avg=20291.15, stdev=981.90, samples=240
    lat (msec) : 2=5.70%, 4=85.84%, 10=6.27%, 20=1.93%, 50=0.24%
    lat (msec) : 100=0.02%, 250=0.01%
    cpu : usr=2.22%, sys=2.66%, ctx=453148, majf=0, minf=38
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.4%, 16=49.6%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.0%, 8=1.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,613237,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    WRITE: bw=79.8MiB/s (83.7MB/s), 79.8MiB/s-79.8MiB/s (83.7MB/s-83.7MB/s), io=2395MiB (2512MB), run=30003-30003msec
  2. brianmichel revised this gist Aug 22, 2025. 3 changed files with 373 additions and 0 deletions.
    124 changes: 124 additions & 0 deletions bare-metal-host-simple-suite.txt
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,124 @@
    Testing with async I/O engine: posixaio
    seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [R(1)][100.0%][r=15.3GiB/s][r=15.7k IOPS][eta 00m:00s]
    seq-read: (groupid=0, jobs=1): err= 0: pid=42586: Fri Aug 22 15:15:53 2025
    read: IOPS=15.7k, BW=15.3GiB/s (16.4GB/s)(459GiB/30001msec)
    clat (usec): min=60, max=222, avg=63.66, stdev= 2.44
    lat (usec): min=60, max=222, avg=63.68, stdev= 2.44
    clat percentiles (usec):
    | 1.00th=[ 62], 5.00th=[ 63], 10.00th=[ 63], 20.00th=[ 63],
    | 30.00th=[ 64], 40.00th=[ 64], 50.00th=[ 64], 60.00th=[ 64],
    | 70.00th=[ 65], 80.00th=[ 65], 90.00th=[ 67], 95.00th=[ 68],
    | 99.00th=[ 73], 99.50th=[ 76], 99.90th=[ 95], 99.95th=[ 100],
    | 99.99th=[ 108]
    bw ( MiB/s): min=15337, max=15787, per=100.00%, avg=15682.03, stdev=80.43, samples=59
    iops : min=15337, max=15787, avg=15681.61, stdev=80.34, samples=59
    lat (usec) : 100=99.94%, 250=0.06%
    cpu : usr=0.51%, sys=99.48%, ctx=12, majf=0, minf=74
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=470286,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    READ: bw=15.3GiB/s (16.4GB/s), 15.3GiB/s-15.3GiB/s (16.4GB/s-16.4GB/s), io=459GiB (493GB), run=30001-30001msec
    seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [f(1)][100.0%][w=11.2GiB/s][w=11.5k IOPS][eta 00m:00s]
    seq-write: (groupid=0, jobs=1): err= 0: pid=42606: Fri Aug 22 15:16:25 2025
    write: IOPS=12.0k, BW=11.7GiB/s (12.6GB/s)(352GiB/30001msec); 0 zone resets
    clat (usec): min=70, max=115807, avg=81.20, stdev=259.16
    lat (usec): min=72, max=115809, avg=83.05, stdev=259.17
    clat percentiles (usec):
    | 1.00th=[ 74], 5.00th=[ 75], 10.00th=[ 75], 20.00th=[ 76],
    | 30.00th=[ 77], 40.00th=[ 77], 50.00th=[ 77], 60.00th=[ 78],
    | 70.00th=[ 79], 80.00th=[ 80], 90.00th=[ 81], 95.00th=[ 83],
    | 99.00th=[ 92], 99.50th=[ 115], 99.90th=[ 873], 99.95th=[ 1811],
    | 99.99th=[ 6325]
    bw ( MiB/s): min= 832, max=12543, per=98.93%, avg=11896.32, stdev=2208.11, samples=60
    iops : min= 832, max=12543, avg=11895.90, stdev=2208.04, samples=60
    lat (usec) : 100=99.26%, 250=0.43%, 500=0.07%, 750=0.05%, 1000=0.10%
    lat (msec) : 2=0.06%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    lat (msec) : 250=0.01%
    cpu : usr=2.60%, sys=94.13%, ctx=8044, majf=0, minf=11
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,360774,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    WRITE: bw=11.7GiB/s (12.6GB/s), 11.7GiB/s-11.7GiB/s (12.6GB/s-12.6GB/s), io=352GiB (378GB), run=30001-30001msec
    rand-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [r(4)][100.0%][r=176MiB/s][r=45.1k IOPS][eta 00m:00s]
    rand-read: (groupid=0, jobs=4): err= 0: pid=42629: Fri Aug 22 15:16:59 2025
    read: IOPS=42.0k, BW=164MiB/s (172MB/s)(4917MiB/30002msec)
    slat (nsec): min=0, max=44000, avg=359.97, stdev=657.69
    clat (usec): min=858, max=55603, avg=1524.40, stdev=571.86
    lat (usec): min=858, max=55604, avg=1524.76, stdev=571.85
    clat percentiles (usec):
    | 1.00th=[ 1172], 5.00th=[ 1237], 10.00th=[ 1287], 20.00th=[ 1336],
    | 30.00th=[ 1369], 40.00th=[ 1401], 50.00th=[ 1434], 60.00th=[ 1467],
    | 70.00th=[ 1516], 80.00th=[ 1582], 90.00th=[ 1729], 95.00th=[ 1926],
    | 99.00th=[ 3687], 99.50th=[ 3982], 99.90th=[ 5080], 99.95th=[ 6325],
    | 99.99th=[29754]
    bw ( KiB/s): min=141032, max=185331, per=100.00%, avg=167860.66, stdev=1750.16, samples=236
    iops : min=35258, max=46332, avg=41963.85, stdev=437.53, samples=236
    lat (usec) : 1000=0.01%
    lat (msec) : 2=95.96%, 4=3.55%, 10=0.47%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%
    cpu : usr=2.55%, sys=2.57%, ctx=1083301, majf=0, minf=39
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=48.7%, 16=51.3%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.1%, 8=0.9%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=1258702,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    READ: bw=164MiB/s (172MB/s), 164MiB/s-164MiB/s (172MB/s-172MB/s), io=4917MiB (5156MB), run=30002-30002msec
    rand-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [w(4)][100.0%][w=108MiB/s][w=27.6k IOPS][eta 00m:00s]
    rand-write: (groupid=0, jobs=4): err= 0: pid=42747: Fri Aug 22 15:17:34 2025
    write: IOPS=26.7k, BW=104MiB/s (109MB/s)(3129MiB/30002msec); 0 zone resets
    slat (nsec): min=0, max=46000, avg=377.97, stdev=721.84
    clat (usec): min=706, max=46304, avg=2395.90, stdev=4005.77
    lat (usec): min=707, max=46305, avg=2396.28, stdev=4005.83
    clat percentiles (usec):
    | 1.00th=[ 1090], 5.00th=[ 1172], 10.00th=[ 1221], 20.00th=[ 1270],
    | 30.00th=[ 1319], 40.00th=[ 1352], 50.00th=[ 1401], 60.00th=[ 1434],
    | 70.00th=[ 1483], 80.00th=[ 1582], 90.00th=[ 1827], 95.00th=[10814],
    | 99.00th=[22152], 99.50th=[23462], 99.90th=[27657], 99.95th=[29230],
    | 99.99th=[33817]
    bw ( KiB/s): min=10806, max=183091, per=99.08%, avg=105816.82, stdev=6054.33, samples=240
    iops : min= 2700, max=45772, avg=26452.88, stdev=1513.63, samples=240
    lat (usec) : 750=0.01%, 1000=0.13%
    lat (msec) : 2=91.64%, 4=1.41%, 10=1.68%, 20=2.99%, 50=2.15%
    cpu : usr=1.66%, sys=1.69%, ctx=695409, majf=0, minf=38
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=48.4%, 16=51.6%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.1%, 8=0.9%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,801036,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    WRITE: bw=104MiB/s (109MB/s), 104MiB/s-104MiB/s (109MB/s-109MB/s), io=3129MiB (3281MB), run=30002-30002msec
    124 changes: 124 additions & 0 deletions tart-vm-no-options-simple-suite.txt
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,124 @@
    Testing with async I/O engine: posixaio
    seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [R(1)][100.0%][r=14.5GiB/s][r=14.9k IOPS][eta 00m:00s]
    seq-read: (groupid=0, jobs=1): err= 0: pid=696: Fri Aug 22 19:28:22 2025
    read: IOPS=16.4k, BW=16.0GiB/s (17.2GB/s)(480GiB/30001msec)
    clat (usec): min=52, max=8159, avg=60.73, stdev=38.24
    lat (usec): min=52, max=8160, avg=60.78, stdev=38.98
    clat percentiles (usec):
    | 1.00th=[ 55], 5.00th=[ 56], 10.00th=[ 57], 20.00th=[ 57],
    | 30.00th=[ 58], 40.00th=[ 59], 50.00th=[ 59], 60.00th=[ 60],
    | 70.00th=[ 61], 80.00th=[ 62], 90.00th=[ 65], 95.00th=[ 71],
    | 99.00th=[ 102], 99.50th=[ 123], 99.90th=[ 200], 99.95th=[ 343],
    | 99.99th=[ 1565]
    bw ( MiB/s): min=11592, max=17158, per=100.00%, avg=16415.70, stdev=1090.23, samples=59
    iops : min=11592, max=17158, avg=16415.20, stdev=1090.19, samples=59
    lat (usec) : 100=98.89%, 250=1.04%, 500=0.03%, 750=0.01%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%
    cpu : usr=0.91%, sys=99.09%, ctx=18, majf=0, minf=74
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=491969,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    READ: bw=16.0GiB/s (17.2GB/s), 16.0GiB/s-16.0GiB/s (17.2GB/s-17.2GB/s), io=480GiB (516GB), run=30001-30001msec
    seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [W(1)][100.0%][w=12.0GiB/s][w=12.3k IOPS][eta 00m:00s]
    seq-write: (groupid=0, jobs=1): err= 0: pid=704: Fri Aug 22 19:28:54 2025
    write: IOPS=11.7k, BW=11.4GiB/s (12.2GB/s)(342GiB/30001msec); 0 zone resets
    clat (usec): min=63, max=17462, avg=83.35, stdev=90.22
    lat (usec): min=65, max=17464, avg=85.50, stdev=90.37
    clat percentiles (usec):
    | 1.00th=[ 71], 5.00th=[ 72], 10.00th=[ 73], 20.00th=[ 74],
    | 30.00th=[ 74], 40.00th=[ 75], 50.00th=[ 76], 60.00th=[ 77],
    | 70.00th=[ 79], 80.00th=[ 81], 90.00th=[ 86], 95.00th=[ 97],
    | 99.00th=[ 239], 99.50th=[ 441], 99.90th=[ 955], 99.95th=[ 1369],
    | 99.99th=[ 3326]
    bw ( MiB/s): min= 1431, max=12802, per=100.00%, avg=11669.95, stdev=2141.32, samples=59
    iops : min= 1431, max=12802, avg=11669.44, stdev=2141.25, samples=59
    lat (usec) : 100=95.45%, 250=3.56%, 500=0.64%, 750=0.10%, 1000=0.15%
    lat (msec) : 2=0.05%, 4=0.02%, 10=0.01%, 20=0.01%
    cpu : usr=3.23%, sys=96.26%, ctx=770, majf=0, minf=11
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,349861,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    WRITE: bw=11.4GiB/s (12.2GB/s), 11.4GiB/s-11.4GiB/s (12.2GB/s-12.2GB/s), io=342GiB (367GB), run=30001-30001msec
    rand-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [r(4)][100.0%][r=186MiB/s][r=47.7k IOPS][eta 00m:00s]
    rand-read: (groupid=0, jobs=4): err= 0: pid=714: Fri Aug 22 19:29:30 2025
    read: IOPS=48.9k, BW=191MiB/s (200MB/s)(5726MiB/30002msec)
    slat (nsec): min=0, max=699000, avg=710.78, stdev=1047.48
    clat (usec): min=102, max=17113, avg=1306.96, stdev=380.52
    lat (usec): min=148, max=17114, avg=1307.67, stdev=380.52
    clat percentiles (usec):
    | 1.00th=[ 766], 5.00th=[ 898], 10.00th=[ 979], 20.00th=[ 1074],
    | 30.00th=[ 1139], 40.00th=[ 1205], 50.00th=[ 1270], 60.00th=[ 1336],
    | 70.00th=[ 1418], 80.00th=[ 1500], 90.00th=[ 1647], 95.00th=[ 1778],
    | 99.00th=[ 2180], 99.50th=[ 2540], 99.90th=[ 4228], 99.95th=[ 5735],
    | 99.99th=[14222]
    bw ( KiB/s): min=155388, max=221817, per=100.00%, avg=195604.73, stdev=3433.39, samples=236
    iops : min=38846, max=55452, avg=48899.83, stdev=858.34, samples=236
    lat (usec) : 250=0.01%, 500=0.01%, 750=0.75%, 1000=11.34%
    lat (msec) : 2=86.13%, 4=1.65%, 10=0.09%, 20=0.03%
    cpu : usr=4.42%, sys=4.90%, ctx=756208, majf=0, minf=38
    IO depths : 1=0.1%, 2=0.1%, 4=0.5%, 8=58.3%, 16=41.2%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=97.7%, 8=2.1%, 16=0.2%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=1465974,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    READ: bw=191MiB/s (200MB/s), 191MiB/s-191MiB/s (200MB/s-200MB/s), io=5726MiB (6005MB), run=30002-30002msec
    rand-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [w(4)][100.0%][w=27.9MiB/s][w=7131 IOPS][eta 00m:00s]
    rand-write: (groupid=0, jobs=4): err= 0: pid=727: Fri Aug 22 19:30:05 2025
    write: IOPS=17.4k, BW=68.1MiB/s (71.4MB/s)(2043MiB/30007msec); 0 zone resets
    slat (nsec): min=0, max=4641.0k, avg=795.02, stdev=9453.01
    clat (usec): min=87, max=245131, avg=3668.14, stdev=4107.40
    lat (usec): min=137, max=245132, avg=3668.94, stdev=4107.58
    clat percentiles (usec):
    | 1.00th=[ 1926], 5.00th=[ 2147], 10.00th=[ 2278], 20.00th=[ 2442],
    | 30.00th=[ 2606], 40.00th=[ 2737], 50.00th=[ 2868], 60.00th=[ 2999],
    | 70.00th=[ 3195], 80.00th=[ 3556], 90.00th=[ 4948], 95.00th=[ 8029],
    | 99.00th=[ 18744], 99.50th=[ 21627], 99.90th=[ 40633], 99.95th=[ 57410],
    | 99.99th=[235930]
    bw ( KiB/s): min=19714, max=94453, per=99.37%, avg=69291.70, stdev=4785.17, samples=240
    iops : min= 4926, max=23612, avg=17321.73, stdev=1196.28, samples=240
    lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01%
    lat (msec) : 2=1.91%, 4=83.03%, 10=11.74%, 20=2.58%, 50=0.67%
    lat (msec) : 100=0.05%, 250=0.01%
    cpu : usr=2.05%, sys=2.62%, ctx=390457, majf=0, minf=39
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.4%, 16=49.6%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.0%, 8=1.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,523119,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    WRITE: bw=68.1MiB/s (71.4MB/s), 68.1MiB/s-68.1MiB/s (71.4MB/s-71.4MB/s), io=2043MiB (2143MB), run=30007-30007msec
    125 changes: 125 additions & 0 deletions tart-vm-sync-none-simple-suite.txt
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,125 @@
    Testing with async I/O engine: posixaio
    seq-read: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [R(1)][100.0%][r=14.1GiB/s][r=14.4k IOPS][eta 00m:00s]
    seq-read: (groupid=0, jobs=1): err= 0: pid=728: Fri Aug 22 19:36:00 2025
    read: IOPS=13.8k, BW=13.5GiB/s (14.5GB/s)(405GiB/30001msec)
    clat (usec): min=60, max=42925, avg=72.10, stdev=79.60
    lat (usec): min=60, max=42925, avg=72.14, stdev=79.61
    clat percentiles (usec):
    | 1.00th=[ 64], 5.00th=[ 65], 10.00th=[ 65], 20.00th=[ 67],
    | 30.00th=[ 68], 40.00th=[ 68], 50.00th=[ 69], 60.00th=[ 70],
    | 70.00th=[ 71], 80.00th=[ 74], 90.00th=[ 80], 95.00th=[ 90],
    | 99.00th=[ 129], 99.50th=[ 157], 99.90th=[ 293], 99.95th=[ 424],
    | 99.99th=[ 1221]
    bw ( MiB/s): min= 8617, max=14838, per=100.00%, avg=13830.91, stdev=878.86, samples=59
    iops : min= 8617, max=14838, avg=13830.44, stdev=878.88, samples=59
    lat (usec) : 100=97.17%, 250=2.69%, 500=0.10%, 750=0.02%, 1000=0.01%
    lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
    cpu : usr=0.86%, sys=99.13%, ctx=15, majf=0, minf=75
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=414541,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    READ: bw=13.5GiB/s (14.5GB/s), 13.5GiB/s-13.5GiB/s (14.5GB/s-14.5GB/s), io=405GiB (435GB), run=30001-30001msec
    seq-write: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=psync, iodepth=1
    fio-3.40
    Starting 1 process
    seq-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [f(1)][100.0%][w=10.7GiB/s][w=11.0k IOPS][eta 00m:00s]
    seq-write: (groupid=0, jobs=1): err= 0: pid=736: Fri Aug 22 19:36:31 2025
    write: IOPS=11.2k, BW=10.9GiB/s (11.7GB/s)(327GiB/30001msec); 0 zone resets
    clat (usec): min=65, max=12770, avg=87.03, stdev=74.06
    lat (usec): min=67, max=19127, avg=89.26, stdev=80.53
    clat percentiles (usec):
    | 1.00th=[ 71], 5.00th=[ 73], 10.00th=[ 74], 20.00th=[ 75],
    | 30.00th=[ 77], 40.00th=[ 79], 50.00th=[ 80], 60.00th=[ 82],
    | 70.00th=[ 83], 80.00th=[ 85], 90.00th=[ 90], 95.00th=[ 102],
    | 99.00th=[ 326], 99.50th=[ 519], 99.90th=[ 922], 99.95th=[ 1004],
    | 99.99th=[ 2114]
    bw ( MiB/s): min= 1677, max=12218, per=100.00%, avg=11177.83, stdev=2152.19, samples=59
    iops : min= 1677, max=12218, avg=11177.36, stdev=2152.14, samples=59
    lat (usec) : 100=94.33%, 250=4.39%, 500=0.70%, 750=0.37%, 1000=0.17%
    lat (msec) : 2=0.04%, 4=0.01%, 10=0.01%, 20=0.01%
    cpu : usr=3.13%, sys=95.61%, ctx=2488, majf=0, minf=11
    IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,335168,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=1

    Run status group 0 (all jobs):
    WRITE: bw=10.9GiB/s (11.7GB/s), 10.9GiB/s-10.9GiB/s (11.7GB/s-11.7GB/s), io=327GiB (351GB), run=30001-30001msec
    rand-read: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    rand-read: Laying out IO file (1 file / 2048MiB)
    Jobs: 4 (f=4): [r(4)][100.0%][r=204MiB/s][r=52.2k IOPS][eta 00m:00s]
    rand-read: (groupid=0, jobs=4): err= 0: pid=751: Fri Aug 22 19:37:10 2025
    read: IOPS=41.4k, BW=162MiB/s (170MB/s)(4854MiB/30002msec)
    slat (nsec): min=0, max=3849.0k, avg=704.57, stdev=4341.73
    clat (usec): min=19, max=103071, avg=1542.57, stdev=1326.84
    lat (usec): min=41, max=103072, avg=1543.27, stdev=1326.96
    clat percentiles (usec):
    | 1.00th=[ 807], 5.00th=[ 963], 10.00th=[ 1057], 20.00th=[ 1172],
    | 30.00th=[ 1254], 40.00th=[ 1336], 50.00th=[ 1418], 60.00th=[ 1500],
    | 70.00th=[ 1598], 80.00th=[ 1729], 90.00th=[ 1942], 95.00th=[ 2180],
    | 99.00th=[ 3752], 99.50th=[ 5669], 99.90th=[17433], 99.95th=[28967],
    | 99.99th=[50594]
    bw ( KiB/s): min=96931, max=211336, per=99.88%, avg=165460.69, stdev=6196.21, samples=236
    iops : min=24232, max=52834, avg=41363.85, stdev=1549.04, samples=236
    lat (usec) : 20=0.01%, 50=0.01%, 100=0.01%, 250=0.01%, 500=0.01%
    lat (usec) : 750=0.44%, 1000=6.40%
    lat (msec) : 2=84.97%, 4=7.28%, 10=0.66%, 20=0.15%, 50=0.07%
    lat (msec) : 100=0.01%, 250=0.01%
    cpu : usr=3.97%, sys=4.38%, ctx=643923, majf=0, minf=39
    IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=54.3%, 16=45.4%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=98.2%, 8=1.7%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=1242543,0,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    READ: bw=162MiB/s (170MB/s), 162MiB/s-162MiB/s (170MB/s-170MB/s), io=4854MiB (5089MB), run=30002-30002msec
    rand-write: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=64
    ...
    fio-3.40
    Starting 4 processes
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    rand-write: Laying out IO file (1 file / 2048MiB)
    Jobs: 1 (f=1): [f(1),E(3)][100.0%][w=3549KiB/s][w=887 IOPS][eta 00m:00s]
    rand-write: (groupid=0, jobs=4): err= 0: pid=772: Fri Aug 22 19:37:46 2025
    write: IOPS=20.4k, BW=79.8MiB/s (83.7MB/s)(2395MiB/30003msec); 0 zone resets
    slat (nsec): min=0, max=1880.0k, avg=709.88, stdev=2826.83
    clat (usec): min=1185, max=123495, avg=3128.70, stdev=2583.57
    lat (usec): min=1185, max=123496, avg=3129.41, stdev=2583.69
    clat percentiles (usec):
    | 1.00th=[ 1778], 5.00th=[ 1975], 10.00th=[ 2089], 20.00th=[ 2245],
    | 30.00th=[ 2376], 40.00th=[ 2507], 50.00th=[ 2606], 60.00th=[ 2769],
    | 70.00th=[ 2933], 80.00th=[ 3228], 90.00th=[ 3785], 95.00th=[ 5473],
    | 99.00th=[15139], 99.50th=[17695], 99.90th=[24773], 99.95th=[34866],
    | 99.99th=[74974]
    bw ( KiB/s): min=39056, max=104244, per=99.28%, avg=81170.23, stdev=3927.76, samples=240
    iops : min= 9762, max=26058, avg=20291.15, stdev=981.90, samples=240
    lat (msec) : 2=5.70%, 4=85.84%, 10=6.27%, 20=1.93%, 50=0.24%
    lat (msec) : 100=0.02%, 250=0.01%
    cpu : usr=2.22%, sys=2.66%, ctx=453148, majf=0, minf=38
    IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=50.4%, 16=49.6%, 32=0.0%, >=64=0.0%
    submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
    complete : 0=0.0%, 4=99.0%, 8=1.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
    issued rwts: total=0,613237,0,0 short=0,0,0,0 dropped=0,0,0,0
    latency : target=0, window=0, percentile=100.00%, depth=64

    Run status group 0 (all jobs):
    WRITE: bw=79.8MiB/s (83.7MB/s), 79.8MiB/s-79.8MiB/s (83.7MB/s-83.7MB/s), io=2395MiB (2512MB), run=30003-30003msec
  3. brianmichel created this gist Aug 22, 2025.
    30 changes: 30 additions & 0 deletions macos-iops-testing.sh
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,30 @@
    #!/bin/bash
    TEST_DIR="/tmp/fio-test"
    mkdir -p $TEST_DIR

    # For macOS, use posixaio engine
    ENGINE="posixaio"

    echo "Testing with async I/O engine: $ENGINE"

    # Sequential tests (these are already good)
    fio --name=seq-read --rw=read --size=2G --bs=1M \
    --numjobs=1 --time_based --runtime=30 \
    --group_reporting --directory=$TEST_DIR

    fio --name=seq-write --rw=write --size=2G --bs=1M \
    --numjobs=1 --time_based --runtime=30 \
    --group_reporting --directory=$TEST_DIR

    # Random tests with proper async I/O
    fio --name=rand-read --rw=randread --size=2G --bs=4k \
    --numjobs=4 --iodepth=64 --ioengine=$ENGINE \
    --time_based --runtime=30 --group_reporting \
    --directory=$TEST_DIR

    fio --name=rand-write --rw=randwrite --size=2G --bs=4k \
    --numjobs=4 --iodepth=64 --ioengine=$ENGINE \
    --time_based --runtime=30 --group_reporting \
    --directory=$TEST_DIR

    rm -rf $TEST_DIR/*