Re: block covering in fio-1.16.9

On Thu, Jul 26 2007, ljzhang,Yaxin Hu,Jianchao Tang wrote:
> 1. The job file causing the problem:
> ----------cover-------------------
> [global]
> directory=./temp
> nrfiles=1
> rw=randread
> size=256K
> bsrange=4k-16k
> thread
> [cover]
> description="Not covering every block."
> -----------------------------------------
>
> According to HOWTO,the job file will makes fio to cover every block of
> the 256k file.But running that, we sometimes got a io gross more than
> 256k:
> ----------------------------------------
> nonggia_at_nonggia-desktop:~$ fio --version
> fio 1.16.9
> nonggia_at_nonggia-desktop:~/fio$ fio cover
> cover: (g=0): rw=randread, bs=4K-16K/4K-4K, ioengine=sync, iodepth=1
> Starting 1 thread
>
> cover: (groupid=0, jobs=1): err= 0: pid=6969
> Description : ["Not covering every block."]
> read : io=264KiB, bw=90112KiB/s, iops=7666, runt= 3msec
> clat (usec): min= 66, max= 438, avg=151.00, stdev=88.90
> cpu : usr=0.00%, sys=133.33%, ctx=0
> IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >=64=0.0%
> issued r/w: total=23/0, short=0/0
> lat (usec): 100=34.78%, 250=56.52%, 500=8.70%
>
>
> Run status group 0 (all jobs):
> READ: io=264KiB, aggrb=90112KiB/s, minb=90112KiB/s, maxb=90112KiB/s,
> mint=3msec, maxt=3msec
>
> Disk stats (read/write):
> sda: ios=49/0, merge=0/0, ticks=4/0, in_queue=4, util=2.86%
> --------------------------------------------
>
> It seems fio doesn't cover every block.And using gdb we can see it in
> detail:
> --------------------------------------------
> (gdb) where
> #0 do_io (td=0xb7bec000) at fio.c:423
> #1 0x0804e1b1 in thread_main (data=0xb7bec000) at fio.c:873
> #2 0xb7f00504 in start_thread ()
> from /lib/tls/i686/cmov/libpthread.so.0
> #3 0xb7e5e51e in clone () from /lib/tls/i686/cmov/libc.so.6
> (gdb) p td->files[0].num_maps
> $1 = 2
> (gdb) x/2w td->files[0].file_map
> 0x8073ff0: 0x00000000 0x00000000
> (gdb) finish
> Run till exit from #0 do_io (td=0xb7bec000) at fio.c:423
> thread_main (data=0xb7bec000) at fio.c:875 0 kb/s] [eta 00m:00s]
> 875 clear_state = 1;
> (gdb) x/2w td->files[0].file_map
> 0x8073ff0: 0xffffeffd 0x7ce2df47
> (gdb) c
> Continuing.
> [Thread -1212236896 (LWP 7099) exited]
>
> cover: (groupid=0, jobs=1): err= 0: pid=7098
> Description : ["Not covering every block."]
> read : io=264KiB, bw=7KiB/s, iops=0, runt= 37522msec
> clat (usec): min= 67, max= 347, avg=143.28, stdev=67.59
> bw (KiB/s) : min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
> cpu : usr=0.00%, sys=0.01%, ctx=13
> IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >=64=0.0%
> issued r/w: total=29/0, short=0/0
> lat (usec): 100=31.03%, 250=62.07%, 500=6.90%
>
>
> Run status group 0 (all jobs):
> READ: io=264KiB, aggrb=7KiB/s, minb=7KiB/s, maxb=7KiB/s,
> mint=37522msec, maxt=37522msec
>
> Disk stats (read/write):
> sda: ios=50/23, merge=0/32, ticks=0/160, in_queue=160, util=0.13%
>
> Program exited normally.
> --------------------------------------------------
>
>
> 2. Reason for problem:
> We think when there is no 'norandommap' in the job file, and no ':int'
> appended with 'rw',just like the job file above, fio is meant to use
> only the blocks not covered yet.
> But it seems in the function mark_random_map(), the condition before
> calling random_map_free() will never be satisfied.
>
> 3. Fix suggestion:
> Here is the patch:
> -----------------------------------------------------
> diff -Nraup fio-7.25/io_u.c fio-7.25-cover/io_u.c
> --- fio-7.25/io_u.c 2007-07-25 14:25:05.000000000 +0800
> +++ fio-7.25-cover/io_u.c 2007-07-25 20:43:58.000000000 +0800
> @@ -55,7 +55,7 @@ static void mark_random_map(struct threa
> * If we have a mixed random workload, we may
> * encounter blocks we already did IO to.
> */
> - if (!td->o.ddir_nr && !random_map_free(td, f, block))
> + if (td->o.ddir_nr==1 && !random_map_free(td, f, block))
> break;
>
> idx = RAND_MAP_IDX(td, f, block);
> ------------------------------------------------------
>
> After that, we check it with gdb:
> ----------------------------------------------------
> (gdb) where
> #0 do_io (td=0xb7c14000) at fio.c:423
> #1 0x0804e1b1 in thread_main (data=0xb7c14000) at fio.c:873
> #2 0xb7f28504 in start_thread ()
> from /lib/tls/i686/cmov/libpthread.so.0
> #3 0xb7e8651e in clone () from /lib/tls/i686/cmov/libc.so.6
> (gdb) p td->files[0].num_maps
> $2 = 2
> (gdb) x/2w td->files[0].file_map
> 0x8073ff0: 0x00000000 0x00000000
> (gdb) finish
> Run till exit from #0 do_io (td=0xb7c48000) at fio.c:423
> thread_main (data=0xb7c48000) at fio.c:875 0 kb/s] [eta 00m:00s]
> 875 clear_state = 1;
> (gdb) x/2w td->files[0].file_map
> 0x8073ff0: 0xffffffff 0xffffffff
> (gdb) c
> Continuing.
> [Thread -1212073056 (LWP 7443) exited] 5/ 0 kb/s] [eta 00m:00s]
>
> cover: (groupid=0, jobs=1): err= 0: pid=7442
> Description : ["Not covering every block."]
> read : io=256KiB, bw=5KiB/s, iops=0, runt= 46612msec
> clat (usec): min= 62, max= 522, avg=119.50, stdev=88.07
> bw (KiB/s) : min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
> cpu : usr=0.00%, sys=0.01%, ctx=11
> IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >=64=0.0%
> issued r/w: total=38/0, short=0/0
> lat (usec): 100=55.26%, 250=39.47%, 500=2.63%, 750=2.63%
>
>
> Run status group 0 (all jobs):
> READ: io=256KiB, aggrb=5KiB/s, minb=5KiB/s, maxb=5KiB/s,
> mint=46612msec, maxt=46612msec
>
> Disk stats (read/write):
> sda: ios=64/45, merge=0/62, ticks=8/476, in_queue=484, util=0.19%
>
> Program exited normally.
> (gdb)
> ---------------------------------------------------------
> Now we can see the expected '0xffffffff 0xffffffff' and the 'io=256KiB'.
> By the way, we tested this both on a 32bit machine and a 64bit one.

Again, excellent bug report! I've applied your fix.

-- 
Jens Axboe