block covering in fio-1.16.9

From: ljzhang,Yaxin Hu,Jianchao Tang <nonggia_at_sjtu.edu.cn>
Date: Thu, 26 Jul 2007 13:11:16 +0800

1. The job file causing the problem:
----------cover-------------------
[global]
directory=./temp
nrfiles=1
rw=randread
size=256K
bsrange=4k-16k
thread
[cover]
description="Not covering every block."
-----------------------------------------

According to HOWTO,the job file will makes fio to cover every block of
the 256k file.But running that, we sometimes got a io gross more than
256k:
----------------------------------------
nonggia_at_nonggia-desktop:~$ fio --version
fio 1.16.9
nonggia_at_nonggia-desktop:~/fio$ fio cover
cover: (g=0): rw=randread, bs=4K-16K/4K-4K, ioengine=sync, iodepth=1
Starting 1 thread

cover: (groupid=0, jobs=1): err= 0: pid=6969
  Description : ["Not covering every block."]
  read : io=264KiB, bw=90112KiB/s, iops=7666, runt= 3msec
    clat (usec): min= 66, max= 438, avg=151.00, stdev=88.90
  cpu : usr=0.00%, sys=133.33%, ctx=0
  IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
     issued r/w: total=23/0, short=0/0
     lat (usec): 100=34.78%, 250=56.52%, 500=8.70%

Run status group 0 (all jobs):
   READ: io=264KiB, aggrb=90112KiB/s, minb=90112KiB/s, maxb=90112KiB/s,
mint=3msec, maxt=3msec

Disk stats (read/write):
  sda: ios=49/0, merge=0/0, ticks=4/0, in_queue=4, util=2.86%
--------------------------------------------

It seems fio doesn't cover every block.And using gdb we can see it in
detail:
--------------------------------------------
(gdb) where
#0 do_io (td=0xb7bec000) at fio.c:423
#1 0x0804e1b1 in thread_main (data=0xb7bec000) at fio.c:873
#2 0xb7f00504 in start_thread ()
from /lib/tls/i686/cmov/libpthread.so.0
#3 0xb7e5e51e in clone () from /lib/tls/i686/cmov/libc.so.6
(gdb) p td->files[0].num_maps
$1 = 2
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0x00000000 0x00000000
(gdb) finish
Run till exit from #0 do_io (td=0xb7bec000) at fio.c:423
thread_main (data=0xb7bec000) at fio.c:875 0 kb/s] [eta 00m:00s]
875 clear_state = 1;
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0xffffeffd 0x7ce2df47
(gdb) c
Continuing.
[Thread -1212236896 (LWP 7099) exited]

cover: (groupid=0, jobs=1): err= 0: pid=7098
  Description : ["Not covering every block."]
  read : io=264KiB, bw=7KiB/s, iops=0, runt= 37522msec
    clat (usec): min= 67, max= 347, avg=143.28, stdev=67.59
    bw (KiB/s) : min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
  cpu : usr=0.00%, sys=0.01%, ctx=13
  IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
     issued r/w: total=29/0, short=0/0
     lat (usec): 100=31.03%, 250=62.07%, 500=6.90%

Run status group 0 (all jobs):
   READ: io=264KiB, aggrb=7KiB/s, minb=7KiB/s, maxb=7KiB/s,
mint=37522msec, maxt=37522msec

Disk stats (read/write):
  sda: ios=50/23, merge=0/32, ticks=0/160, in_queue=160, util=0.13%

Program exited normally.
--------------------------------------------------

2. Reason for problem:
We think when there is no 'norandommap' in the job file, and no ':int'
appended with 'rw',just like the job file above, fio is meant to use
only the blocks not covered yet.
But it seems in the function mark_random_map(), the condition before
calling random_map_free() will never be satisfied.
 
3. Fix suggestion:
Here is the patch:
-----------------------------------------------------
diff -Nraup fio-7.25/io_u.c fio-7.25-cover/io_u.c
--- fio-7.25/io_u.c 2007-07-25 14:25:05.000000000 +0800
+++ fio-7.25-cover/io_u.c 2007-07-25 20:43:58.000000000 +0800
@@ -55,7 +55,7 @@ static void mark_random_map(struct threa
                 * If we have a mixed random workload, we may
                 * encounter blocks we already did IO to.
                 */
- if (!td->o.ddir_nr && !random_map_free(td, f, block))
+ if (td->o.ddir_nr==1 && !random_map_free(td, f, block))
                        break;
 
                idx = RAND_MAP_IDX(td, f, block);
------------------------------------------------------

After that, we check it with gdb:
----------------------------------------------------
(gdb) where
#0 do_io (td=0xb7c14000) at fio.c:423
#1 0x0804e1b1 in thread_main (data=0xb7c14000) at fio.c:873
#2 0xb7f28504 in start_thread ()
from /lib/tls/i686/cmov/libpthread.so.0
#3 0xb7e8651e in clone () from /lib/tls/i686/cmov/libc.so.6
(gdb) p td->files[0].num_maps
$2 = 2
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0x00000000 0x00000000
(gdb) finish
Run till exit from #0 do_io (td=0xb7c48000) at fio.c:423
thread_main (data=0xb7c48000) at fio.c:875 0 kb/s] [eta 00m:00s]
875 clear_state = 1;
(gdb) x/2w td->files[0].file_map
0x8073ff0: 0xffffffff 0xffffffff
(gdb) c
Continuing.
[Thread -1212073056 (LWP 7443) exited] 5/ 0 kb/s] [eta 00m:00s]

cover: (groupid=0, jobs=1): err= 0: pid=7442
  Description : ["Not covering every block."]
  read : io=256KiB, bw=5KiB/s, iops=0, runt= 46612msec
    clat (usec): min= 62, max= 522, avg=119.50, stdev=88.07
    bw (KiB/s) : min= 0, max= 0, per=0.00%, avg= 0.00, stdev= 0.00
  cpu : usr=0.00%, sys=0.01%, ctx=11
  IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
     issued r/w: total=38/0, short=0/0
     lat (usec): 100=55.26%, 250=39.47%, 500=2.63%, 750=2.63%

Run status group 0 (all jobs):
   READ: io=256KiB, aggrb=5KiB/s, minb=5KiB/s, maxb=5KiB/s,
mint=46612msec, maxt=46612msec

Disk stats (read/write):
  sda: ios=64/45, merge=0/62, ticks=8/476, in_queue=484, util=0.19%

Program exited normally.
(gdb)
---------------------------------------------------------
Now we can see the expected '0xffffffff 0xffffffff' and the 'io=256KiB'.
By the way, we tested this both on a 32bit machine and a 64bit one.

Received on Thu Jul 26 2007 - 07:11:16 CEST

This archive was generated by hypermail 2.2.0 : Thu Jul 26 2007 - 09:00:02 CEST