On Fri, Jul 27 2007, ljzhang,Yaxin Hu,Jianchao Tang wrote:
> 1. The job file we prepared:
> ----------offset-----------------
> [global]
> filename=./temp/HOWTO
> nrfiles=1
> rw=read
> size=8K
> offset=16k
> bs=1k
> thread
> loops=10
> [offset]
> description="Option 'offset' doesn't work."
> ---------------------------------
> Besides, we copied the file HOWTO to the directory ./temp/.
> So we thought fio would read 8K text sequentially from the point 16K off
> the beginning of HOWTO.
>
> we run fio with gdb to see what was read out after the first io, and we
> got this:
> ---------------------------------
> ... ...
> (gdb) bt
> #0 0x0804cb4d in do_io (td=0xb7ca8000) at fio.c:449
> #1 0x0804e1b1 in thread_main (data=0xb7ca8000) at fio.c:873
> #2 0xb7fbc504 in start_thread ()
> from /lib/tls/i686/cmov/libpthread.so.0
> #3 0xb7f1a51e in clone () from /lib/tls/i686/cmov/libc.so.6
> (gdb) x/4w io_u->xfer_buf
> 0x8073f08: 0x6c626154 0x666f2065 0x6e6f6320
> 0x746e6574
> (gdb)
> ---------------------------------
> The above was performed after the first io finished by td_io_queue.
> The contents read out as we can see was the ASCII codes for 'Table of
> content', which was right from the beginning of file HOWTO.
> That means fio read from the beginning but not the point 16K off the
> beginning.Is that right?
>
> 2. Reason for the problem:
> It seems the offset is saved in td->o.start_offset. But it isn't passed
> to f->file_offset.And the setting up of f->io_size doesn't refer to
> f->file_offset.
>
> 3. Fix suggestion:
> I tried this:
> ---------------------------------------------------------------
> diff -Nraup fio-7.25/filesetup.c fio-7.25-offset/filesetup.c
> --- fio-7.25/filesetup.c 2007-07-25 14:25:05.000000000 +0800
> +++ fio-7.25-offset/filesetup.c 2007-07-27 19:10:29.000000000 +0800
> @@ -378,6 +378,8 @@ int setup_files(struct thread_data *td)
> extend_size = total_size = 0;
> need_extend = 0;
> for_each_file(td, f, i) {
> + f->file_offset = td->o.start_offset;
> +
> if (!td->o.file_size_low) {
> /*
> * no file size range given, file size is equal to
> @@ -385,20 +387,27 @@ int setup_files(struct thread_data *td)
> * zero, set it to the real file size.
> */
> f->io_size = td->o.size / td->o.nr_files;
> - if (!f->io_size)
> - f->io_size = f->real_file_size;
> + if (!f->io_size) {
> + if (f->file_offset > f->real_file_size)
> + goto err_offset;
> + f->io_size = f->real_file_size - f->file_offset;
> + }
> } else if (f->real_file_size < td->o.file_size_low ||
> f->real_file_size > td->o.file_size_high) {
> + if (f->file_offset > td->o.file_size_low)
> + goto err_offset;
> /*
> * file size given. if it's fixed, use that. if it's a
> * range, generate a random size in-between.
> */
> if (td->o.file_size_low == td->o.file_size_high)
> - f->io_size = td->o.file_size_low;
> + f->io_size = td->o.file_size_low - f->file_offset;
> else
> - f->io_size = get_rand_file_size(td);
> - } else
> - f->io_size = f->real_file_size;
> + f->io_size = get_rand_file_size(td) - f->file_offset;
> + } else if (f->file_offset > f->real_file_size)
> + goto err_offset;
> + else
> + f->io_size = f->real_file_size - f->file_offset;
>
> if (f->io_size == -1ULL)
> total_size = -1ULL;
> @@ -406,12 +415,12 @@ int setup_files(struct thread_data *td)
> total_size += f->io_size;
>
> if (f->filetype == FIO_TYPE_FILE &&
> - f->io_size > f->real_file_size &&
> + (f->io_size + f->file_offset) > f->real_file_size &&
> !(td->io_ops->flags & FIO_DISKLESSIO)) {
> need_extend++;
> - extend_size += f->io_size;
> + extend_size += (f->io_size + f->file_offset);
> f->flags |= FIO_FILE_EXTEND;
> - }
> + }
> }
>
> if (!td->o.size)
> @@ -431,7 +440,7 @@ int setup_files(struct thread_data *td)
>
> assert(f->filetype == FIO_TYPE_FILE);
> f->flags &= ~FIO_FILE_EXTEND;
> - f->real_file_size = f->io_size;
> + f->real_file_size = (f->io_size + f->file_offset);
> err = extend_file(td, f);
> if (err)
> break;
> @@ -447,6 +456,9 @@ int setup_files(struct thread_data *td)
>
> td->total_io_size = td->o.size * td->o.loops;
> return 0;
> +err_offset:
> + log_err("%s: you need to specify valid offset=\n", td->o.name);
> + return 1;
> }
>
> int init_random_map(struct thread_data *td)
> diff -Nraup fio-7.25/ioengines.c fio-7.25-offset/ioengines.c
> --- fio-7.25/ioengines.c 2007-07-25 14:25:05.000000000 +0800
> +++ fio-7.25-offset/ioengines.c 2007-07-27 17:14:05.000000000 +0800
> @@ -279,7 +279,7 @@ int td_io_open_file(struct thread_data *
>
> f->last_free_lookup = 0;
> f->last_completed_pos = 0;
> - f->last_pos = 0;
> + f->last_pos = f->file_offset;
> f->flags |= FIO_FILE_OPEN;
> f->flags &= ~FIO_FILE_CLOSING;
>
> diff -Nraup fio-7.25/io_u.c fio-7.25-offset/io_u.c
> --- fio-7.25/io_u.c 2007-07-25 14:25:05.000000000 +0800
> +++ fio-7.25-offset/io_u.c 2007-07-27 16:56:35.000000000 +0800
> @@ -165,7 +165,7 @@ static int get_next_offset(struct thread
> if (f->last_pos >= f->real_file_size)
> return 1;
>
> - b = f->last_pos / td->o.min_bs[ddir];
> + b = (f->last_pos - f->file_offset) / td->o.min_bs[ddir];
> }
>
> io_u->offset = (b * td->o.min_bs[ddir]) + f->file_offset;
>
> ---------------------------------------------------------
>
> And now we can check it with gdb:
> --------------------------------------------------------
> ... ...
> (gdb) bt
> #0 0x0804cb4d in do_io (td=0xb7c50000) at fio.c:449
> #1 0x0804e1b1 in thread_main (data=0xb7c50000) at fio.c:873
> #2 0xb7f64504 in start_thread ()
> from /lib/tls/i686/cmov/libpthread.so.0
> #3 0xb7ec251e in clone () from /lib/tls/i686/cmov/libc.so.6
> (gdb) x/4w io_u->xfer_buf
> 0x8073f08: 0x206b636f 0x7420666f 0x66206568
> 0x20656c69
> (gdb)
> ---------------------------------------------------------
> After the first io, I got '0x206b636f 0x7420666f 0x66206568 0x20656c69'
> read out.And they represent the string 'ock of the file '.Then i search
> the string in HOWTO and got only one line containing the string:
> 'norandommap Normally fio will cover every block of the file when doing'
> I removed the text following that line in HOWTO, and then I found that
> the size of the rest text in HOWTO was 16K, which equaled to the offset
> we set in job file.
> So I think the option 'offset' does work after the changes.
>
> In fact, I am not sure of the fix, but i do hope it can help.
It's tricky. I think the fix looks correct, but I'd be a lot more
comfortable if you could reassure me that it didn't break any other
tests you have? I'll merge it, but please do run through your normal io
tests and verify that they seem sane and work as expected. Thanks!
-- Jens AxboeReceived on Fri Jul 27 2007 - 13:27:29 CEST
This archive was generated by hypermail 2.2.0 : Fri Jul 27 2007 - 13:30:02 CEST