Re: Proposed fio changes.

On Fri, Jul 20 2007, Shawn Lewis wrote:
> Hi,
>
> We'd like to add the following features to fio. We have most of these
> already implemented under fio 1.6, which I plan to port forward to head but
> I'd like to get input on what we can do to make these changes acceptable
> for
> the main fio branch.

Your additions all sound very reasonable and useful, so I suggest you
just send a patch against the current -git tree and I'll review and
integrate them. Thanks!

>
> 1) The ability to verify without using a history log. The user can specify
> a
> read phase following a write phase, making sure that the two phases will
> write and read the same blocks. The read phase will verify each block.
>
> 2) The ability to fill buffers with a pattern instead of random data.
>
> 3) The ability to write a signature with some extra verification
> information
> to the header of a buffer. This information is: the block number, the fill
> pattern used above, a write timestamp, the td->thread_number, and the
> td->request_number. This is implemented as another "verify" option.
>
> 4) The ability to store the buffer header at a user specified offset within
> the buffer.
>
> 5) The ability to store the buffer header at a finer granularity then the
> buffer size. For example you could compute crc for 512 byte chunks even if
> bs=4096.
>
>
> Our proposed modifications to the job file parameters are as follows. I'm
> not sure if changing option names is acceptable. Any suggestions are
> appreciated.
>
> "verify" will have a new option: "sig" which writes the data listed in 3
> above instead of a checksum. This field could be renamed to "block_header"
>
> A new option "do_verify" will be added. If it is included in a write phase
> then the io_hist_log will be updated and do_verify() will be called after
> all the writes. If it is included in a read phase verify_io_u() will be
> called for every block read. We'll have to calculate the checksum for the
> data read in and compare it to the data in the header instead of comparing
> it to the logged checksum in this case.
>
> io_hist_list will not be used unless "do_verify" is on during a write
> phase.
> "do_verify" will default to on for write phases that specify "verify" to
> preserve old behavior.
>
> A new option "fill_type" will be added. It can be "random" or
> "byte_pattern". If fill is "byte_pattern" and "do_verify" is on we'll
> verify
> that we read the correct bytes back. The field "zero_buffers" could be
> replaced with this one.
>
> A new option "pattern" will be added. It will specify the byte_pattern to
> be
> used.
>
> A new option "verify_hdr_offset" will be added. It will specify the offset
> for the header within a block. If "verify" above is renamed this could be
> called "header_offset"
>
> A new option "verify_size" will be added. It defaults to block size. This
> could be called "header_period" (there's probably a better word).
>
>
> Shawn Lewis
> Google

-- 
Jens Axboe