Proposed fio changes.

Hi,

We'd like to add the following features to fio. We have most of these
already implemented under fio 1.6, which I plan to port forward to head but
I'd like to get input on what we can do to make these changes acceptable for
the main fio branch.

1) The ability to verify without using a history log. The user can specify a
read phase following a write phase, making sure that the two phases will
write and read the same blocks. The read phase will verify each block.

2) The ability to fill buffers with a pattern instead of random data.

3) The ability to write a signature with some extra verification information
to the header of a buffer. This information is: the block number, the fill
pattern used above, a write timestamp, the td->thread_number, and the
td->request_number. This is implemented as another "verify" option.

4) The ability to store the buffer header at a user specified offset within
the buffer.

5) The ability to store the buffer header at a finer granularity then the
buffer size. For example you could compute crc for 512 byte chunks even if
bs=4096.

Our proposed modifications to the job file parameters are as follows. I'm
not sure if changing option names is acceptable. Any suggestions are
appreciated.

"verify" will have a new option: "sig" which writes the data listed in 3
above instead of a checksum. This field could be renamed to "block_header"

A new option "do_verify" will be added. If it is included in a write phase
then the io_hist_log will be updated and do_verify() will be called after
all the writes. If it is included in a read phase verify_io_u() will be
called for every block read. We'll have to calculate the checksum for the
data read in and compare it to the data in the header instead of comparing
it to the logged checksum in this case.

io_hist_list will not be used unless "do_verify" is on during a write phase.
"do_verify" will default to on for write phases that specify "verify" to
preserve old behavior.

A new option "fill_type" will be added. It can be "random" or
"byte_pattern". If fill is "byte_pattern" and "do_verify" is on we'll verify
that we read the correct bytes back. The field "zero_buffers" could be
replaced with this one.

A new option "pattern" will be added. It will specify the byte_pattern to be
used.

A new option "verify_hdr_offset" will be added. It will specify the offset
for the header within a block. If "verify" above is renamed this could be
called "header_offset"

A new option "verify_size" will be added. It defaults to block size. This
could be called "header_period" (there's probably a better word).

Shawn Lewis
Google
Received on Fri Jul 20 2007 - 22:57:21 CEST