Re: fi massive testing

On Wed, 2008-03-05 at 09:30 +0100, Jens Axboe wrote:
> On Wed, Mar 05 2008, Joshua Root wrote:
> > Zhang, Yanmin wrote:
> > >Do you have any recommendations about the job file which could simulate
> > >application
> > >models?
> >
> > If you're interested in simulating the workload as seen at the block
> > layer, I have a script that will generate an fio job that approximates
> > the workload seen in a blktrace. There's a download link here:
> > <http://www.gelato.unsw.edu.au/IA64wiki/JoshuaRoot/MarkovChains>
> >
> > As that page indicates, the primary purpose of the package is a little
> > different, and I just added the fio job creation recently. You first
> > need to run prep.sh on your blktrace files as described in the first
> > paragraph of the README. But then, instead of buildChain.py, you want to
> > run traceStats.py like so (which requires SciPy to be installed):
> >
> > ./traceStats.py -i OPFILE -f > output.job
> >
> > You'll probably want to change a few of the values in the global section.
> >
> > Most of the aspects of the workload are reproduced pretty well. Some
> > parts are still a little rough though; like the queue depth and
> > thinktime (they end up really huge sometimes, so I artificially cap
> > them). Sync vs async also doesn't work (everything ends up as async).
> > Same with priorities.
>
> That's pretty cool! I've long played with the idea of enhancing fio a
> bit, so that you could more closely model anything. Right now the io_u
> tells you where to do IO and and what offset. It was also extended to
> pass information on doing an fsync. My plan would be to extend it even
> more, so you could describe things like file open etc with the io_u.
> Once that is done, you could inject a pending list of things to do into
> fio, or even generate it on-the-fly by parsing a workload description
> language.
>
> Of course, you can also replay blktrace traces directly with fio if you
> want, that works ok right now.
Thank your guys for the good suggestion!

-yanmin
Received on Wed Mar 05 2008 - 09:42:51 CET