Re: [BUG] fill_random_bytes broken

On 7/27/07, Jens Axboe <jens.axboe_at_oracle.com> wrote:
>
> On Fri, Jul 27 2007, Shawn Lewis wrote:
> > On 7/27/07, Jens Axboe <jens.axboe_at_oracle.com> wrote:
> > >
> > > On Thu, Jul 26 2007, Shawn Lewis wrote:
> > > > On 7/26/07, Jens Axboe <jens.axboe_at_oracle.com> wrote:
> > > >>
> > > >> On Thu, Jul 26 2007, Jens Axboe wrote:
> > > >> > On Wed, Jul 25 2007, Shawn Lewis wrote:
> > > >> > > verify.c: fill_random_bytes() doesn't work as intended.
> > > >> > >
> > > >> > > drand48 on linux returns a double whose value is evenly
> distributed
> > > >> between
> > > >> > > 0.0 and 0.1. This is not the same as having an 8-byte value
> where
> > > each
> > > >> bit
> > > >> > > has the same chance of being 0 as it does of being 1.
> > > >> > >
> > > >> > > I'd fix this but it should probably be done in a
> cross-platformish
> > > way
> > > >> and
> > > >> > > I'm not quite sure how to do it. Maybe just use smaller
> randoms?
> > > Have
> > > >> an
> > > >> > > os_random_uint32?
> > > >> >
> > > >> > Hmm indeed, that looks problematic. If you have a suggested fix,
> fire
> > > >> > away :-)
> > > >>
> > > >> OK, I just got rid of os_random_double() and used os_random_long()
> for
> > > >> the verify stuff. It looks like it's working fine, can you confirm?
> > > >
> > > >
> > > > Hmm, almost ok. But os/os-linux.h is broken also. lrand48 returns a
> > > > non-negative long, so the high bit is never touched. How about using
> > > > mrand48, patch upcoming :)
> > >
> > > Irk, the pain...
> > >
> > > Regarding the *rand48() usage for verify buffers, I gather you had
> > > pattern fill plans in that area. I think we should default to using a
> > > range of patterns, the current fully random buffers are really slow to
> > > generate. If I test it here on my notebook using a null io engine, it
> > > can fill and sum only about ~60MiB/sec. Replacing it with a memset()
> > > with 0x5a, I get ~220MiB/sec throughput.
> > >
> > > It'd probably be better to do repeated loops using fixed patterns,
> also
> > > allowing you to detect certain types of corruption. I'm much looking
> > > forward to your patches in this area! Being able to write blocks of eg
> > > 64k but have sums at 512b granularity would be awesome as well. I'm
> > > tempted to just add that, but the wise thing todo is probably to just
> > > wait for you to post your stuff. No point in reinventing the same
> code.
> > > IOW, please post it :-)
> >
> >
> > Ok, I was going to wait until I had a patchset that has all the features
> we
> > want because I might realize I need to go back and change some stuff.
> But
> > I'll send my header_interval patch to you today as soon as I'm done
> > resolving all these conflicts :). And I'll add some example jobs once
> you
> > have the full patch set.
>
> Please, just do smaller patches instead of a long series that you
> "saved" up to, it's much easier for me! Changing code later or fixing
> bugs in that area is not a problem, merging/updating is a zinch. It'll
> save you merging conflicts as well, if/when I go around changing some of
> this code.

Ya I was kind of considering my branch an experimental branch because I
wasn't quite sure how I wanted to do things, but it's separated into a bunch
of smaller patches. But I will try to submit them incrementally instead of
stacking them all up if that helps :).

My plan is to merge your stuff up, get some more testing in, and then
> release a 1.17 with everything.

Sounds good, hopefully you will have all of my patches by end of day Monday
(I'm in PST).

--
> Jens Axboe
>
>