[notmuch] Notmuch performance (literally, in my case)

[Olly Betts]

On Mon, Mar 15, 2010 at 10:29:36AM -0700, Ben Gamari wrote:
> On Mon, 15 Mar 2010 09:29:35 +0000 (UTC), Olly Betts <olly at survex.com> wrote:
> > http://oligarchy.co.uk/xapian/patches/xapian-1.0.18-flint-group-fsyncs.patch
> > 
> > What this does it to at least pair up the calls to fdatasync().  It's
> > possible to move them all together, but requires more effort, so it'd be
> > nice to know if this is actually going to help.
> 
> This does seem to help. Of course, latency is a difficult thing to measure,
> but notmuch does _feel_ faster. That being said, iostat still only shows
> 700kByte/second read and 300kByte/second write, so things haven't changed in
> the throughput side of things.

For the issue of a background task interfering with interactive use, the feel
arguably matters more than the throughput.

I'll probably put that patch in 1.0.19, and look at moving all the fdatasync()
calls together.  This is http://trac.xapian.org/ticket/426 BTW.

The kernel should be able to handle this workload better though, so I would
say it was worthwhile to bring up on LKML if you have the energy.  It certainly
isn't just you, as apt-xapian-index seems to trigger it for some Ubuntu users,
and madduck mentioned it on #notmuch a week or so ago.

Cheers,
    Olly