[notmuch] Notmuch performance (literally, in my case)

[Aneesh Kumar K. V]

On Tue, 16 Mar 2010 08:37:54 -0700 (PDT), Ben Gamari <bgamari.foss at gmail.com> wrote:
> On Tue, 16 Mar 2010 11:08:47 +0000, Olly Betts <olly at survex.com> wrote:
> > For the issue of a background task interfering with interactive use, the feel
> > arguably matters more than the throughput.
> > 
> > I'll probably put that patch in 1.0.19, and look at moving all the fdatasync()
> > calls together.  This is http://trac.xapian.org/ticket/426 BTW.
> > 
> > The kernel should be able to handle this workload better though, so I would
> > say it was worthwhile to bring up on LKML if you have the energy.  It certainly
> > isn't just you, as apt-xapian-index seems to trigger it for some Ubuntu users,
> > and madduck mentioned it on #notmuch a week or so ago.
> 
> Alright. This issue has been bothering me for a very long time and it's frankly
> pretty pathetic how badly the kernel falls apart under this sort of workload.
> I just wrote up a message (4b9fa440.12135e0a.7fc8.ffffe745 at mx.google.com), so
> we'll see what happens. In the past kernel developers have been very eager to
> write this issue off as not reproducible enough (perhaps wisely), so if anyone
> has anything to say, please contribute it to the thread.
> 

Ext3 fsync related issue is a know problem due to the way journalling is
handled in ext3. The solution for that would be data=writeback ( with
its loss of data integrity ) or not yet upstreamed data=guarded. Another
option would be to try ext4 which should not be impacted that badly by
the data=ordered journalled mode

-aneesh