Notmuch DB Problems

[Olly Betts]

On Mon, Sep 10, 2018 at 08:01:06AM -0300, David Bremner wrote:
> Mueen Nawaz <mueen at nawaz.org> writes:
> > Now killing all those jobs did not fix the database. It was still
> > broken. And as we saw the second time round, it was /really/ broken - it
> > would not even open in read-only mode.
> 
> That seems like something the Xapian devs (in copy) might be interested
> in fixing, if you could come up with a simple reproducer.

I'm certainly happy to investigate if someone can provide a way for
me to make it happen on demand.

It doesn't make much sense to me that holding the lock alone could be
causing any sort of corruption - that's just an fcntl() lock.

I would suggest to make sure you're running Xapian 1.4.7 as that fixed a
cursor handling bug which affected notmuch.  I didn't find a way to make
it corrupt on-disk data, but it's hard to be completely certain that it
couldn't ever do that, so ruling out that as a cause would be good.

> notmuch could be cleverer about timing out on trying to acquire a
> lock. I suspect it's a bit delicate to get that right, and I've been
> hoping the underlying primitives would get a bit more flexible
> w.r.t. locking.

You mean in Xapian?  If so, a wishlist bug saying what you're hoping
for might help it happen.

Cheers,
    Olly