Notmuch success: Xapian database corrupt

[John Fremlin]

After the encouraging message from Sebastian. I deleted the
.notmuch/xapian dir and started again.

It went off a good rate (300+ files/sec) and here was the final score

Processed 494764 total files in 2h 54m 41s (47 files/sec.).                 
Added 226817 new messages to the database.

This is much faster than before. As I haven't changed the storage or the
filesystem (ext4,data=ordered over encrypted aes-xts-plain), I just
don't know what made the difference. My kernel is now 2.6.32-21-generic
#32-Ubuntu and I had an older one the first try a month or so ago.

Carl Worth <cworth at cworth.org> writes:
[...]
> Welcome to notmuch, and I'm so sorry to hear that your initial attempt
> to use it was so frustrating.

Thanks for the welcome! I was initially impressed by it but rather
worried about relying on it after the database corruption.

> I'm not aware of any bugs in notmuch that can result in a corrupt Xapian
> database. In fact, this can't be a bug in notmuch alone (since Xapian is
> detecting the corruption). There must at least be a bug in Xapian or
> else some lower-level failure is occurring (disk full?) that Xapian
> can't deal with.

Disk full is quite likely. I'll try to avoid that in future.

[...]
> So there's some performance problem that you're having in addition to
> the database corruption. Hopefully we can figure that out. What kernel
> and filesystem are you using? Are you using an encrypted partition?

Happy to say (though frustrating for you), this time it's much
faster. Maybe because I had more disk free this time round so the Xapian
database became less fragmented? (Speculation, no evidence.)