Database corruption after clean rebuild

[Javier Garcia]

I've applied the path to notmuch 0.26.1 without success.

$ rm -rf ~/.mail/.notmuch
$ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH
./notmuch new
   Found 20065 total files (that's not much mail).
   Processed 20065 total files in 58s (341 files/sec.).
   Added 19605 new messages to the database.

$ xapian-check .mail/.notmuch/xapian/
   docdata:
   blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0
   B-tree checked okay
   docdata table structure checked OK
   termlist:
   blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748
   xapian-check: DatabaseError: 1 unused block(s) missing from the free
list, first is 0

With or without the patch, the "corrupted" database works fine most of
the time. For instance this works:

$ notmuch tag +new2 -- tag:new

It's just that afew can't work with the db in this state, it complains
saying the database is corrupted.

$ rm -rf ~/.mail/.notmuch
$ notmuch new
$ afew -tn -vv
   <normal operations>
   terminate called after throwing an instance of
'Xapian::DatabaseCorruptError'
   Aborted (core dumped)

Afew doesn't always crash, even though the database is always corrupted.
Afew crashes when it's called just after a fresh notmuch database is
built and randomly thereafter. The error is always the same.

The following one-liner can solve most of afew crashes. It works well
for the cases in which afew is called right after notmuch database
creation. For random crashes is not as effective.

$ notmuch tag +new2 -- tag:new


On 07/04/18 03:49, David Bremner wrote:
> Javier Garcia <javiertury at gmail.com> writes:
>
>> I can't build a healthy database for notmuch. My mail directory has
>> quite a few mails, around 20,000.
>>
>> $ rm -rf ~/.mail/.notmuch
>> $ notmuch new
>> $ xapian-check ~/.mail/.notmuch/xapian/
>>> docdata:
>>> blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0
>>> B-tree checked okay
>>> docdata table structure checked OK
>>>
>>> termlist:
>>> blocksize=8K items=43520 firstunused=8291 revision=2 levels=2 root=748
>>> xapian-check: DatabaseError: 1 unused block(s) missing from the free
>> list, first is 0
>>
> There was recently a similar report that turned out to be related to a
> reference loop in the mail.  Do you actually have any symptoms of
> database corruption other than the message about the free list? if not,
> it might be worth trying the attached patch, which attempts to break
> reference loops.
>