Database corruption after clean rebuild

[Javier Garcia]

Unfortunately I can't share my emails without the approval of other
parties. The minimum subsets that trigger the error are in the range of
1000-5000 mails, so asking each and everyone of them is out of my reach.
I tried to replicate the problem using just spam folders without success.

The following is a solid workaround I've stumbled upon. Afew no longer
complains and database corruption is gone.

$ notmuch compact
$ xapian-check ~/.mail/.notmuch/xapian
   <check messages>
   No errors found

I built xapian-core 1.50 but I can't compile notmuch 0.26.1 against it.
I will wait and test again in a few weeks.

If you are interested in my setup, the error happens with this minimal
configuration.

#~/.config/afew/config
[Filter.1]
query = 'folder:"//(INBOX|Inbox|inbox)$/" AND (NOT tag:inbox)'
tags = +inbox;-new
message = Messages in INBOX folder are tagged as inbox

[Filter.2]
query = '(NOT folder:"//(INBOX|Inbox|inbox)$/") AND (tag:inbox)'
tags = -inbox
message = Messages not in INBOX folder cannot be inbox

#~/.notmuch-config
[database]
path=/home-path/.mail
[new]
tags=new

On 07/04/18 12:51, David Bremner wrote:
> Javier Garcia <javiertury at gmail.com> writes:
>
>> I've applied the path to notmuch 0.26.1 without success.
>>
>> $ rm -rf ~/.mail/.notmuch
>> $ LD_LIBRARY_PATH=/hidden-path/notmuch-0.26.1/lib/:$LD_LIBRARY_PATH
>> ./notmuch new
>> Â Â  Found 20065 total files (that's not much mail).
>> Â Â  Processed 20065 total files in 58s (341 files/sec.).
>> Â Â  Added 19605 new messages to the database.
>>
>> $ xapian-check .mail/.notmuch/xapian/
>> Â Â  docdata:
>> Â Â  blocksize=8K items=63 firstunused=1 revision=2 levels=0 root=0
>> Â Â  B-tree checked okay
>> Â Â  docdata table structure checked OK
>> Â Â  termlist:
>> Â Â  blocksize=8K items=43520 firstunused=8293 revision=2 levels=2 root=748
>> Â Â  xapian-check: DatabaseError: 1 unused block(s) missing from the free
>> list, first is 0
> OK, so probably not related to reference loops (although that patch is
> not very well tested).  It's not clear how notmuch is triggering it, but
> this looks like the same bug in Xapian that olly fixed recently [1].
>
> A possible next step is to try building xapian master, and linking
> notmuch against that.
>
> Maybe Patrick or Justus (in copy) has some idea why you're only seeing
> problems in afew.
>
> Another debugging direction is to try to duplicate your problem with
> some subset of mail that you're willing to share (bisection is the usual
> strategy).
>
> [1] https://notmuchmail.org/pipermail/notmuch/2018/026369.html