notmuch python bindings corrupt db index (was: gmail importer script)

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-11 13:47:32)
> On Tue, Dec 11, 2012 at 11:04 AM, Patrick Totzke
> <patricktotzke at gmail.com> wrote:
> > I ended up killing the process after i saw that ETA was >3h.
> 
> Gmail's bottleneck. Very unfortunate. Leave it going over night.

OK, i assumed i'd not have to download all messages because i already have them
on disk -- synced via offlineimap. but then i remembered that OI stores uses
its own naming scheme for the files in a maildir. so thats probably why your script
did not realize theyre already there. (maybe it should? it could look up the MIDs from notmuch)

> > This broke my index!
> >
> > afterwards, a `notmuch new` printed
> >
> > A Xapian exception occurred creating a directory: Expected block
> > 485 to be level 1, not 0.
> >
> > I wasnt able to properly read the index anymore and had to
> > restore an old index dump. not cool!
> 
> 
> Can you provide any more information about this? The script properly
> uses atomic transactions. Any corruption here is the result of a
> notmuch library bug, which should be investigated.

i am kind of reluctant to reproduce this again (on my work-desktop).
All i did was start your script, writing to the index in another terminal in between (using alot)
possibly my cronjob fired a "notmuch new; afew.." in between also.
when I killed your script the index war broken.

Can i turn on some global logging for notmuch to debug this?
Maybe libnotmuch had trouble with so many db-writes at a time?

/p


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: signature
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20121211/254fef11/attachment.pgp>