[notmuch] Rename support and database upgrade

[Carl Worth]

Hi notmuch users,

I know that things have been fairly silent from me on both the mailing
list and the git repository lately. Hopefully that will change now.

Just yesterday I merged in my "rename-support" branch to master (it's
about 60 commits and took me two or three weeks for what I originally
thought would be two or three hours of work).

The big feature here is that notmuch now deals sanely with files being
deleted or renamed within the mail store. This is a feature we've talked
about for some time that will make it feasible to use notmuch while also
using various other email software on the same mail store.

To support this feature, I did change some details about how we store
data in the Xapian database. I wanted to avoid a flag day where all
users would have to dump, rebuild, and restore their database, (with
risk of making a mistake and losing data). So instead, I added
versioning to the database and implemented an automatic upgrade of the
database content. I've tested this upgrade support enough to be have
quite a bit of confidence in it.

So after you upgrade your notmuch, the next run of "notmuch new" will
detect that the database needs to be upgraded and will do that
automatically. The upgrade process does take quite a bit of time, but is
about 4 times faster than building the same database from scratch (even
ignoring the time it would take to restore all tags.

Oh, and that "4 times faster" result is with an upgraded Xapian as well,
(with the recent fix for defect #250). I'll say more about Xapian in a
separate post. But even if you have an older Xapian, the upgrade should
be no slower than a rebuild, (and still avoids the step to restore all
tags which is *really* slow with old Xapian).

So I definitely recommend everyone try the upgrade path. Of course, it's
always possible that something will go very wrong with the process. So
it's still a good idea to do a "notmuch dump" to save all your tags
before beginning. (It's really a good idea to do a "notmuch tag" on a
regular basis anyway---such as just before nightly backups or similar.)

Anyway, I just wanted to let people know what to expect so they wouldn't
be surprised by the "please wait while upgrading" message from "notmuch
new". And hopefully I'll now be able to start working through the
backlog of mail and patches I've got. I've skimmed things and it looks
like there's some really interesting stuff in there. So thanks!

-Carl
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100110/068047b9/attachment.pgp>