[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Asheesh Laroia <asheesh at asheesh.org> [2010.01.25.1819 +1300]:
> You say "Ouch" but you should know Dovecot *already* does this. I
> don't mind interoperating with that.
>
> See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues
> with the specification", subsection "Locking". I term this theQ
> famous readdir() race.

Yikes. IMAP (including dovecot) just SUCKS.

> Without this lock, Maildir is fundamentally incompatible with IMAP
> -- one Maildir-using process modifying message flags could make
> a different Maildir-using process think said message is actually
> deleted. In the case of temporary disappearing mails in Mutt
> locally, that's not the end of the world. For IMAP, it will make
> the IMAP daemon (one of the Maildir-using processes) send a note
> to IMAP clients saying that the message has been deleted and
> expunged.
[…]
> Just don't fall into the trap of thinking Maildir is compatible
> with IMAP. It's not, because as I understand things, the
> filesystem doesn't guarantee that you can actually iterate across
> a directory's files if another process is modifying the list of
> files.

This is all perfect reason to concentrate even more on designing
a store that could potentially make IMAP obsolete once and for all!

The current idea is to sync Git downstream only, and find a way to
keep multiple copies of a tagstore in sync, by way of the "server
instance" (where mail is received/delivered). Deleting messages
would then be something like setting the notmuch::deleted tag, which
clients would honour; on the server, a cleanup process would run
regularly to actually delete the blobs associated with deleted
messages. This would then propogate the next time one pulls from
Git.

Whether to store history (commit objects) or just collections (tree
objects) needs to be investigated.

> >But there are still good reasons why you'd want to have IMAP
> >capability too, e.g. Webmail. Given the atomicity problems that
> >come from Git, maybe an IMAP server reading from the Git store
> >would make sense.
> 
> It wouldn't be too hard to write a FUSE filesystem that presented
> an interface to a Git repository that didn't allow the contents of
> files to be modified. Then Dovecot could think it's interacting
> with the filesystem.

Yes, a FUSE layer (which adds a daemon), or a lightweight access
API via libnotmuch. Probably the former using the latter. ;)

> Aww, I like Maildir flags, but if there's a sync tool, I'm fine
> with that.
[…]
> I'm not sure, but maybe it's safe if you refuse to ever modify
> a message's flags in the filename.

The main point is that there is nothing really in Maildir filenames
that you couldn't equally (and possibly better) represent in the
notmuch::* tag namespace, and then there is benefit in only having
one used primarily (which means notmuchsync can do whatever it
wants without affecting or messing with notmuch).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"if I can't dance, i don't want to be part of your revolution."
                                                        - emma goldman
 
spamtraps: madduck.bogus at madduck.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/0c16410d/attachment.pgp>