[notmuch] Some thoughts about notmuch sync with other agents

[martin f krafft]

also sprach Paul R <paul.r.ml at gmail.com> [2010.01.28.0316 +1300]:
> As you see, I advocate a NotMuch <-> IMAP synchronisation ASAP :)

Given the limitations of IMAP (non-transactional, non-standard
keywords implementation, …), I think chances of this are rapidly
diminishing, but I'd love to be proven wrong.

> First of all, processing mail with MUA involves some simple logic that
> is shared by most MUA. This is about receiving *new* mails, *reading*
> mail, *replying* to mail and so on... IMHO, this really belongs to the
> mail processing logic and not to the user logic. Hence my first
> request :
> 
>   1: Don't use user tags space to store MUA flags.
> 
>      That means no more "seen", "unread", "replied" as tags. These are
>      MUA processing *flags*, that must belong to an established set.
>      Tags, on the other hand, are user-land information. So no more
>      [seen, replied, grandma, important] tag sets :)

I disagree.

The MUA actually doesn't (shouldn't) care at all about any of these
flags, at least not for core functionality. Sure, a MUA should
probably hide messages tagged 'deleted', and it would be nice if it
could be configured to highlight messages tagged 'important', but
none of the others — "seen", "unread", "replied", … — have any role
in the core functionality. They are purely user-specific.

They are leftovers from days when some MUA designer decided that
having these flags would be a useful way to organise e-mail
handling, but that person probably dealt with a dozen messages
a day, and didn't have half his/her life organised through
electronic mail.

Point being, times and needs have changed. And while we're in the
process of finding the technology that can suit those needs (in the
most generic way), we might just as well (and should) rid ourselves
from these leftovers.

Any solution must be generic enough so that if you rely on the
functionality provided by these tags, you can trivially make it
happen, e.g. with hooks to add/remove flags on certain actions (such
as sending a reply, or reading a message).

Neither IMAP nor Maildir is capable of storing an extensible,
freely-configurable set of tags (keywords). Therefore, we need a new
method anyway.

> Once this is done, notmuch will know, in a robust a predictable
> way, what happened to a mail. Simply put, NotMuch will store and
> expose MUA flags (Passed, Replied, Seen, Trashed, Draft, and
> Flagged [1]). For each <flag>, notmuch should associate
> a <flag>_synced flag. When changing <flag> from notmuch, it should
> set the <flag>_synced bit to 0. These are MUA mail flags.

I think the semantics would be clearer the other way around: setting
*_changed when a flag is changed.

> Additionally, in a third « space », notmuch should store its « new »
> bit, as well as a « missing » bit probably. Again, this is neither MUA
> logic or user logic, so this should not interfer with user
> classification facility provided by tags, nor with MUA flags. It,
> really, is something else, let's name it "status".

Or "lifetime".

> Once this is done, the 'notmuch new' command should find new mails
> and set the 'new' bit for them, and find the missing mails and set
> the 'missing' bit for them. This will allow for robust external
> processing.

Why would I want to keep around a record in the database when the
physical file is no longer present?

> # 1/ Sync from NotMuch to MailDir
> 
>     notmuch list flags:(seen and not seen_synced) 
>       | notmuch-maildir --mark-mail seen
>       | notmuch move --stdin
>       | notmuch set flags:+seen_synced --stdin

Have you seen/look at notmuchsync?

> The output of the first command would be a list of filenames for mails
> 'seen' since last sync. The second one, an external notmuch--maildir
> helper, would propagate this logic to the MailDir store (easy, this is
> simply a rename), and output the list of (old-name ! new-name). Then
> notmuch would use this information, via a generic 'move' switch, to know
> that mail has been moved, and would output the list of the new places.
> Finaly, notmuch would set the seen_synced flag.

What happens if notmuch-move gets killed due to out-of-memory half
way through?

> As we have seen, this would allow most IMAP <-> MailDir <-> NotMuch
> synchronisation, without having to implement any kind of
> MailDir-specific logic inside notmuch.

It would not take care of any tags beyond the very strictly defined
and limited set of tags you listed in your mail. My point is that we
need to solve this problem more generally anyway — why should
a "replied" tag be synchronised, but a "no-need-to-reply" tag not?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
#include <signature.h>
 
spamtraps: madduck.bogus at madduck.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100201/8ffddb4f/attachment.pgp>