[notmuch] Missing messages breaking threads

[James Westby]

Hi,

I like the architecture of notmuch, and have just switched
to using it as my primary client, so thanks.

I however have discovered one issue that is a pain.

I use a bugtracker a lot which has workflows that mean that
you don't always get teh initial messages from a bug.

To put it in more common terms, imagine this:

  * Alice sends a mail to a large group of your friends, but
    not you.
  * Each of these friends replies, and puts you in Cc for the
    reply.

This will mean that you get several messages that all have
References and In-Reply-To set to ids that aren't known to
notmuch.

This means that it doesn't thread them, and so they aren't
grouped in the UI. This becomes painful when you are dealing
with several bugs like this. It's almost like being back in
the days of a message oriented client, and we know that's
not the way to do it.

Therefore I'd like to fix this. The obvious way is to
introduce documents in to the db for each id we see, and
threading should then naturally work better.

The only issue I see with doing this is with mail delays.
Once we do this we will sometimes receive a message that
already has a dummy document. What happens currently with
message-id collisions?

Therefore I would propose this:

  * When doing a thread resolution and we have ids that
    we don't know, add a document to the db that is
    something like

    {id: <message-id>
     thread: <synthesised thread-id>
     dummy: True
     content: "Messages missing"
    }

  * When we get a message-id conflict check for dummy:True
    and replace the document if it is there.

How does this sound?

There could be an issue with synthesising too many threads
and then ending up having to try and put a message in two
threads? I see there is code for merging threads, would that
handle this?

Thanks,

James