Bug?: notmuch-search-show-thread shows several threads; only one containing matching messages

[Mark Walters]

On Mon, 30 Jan 2012 23:34:16 +0100, Gregor Zattler <telegraph at gmx.net> wrote:
> Hi Mark,
> * Mark Walters <markwalters1009 at gmail.com> [30. Jan. 2012]:
> > On Mon, 30 Jan 2012 20:04:25 +0100, Gregor Zattler <telegraph at gmx.net> wrote:
> >> * Pieter Praet <pieter at praet.org> [30. Jan. 2012]:
> >>> On Mon, 30 Jan 2012 00:42:14 +0100, Gregor Zattler <telegraph at gmx.net> wrote:
> >>>> * Pieter Praet <pieter at praet.org> [26. Jan. 2012]:
> >>>>> Here's another couple of threads squashed into a single one:
> >>>>> - [O] [Use Question] Capture and long lines
> >>>>>   - id:"BANLkTikoF4tXuNLLufRzNSD6k2ZYs7sUcg at mail.gmail.com"
> >>>>> - [O] Worg update
> >>>>>   - id:"m1wrfiz3ch.fsf at tsdye.com"
> >>>>> - [O] Table formula to convert hex to dec
> >>>>>   - id:"20110724080054.GB16388 at x201"
> >>>>> - [O] ICS import?
> >>>>>   - id:"20120125173421.GQ3747 at x201"
> >>>>> 
> >>>>> 
> >>>>> AFAICT, none of them share Message-Id's...
> >>>> 
> >>>> Do you consider this a bug?
> >>>> 
> >>> 
> >>> I do.  No idea what causes it or how to fix it though... :)
> >> 
> >> First I thougt it' not a severe bug since one see's more not less
> >> messages in notmuch show buffer.  But later I realised one also
> >> sees less not more threads in notmuch search buffer and might not
> >> read certain notmuch threads because of "wrong" $Subject: in
> >> notmuch search buffer.
> 
> > I think notmuch links two messages into the same thread if they have an
> > in-reply-to or reference header in common: i.e the messages reference a
> > common parent message.  (See comment in lib/database.cc "Even before a
> > message is added, it's pre-allocated thread ID is useful so that all
> > descendant messages that reference this common parent can be recognized
> > as belonging to the same thread.")
> 
> So in case message a from thread A and message b from B would
> name the same Message c in their In-Reoply-To:/References:
> headers, while c is not (for some reason) in A or B, notmuch
> would assume both threads linked?  Makes sense.
>  
> > As far as I can see your grep tests haven't checked for that. 
> 
> True.
> 
> > Also, could you email me the mbox you had (I think you said that it was
> > a mailing list so all public) and I will take a look?
> 
> Sure, I do so off-list because of the size of the attachment.

Hi 

I have looked at this and I think this is not notmuch's fault: I think
it is a mua doing strange things:

One of the mails has an in-reply-to header which looks like

In-reply-to: Message from Carsten Dominik <carsten.dominik at gmail.com> of        "Tue, 15 Mar 2011 12:18:51 BST."        <17242340-A14F-495A-B144-20C96D52B620 at gmail.com>

and I think notmuch is taking the carsten.dominik at gmail.com as message
id.

A similar in-reply-to header appears in the other thread so notmuch
pairs them up. According to http://www.jwz.org/doc/threading.html this
form of header is not allowed under RFC2822 but was allowed under the
earlier RFC822.

You can see several such messages on the gnu-mailing list site eg

ftp://lists.gnu.org/emacs-orgmode/2011-11 

search for "in-reply-to: M" but they all appear to be from the same
person (running mh-e 8.3 nmh under emacs 24)

In my collection from the linux kernel mailing list I get some examples
of in-reply-to not just being : <msg-id> but it was only about 200 from
100,000 messages in the second half of 2010 (the most recent archives I
have).

Best wishes

Mark