notmuch reply is confused about multiple (or case-insensitive?) cc headers

[Jani Nikula]

On Fri, 17 Jun 2016, Daniel Kahn Gillmor <dkg at fifthhorseman.net> wrote:
> hey notmuchers--
>
> I'm using notmuch from close to git master.
>
> I just received an e-mail with headers like this:
>
> ------
> From: Alice <alice at example.org>
> To: Daniel Kahn Gillmor <dkg at fifthhorseman.net>
> Cc: Bob <bob at example.org>
> Subject: wowsers!
> cc: Charles <charles at example.org>
> Message-Id: <abc123 at example.org>
> Date: Thu, 16 Jun 2016 22:14:41 -0400
>
> blah blah
> ------
>
> Note the weirdness about the Cc fields!
>
>  https://tools.ietf.org/html/rfc5322#page-21
>
> Makes it clear that there is supposed to only be zero or one cc header
> in an e-mail, but notmuch-emacs is nice enough to show me both.
>
> However, if i do:
>
>   notmuch reply id:abc123 at example.org
>
> then Charles is omitted:
>
> ----------
> From: Daniel Kahn Gillmor <dkg at fifthhorseman.net>
> Subject: Re: wowsers!
> To: Alice <alice at example.org>
> Cc: Bob <bob at example.org>
> In-Reply-To: <abc123 at example.org>
> References: <abc123 at example.org>
>
> On Thu, 16 Jun 2016 22:14:41 -0400, Alice <alice at example.org> wrote:
>> blah blah
> ----------
>
>
> In json form:
>
> 0 dkg at alice:~$ notmuch reply --format=json id:abc123 at example.org | json_pp 
> {
>    "reply-headers" : {
>       "From" : "Daniel Kahn Gillmor <dkg at fifthhorseman.net>",
>       "References" : "<abc123 at example.org>",
>       "To" : "Alice <alice at example.org>",
>       "Cc" : "Bob <bob at example.org>",
>       "Subject" : "Re: wowsers!",
>       "In-reply-to" : "<abc123 at example.org>"
>    },
>    "original" : {
>       "id" : "abc123 at example.org",
>       "filename" : "REDACTED",
>       "timestamp" : 1466129681,
>       "match" : false,
>       "headers" : {
>          "Cc" : "Bob <bob at example.org>, Charles <charles at example.org>",
>          "Date" : "",
>          "Subject" : "wowsers!",
>          "From" : "Alice <alice at example.org>",
>          "To" : "Daniel Kahn Gillmor <dkg at fifthhorseman.net>"
>       },
>      "tags" : [],
>      "excluded" : false"
>    ...
>
>
>
> Note that we see Charles in original[headers[Cc]], but not in
> reply-headers[Cc].
>
> Seems to me like these two should agree.  We should either decide to
> take the strict interpretation of RFC 5322 or not.  I lean toward
> including Charles in both sets, fwiw.

The two come from code paths surprisingly far away from each other.

The original message is formatted using the message printing in
notmuch-show.c. For Cc:, it uses g_mime_message_get_recipients(), which
apparently combines all Cc: fields into one internally.

The addresses in the reply headers, OTOH, are based on headers queried
through libnotmuch. It boils down to g_mime_object_get_header() in
lib/message-file.c, which returns only the first occurence of header.

> Anyway, smells like a bug in a weird corner case, and i haven't had time
> to track it down or fix it, but i figured i'd report it so that other
> people can see it.

I don't think I'd do anything to fix this particular issue. It's a
corner case, we comply with the RFC, and all that.

However, I would start thinking about our usage of gmime directly
vs. querying through the lib in the cli. The lib has some caching for
efficiency, and is probably the way to go for indexing and searching
etc. But perhaps we should just hit the disk directly using gmime when
we're reading and parsing the messages to show in the cli? So perhaps we
should be using gmime calls instead of notmuch lib calls to get the
from/to/cc headers in notmuch reply, for example. This would fix the
issue as a side effect. (You could argue we should do this just to fix
this bug, but I think it's a bigger design decision.)

Side note #1, even in the indexing code in the lib we use
g_mime_message_get_all_recipients(), so I think to:charles will find the
original message.

Side note #2, I think we should provide *all* versions of reply-headers
(reply-to-sender, reply-to-all, mail-followup-to, whatnot) in the
formatted output, and let the user toggle between them in notmuch-emacs
while composing the reply.

BR,
Jani.