[BUG] notmuch-emacs: spoils sendmail -f with emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c and later

[David Edmondson]

On Sunday, 2019-12-01 at 18:10:59 +01, Gregor Zattler wrote:

> [@Stefan: FYI]
> Dear notmuch developers,
>
> I use notmuch-emacs configured with
> (setq mail-specify-envelope-from t)
> (setq mail-envelope-from 'header)
> (setq send-mail-function 'sendmail-send-it)
>
> this should/used to invoke sendmail (in my case exim4) with the email
> address given in the From: header of the message buffer as
> argument to sendmails -f option.

Wow, this was, err, “interesting”.

Comments below.

> Since emacs 27 commit 3a59cc84069376802ba8fd731b524d78db58262c
>
>      3a59cc84069376802ba8fd731b524d78db58262c
>      Author:     Stefan Monnier <monnier at iro.umontreal.ca>
>      AuthorDate: Tue Jul 30 16:37:01 2019 -0400
>      Commit:     Stefan Monnier <monnier at iro.umontreal.ca>
>      CommitDate: Tue Jul 30 16:37:01 2019 -0400
>
>      Parent:     add146f09f * lisp/bindings.el (mode-line-defining-kbd-macro): New defvar.
>      Contained:  master
>      Follows:    emacs-26.1 (6691)
>
>      * lisp/gnus/message.el: Reduce redundancy with send-mail-function
>
>      (message-send-mail-function) <function>: Remove `local-library` tests
>      for libs distributed with Emacs.
>      (message-use-send-mail-function): New function.
>      (message-default-send-mail-function): Default to it, and remove cases
>      already handled by it.
>      (message--default-send-mail-function): New function.
>      (message-send-mail-function) <variable>: Use it as new default.
>      (message-sendmail-f-is-evil): Obey mail-specify-envelope-from if available.
>      (message-check, message-with-reply-buffer): Use `declare`.
>      (message-smtpmail-send-it): smtpmail accepts mail-header-separator,
>      so simplify and declare obsolete.
>      (message-send-mail-with-mailclient): Declare obsolete.
>      (message-check-news-body-syntax): Don't presume that the checksum is
>      a fixnum.
>
>
> this actually invokes sendmail -f with an address derived from the
> EMAIL environment variable.
>
>
> It took me some time to understand, that there is no problem with
> compose-mail/message-send-and-exit which are bound to ^X m and ^C
> ^C respectively within a barely configured emacs which I used for
> testing.
> If instead notmuch is loaded compose-mail/message-send-and-exit
> and notmuch-mua-new-mail/notmuch-mua-send behave different.
>
> With this command (with test.el as attached):
>
> ~/src/emacs/src/emacs -Q -nw -l /tmp/test.el
>
> compose-mail/message-send-and-exit rightly show
> -f example at example.org in the file /tmp/result (and then there is
> an error because the notmuch commands are not known).
>
> While this command
>
> ~/src/emacs/src/emacs -Q -nw -L ~/src/notmuch/emacs/ --eval "(require 'notmuch)" -f notmuch -l /tmp/test.el
>
> wrongly show some other value for -f, depending on your EMAIL
> environment variable, for both compose-mail/message-send-and-exit
> and notmuch-mua-new-mail/notmuch-mua-send.
>
>
> I read the code but I have no idea why and how to debug this.
>
> Ciao; Gregor
> --
>  -... --- .-. . -.. ..--.. ...-.-
>
> (find-file "/tmp/sent")
> (insert "empty")
> (kill-region (point-min) (point-max))
> (write-file "/tmp/sent")
> (kill-buffer)
>
> (find-file "/tmp/result")
> (kill-region (point-min) (point-max))
> (write-file "/tmp/result")
> (kill-buffer)
>
> (find-file "/tmp/show-sendmail-args.sh")
> (kill-region (point-min) (point-max))
> (insert "#!/bin/sh
> echo $* >> /tmp/result")
> (write-file "/tmp/show-sendmail-args.sh")
> (chmod "/tmp/show-sendmail-args.sh" 448)
>
> (setq sendmail-program "/tmp/show-sendmail-args.sh")
> (setq mail-specify-envelope-from t)
> (setq mail-envelope-from 'header)
> (setq send-mail-function 'sendmail-send-it)
>
> (compose-mail "Echo Mail Server <echo at univie.ac.at>" "test"
>                '((From: . "example at example.org")))

This example, and the one below, are broken - you can't use “From:”
here, it has to be “From”. Changing that doesn't fix it, though.

> (message-send-and-exit)
>
> (notmuch-mua-mail "Echo Mail Server <echo at univie.ac.at>" "test"
>                '((From: . "example at example.org")))
> (notmuch-mua-send-and-exit)
>
> (save-buffers-kill-terminal)

As best I can determine, this relates to the order in which things are
loaded.

If you load message.el before setting `mail-specify-envelope-from',
things are broken (sorry for the long lines):

disaster-area ~/s/emacs % ./src/emacs -Q -nw -batch --eval "(progn (require 'message) (setq mail-specify-envelope-from t mail-envelope-from 'header) (message \"%s\" message-sendmail-envelope-from))"
nil
disaster-area ~/s/emacs % 

...but if you load it after, things work fine:

disaster-area ~/s/emacs % ./src/emacs -Q -nw -batch --eval "(progn (setq mail-specify-envelope-from t mail-envelope-from 'header) (require 'message) (message \"%s\" message-sendmail-envelope-from))"
header
disaster-area ~/s/emacs %

This doesn't appear to be new behaviour - 26.1 does the same.

This is related to the way that `message-sendmail-envelope-from' is
initialised from `mail-envelope-from', and it's
`message-sendmail-envelope-from' that matters, because you end up in
`message-send-mail-with-sendmail'.

message.el is being loaded by “(require 'notmuch)” in your example,
which is happening before `mail-specify-envelope-from' is set, so you
see the failure mode described.

The example you provided is obviously contrived to show the problem -
are you hitting it in normal use?

dme.
-- 
Does everyone stare the way I do?