Protected Headers (2nd major revision, more testing!)

[Daniel Kahn Gillmor]

Hi all--

Way back in id:20180511055544.13676-1-dkg at fifthhorseman.net, i
proposed support for protected headers (in particular, for being able
to read and search for subject lines of encrypted messages which
protect the Subject).  Although that series was reviewed by Bremner, i
never managed to get it in shape for merging.

This is a revision of that series, applied against the current master,
having taken into account those reviews and the current state of the
notmuch codebase.  I'm hoping that we can get it into 0.29 before the
feature freeze.

The major change since the earlier version is that i've dropped the
proposed --protected-subject flag for "notmuch reply".  An MUA that
wants to reply to an encrypted message needs to keep a lot of state
active during message composition, including the fact that it was a
reply to an encrypted message, and so forth.  It needs to know that if
the user switches encryption off or on during message composition (for
whatever reason, like adding a Cc to someone for whom we don't have
keys, or discovering that some of the recipients keys are no longer
valid), it needs to think about whether the subject line is stripped
or not actively, and passing a simple --protected-subject flag to
"notmuch reply" during the initial setup of message composition is
insufficient for that purpose.  So this series doesn't pretend to
handle that case directly -- clients will need to consider it
themselves.

See the message in commit "cli/reply: ensure encrypted Subject: line
does not leak in the clear" for more thoughts about what a reasonable
replying MUA might do.

This series also (like its earlier incarnation) doesn't get all the
way to the point of generating encrypted or signed messages that
protect their Subject lines.  That might require some e-lisp hackery
that i haven't done; or it might be best solved by a "notmuch deliver"
outbound message handler (which is also work i haven't done). Or maybe
there's some other better solution that i haven't thought of yet.  I
welcome discussion and suggestions along those lines.

The other thing this series does not do is to expose information about
the protected headers through the library or the python bindings.  I
think the pieces are in place to make that happen, but I have not
considered the API deeply enough to take a concrete attempt.  Again,
suggestions (and patches) welcome!

However, despite the above-mentioned limitations, this series delivers
a concrete improvement: users of notmuch can now read, index, and
search for the subject lines of encrypted messages sent from MUAs like
Enigmail and K-9 mail.

Also: please don't be scared of the length of this series.  Although
there are 17 patches, the distinct majority of them are extensions to
the test suite, to make sure that we cover weird corner cases between
the MIME spec and this now-common form of header protection.

As always, review, feedback, critique, and patches are welcome.

Happy Hacking,

   --dkg