[PATCH] NEWS: cleartext indexing

[Antoine Beaupré]

Hi,

Sorry for the long delay in my response, but it was a long email to
review - there's a lot of stuff in here - so I didn't quite know how to
respond. I'll just respond inline but will try to keep it brief.

On 2017-11-01 04:13:26, Daniel Kahn Gillmor wrote:
> On Mon 2017-10-30 12:16:25 -0400, Antoine Beaupré wrote:
>> I think that assumption should be made clear in the documentation,
>> because "security of your index" means nothing to me. Explicitly mention
>> FDE as an example may be a good start.
>
> again, i'm not convinced that "full disk" encryption is what's
> warranted, although filesystem, directory, or per-file encryption might
> be part of the solution.
>
> I don't want the documentation produced here to prescribe a particular
> solution, because i *want* people to experiment and investigate.
>
> I also don't think that the notmuch documentation is the right place to
> put a primer on filesystem encryption, or a treatise on the comparison
> of data at rest to data in flight.  I wouldn't object to pointers to
> more discussion here though, for people who want to read more.

The problem is we do not have such documentation to point to. The
original text I was critical of is this, to put things back in context:

> +  Note that the contents of the index are sufficient to roughly
> +  reconstruct the cleartext of the message itself, so please ensure
> +  that the notmuch index itself is adequately protected.  DO NOT USE
> +  this feature without considering the security of your index.

In other words, the documentation uses "adequately protected" and
"security of your index" as deliberately vague terms to urge the user to
consider practices. My concerns here is that those words may mean *very*
different things to different people. Just this discussion shows the
range of possible outcomes - I was thinking FDE, but then you suggested
that this was not adequate, with a plethora of possible solutions.

Which one do *you* use? I suspect most people that will adopt this
technology will simply *ignore* that warning and just store their index
in plaintext. In which case this warning is pointless and joins a long
series of fatigue-generating warnings that we constantly train our users
to ignore.

If we will warn, at least we should give guidelines of how to fix the
warning.

[... on-the-fly encryption of notmuch DB discarded ...]

[... possible actual solutions brainstorm for a notmuch lock / unlock
command based on ( ext4 encryption | notmuch-specific filesystem |
removable notmuch storage | etc ) ... ]

> Again, i don't think that notmuch documentation is a great place to
> document these ideas, unless we're actually implementing them in a
> simple and straightforward way so that the user can trigger the actions
> easily.

Sure. But this discussion shows that we do *not* have a clear and simple
recommendation for people. That's too bad, and bound to the limitation
of the system we're using of course (e.g. Linux instead of iOS) but we
shouldn't pretend there *are* solutions when there aren't, out of the
box, ways to secure the index.

[...]

>> Having my emails encrypted adds another layer of security to that
>> content. FDE is good for data "at rest", e.g. when i travel with my
>> laptop. But when my laptop is opened and running, I like to think that a
>> part of it isn't accessible without the security layers behind PGP and
>> actual human confirmation.

[...]

> So in practice, one authorization of your PGP key is likely to enable
> arbitrary programs to access it for the duration of the gpg-agent
> cache.  So your data is still actually accessible to any process that
> can access both the agent and the message store.

Sure. But the point is exactly that - there is *some* level of
control. I don't use a crypto token for encryption now, but when/if I
will there will be a satisfying, hardware, way of ensuring things are
encrypted at rest: just yank the key out. :) Short of that, yes, having
gpg-agent prompt me for stuff is a good compromise.

> That said, if there is a specific message which you think should not be
> in the index, the cleartext-index series and the session-key series both
> provide means for keeping a *specific* encrypted message in the
> mailstore while ensuring that it is not indexed and no session-keys are
> stashed.
>
> An interesting proposal might be to add an additional per-message
> property, which says explicitly "do not index cleartext or store session
> keys for this message".  I don't believe there are many users who would
> actually use this feature, but i would review a patch for it and provide
> feedback.

That would be an interesting feature, but I don't know how it could be
practically implemented. Notmuch feature discovery is limited, at best,
with the current feature set (e.g. there's no "Notmuch menu" where we
could show that feature) so I doubt we could easily add that to Notmuch
in any practical way.

[...]

> I disagree.  This patch series doesn't shift any burden to the user.

[...]

Granted.

[...]

>> I am also not sure if it's the best way to implement such a
>> tradeoff. Why not simply decrypt the actual email on delivery and store
>> them in cleartext if you're going to have a cleartext copy on the side
>> anyways? That would seem like a much simpler solution to the problem
>> you're trying to solve here...
>
> I hear this proposal about every month, i think, usually from different
> people.   I think it's a bad idea on several levels.

[...]

> No thanks!  These are bad tradeoffs, when i can get cleartext indexes
> and fast rendering without bothering with all of these other downsides.

Agreed, that was a straw man on my part. :)

[...]

> I'd be happy to explore what "per-app" encryption might mean on desktop
> systems as well with you, but we should probably do that off-list, and
> come back here when we've got something specific to propose.

Obviously, this will not happen in the short term, so we shouldn't block
this patch series based on the theorical idea of an eventual redesign of
the whole Linux filsystem encryption scheme. ;)

>> I definitely don't mean to block this. But I would like to see some
>> changes to the documentation to better explain those trade-offs, even if
>> it means just linking to this discussion. :)
>
> Please propose a patch to the documentation that would satisfy you!  I
> agree with you that having some more discussion would be useful, but the
> full content of even this thread would be out of place in any of the
> notmuch manpages.

Maybe not. But I believe a summary of alternatives like you did would be
way more honest and direct than alluding to ways of "securing the data"
when there aren't easy workable solutions.

Obviously, we have made this long thread about a fairly minor thing. I'd
love to see the session keys land, and I do not think the series should
block on this documentation item.

If anything, I'd love to hear more of your current thoughts about those
solutions in the documentation, if only have a pointer to this very
thread as a temporary fix. :)

Because, after all, this is history in writing: freezing our thoughts
into those emails may seem obvious to me and you, but will definitely be
useful and inspiring for others that come later. What better way to
discover those ideas than to embed them in the documentation? Otherwise
we expect people to rebuild those ideas from scratch and that seems like
a waste.

I, unfortunately, do not have an direct patch to fix the documentation,
but I hope I made you see the concerns I have about the current
wording. If I would be urged to make a patch or move on, I would add a
link to this thread and move on.

Thanks for your thoughts and hard work!

A.

-- 
The world is a dangerous place, not because of those who do evil,
but because of those who look on and do nothing.
                        - Albert Einstein