RFC: notmuch powered (personal) (end-to-end) e-mail system

[Austin Clements]

Much of the beauty of notmuch is how few assumptions it makes about
your mail system.  It plays well with others.  For example, one deep
insight of notmuch is that it *doesn't* require a custom mail store,
even though a more obvious design might; in fact, it doesn't even
require Maildir.

That said, I think I can see where you're coming from and I also think
you're targeting some of the deficiencies of notmuch, but I also think
you're overengineering the solution.  As a result of notmuch's
simplicity, a fully working mail setup requires a lot of moving parts
besides notmuch and it can take a while for a new user to set all that
up, especially if they're migrating wholesale from some external mail
setup.

On Sun, Mar 20, 2011 at 10:07 AM, Ciprian Dorin Craciun
<ciprian.craciun at gmail.com> wrote:
>    As such I'm thinking on implementing a custom end-to-end email
> system and I would like to hear your feedback before embarking on such
> a task.
>
>    I'm targeting the following features:
>    * (inbound) SMTP integration, thus once an email is received it is
> automatically pushed through the system; (I'm primarily targeting
> those users that afford to run their own SMTP server; but the solution
> could still be adapted for those that only want the other features;)

As others have mentioned, see notmuch-deliver.  I and others have also
suggested inotify support for notmuch before, which would make the
inbound mail mechanism (be it SMTP, IMAP fetching, or whatever)
completely unaware of notmuch, offer some other benefits (for example,
if mail is manipulated outside notmuch via IMAP), and is highly
discoverable for new users (just have notmuch setup ask if they want
notmuch to monitor for new email and then fire up an inotify daemon
the first time notmuch is called).

>    * automatic spam filtering, and tag applying;
>    * automatic email triggers based on tags (such as user
> notifications, forwarding, etc.)

Obviously the above two can be scripted, but I agree that it's
unsatisfying that every user needs to roll their own delivery script.
While tagging and triggering are highly personal, they're not *so*
personal that everyone needs a completely custom solution.  This
should be more approachable.  I'm not sure what the best answer here
is, but I don't think it requires it requires integration with a
monolithic system to do right.

>    * remote RPC-like access to the whole system;

This is another deep insight of notmuch.  It already has an awesome
RPC interface: the CLI.  Perhaps your actual problem is that the only
supported remote transport protocol is SSH.  This comes with a lot of
benefits (authentication, RPC pipelining), but also a lot of baggage
(a full SSH client on the client side).  I've thought about this in
the context of both an HTTP client and an Android client and in both
cases I concluded that a simple HTTPS transport wrapped around the
notmuch CLI would be the way to go.  Just put the CLI arguments in a
POST and send the JSON on stdout back.  This is trivial to prototype
as a Python CGI script, easy to build as a standalone Python server,
and not especially hard to build as a robust C server.

>    * remote Web user interface;

A good web UI would be fantastic.  Based on the rest of your email, I
get the impression this was a requirements driver from much of the
above, especially the integrated tagging/triggering and RPC access.
I've already suggested a simple solution to the RPC problem.  For
tagging/triggering, it's probably worth developing a solution that
allows for machine-editable rules (ideally retaining
user-editableness), which would make it possible to integrate filter
management in to a web UI.  This could be as simple as a standard
delivery script that operates from some simple rule database.