[RFC][alot] design decisions

[Patrick Totzke]

Hi all,

with alot 0.3 released, I started thinking seriously about proper™ MIME-display and gnupg
integration for alot. I have to make a few design decisions here and could really use
some informed opinions from more experienced UI developers. My question in particular is: 

 How do I best integrate calls to external mime-handlers into my MVC event driven code?

I mostly know how to implement the individual parts but I struggle coming up with a
consistent way of organizing it all without compromising my current layout.


Background
----------

Model. Alot uses an abstraction layer over notmuch's thread, message and database objects
to make them less fragile and generally behave more OOP-y: You can e.g. directly call
message.add_tags(['foo','bar']) and so on. This is made possible by a "DatabaseManager"
that maintains centralized and queued access to the notmuch index and deals with notmuch
exceptions like a locked index [0].

View/Controler. We use an event-driven interface (urwid.TwistedEventLoop, [1]) that
handles user input and maintains a tree of widgets that knows how to render itself.
Twisted offers a neat concept of "Deferreds" to organize call- and errbacks that we
already make use of [2].  Alot has a "MessageWidget" that displays a single message.

We can easily access and parse email messages into MIME-trees (msg.get_email() returns a
email.message object; pythons email module allows extensive and RFC compliant dealing with
this [3]).

We already offer full compatibility with the mailcap protocol to constuct shell commands
of external MIME handlers.

We know how to call external commands asynchronously; `alot.helper.call_cmd_async` does this
and returns a Deferred to which we can connect callback functions.


Objective
---------
We want to turn a email.message into nicely rendered text and display it as a widget.
Calls to renderers should be done asynchronously, so that displaying large threads
does not block the interface unnecessarily.
The text representation should be stored in a way that can be incrementally updated
for use with message parts decrypted at a later point.


Bad Solutions
-------------

1. Current solution: walk the MIME-tree depth-first, use blocking calls to handlers
and create a fixed "body" string that is displayed in the widget.
A silly and hackish solution that we want to get rid of for obvious reasons.

2. Use a "PartWidget" that is able to display a node in the MIME-tree:
Its constructor creates the representation in a RFC compliant way, that is,
it decodes 'text/plain' parts, stacks other widgets of the same kind on top of each other
for multipart parts or calls external renderer to get a text representation.

Cons:
 * widgets should not be clever and contain/construct any kind of additional info
 * its hard to update these widgets when a part gets decrypted; 
 * decrypted parts are not available outside the displaying widget, widgets get
   replaced completely when one rebuilds the buffer (refresh cmd)
Pros:
 * we can use Deferreds relatively painlessly in the ui (as opposed to the db-wrapper)

3. Accumulate rendered text in a tree structure *inside* message-objects and let that
massage update the widget that displays it. One could let the widget register
some update-callback with the widget that rebuilds the widgets content.
These callbacks are called after the external handler has finished and the message-
internal text-representation has been updated.

Pros: 
 * rendered text/decrypted message parts end up in the message and are available to process
   and redisplay at will
Cons:
 * the calls to MIME-handlers are dealt with as Deferreds which depend on the twisted event
   loop and indirectly on urwid. This is exactly the kind of tight coupling I'm trying to avoid:
   Messages should be independent of the interface.
 * Messages with multiple externally rendered parts will rebuild their widgets more than once
 * references to old widgets (replaced e.g. after a buffer rebuild) will be kept, which keeps
   them from being garbage-collected

I'm surely missing an easy and elegant solution here.
Any pointers or comments are much appreciated.
Thanks,
/p


[0]: http://alot.readthedocs.org/en/latest/api/database.html
[1]: http://excess.org/urwid/
[2]: http://twistedmatrix.com/documents/current/core/howto/defer.html
[3]: http://docs.python.org/library/email