[notmuch] Thoughts on notmuch and Lua

[Carl Worth]

On Fri, 15 Jan 2010 23:09:34 +0200, Ali Polatel <alip at exherbo.org> wrote:
> Carl Worth yazmış:
> > What do you think, Ali? Would an approach like that satisfy the things
> > you had in mind for hooks?
> 
> It might, here are some thoughts and questions to help you elaborate:
> 
> - How will these scripts manipulate data?
>   e.g.: A "tagger" script may get the new mail from stdin and print out
>   the new tags which notmuch will read and apply to the message.

This can happen with current notmuch entirely with no real "hook"
support.

We've talked about switching from default tags of "inbox" and "unread"
to simply having new mail tagged with a "new" tag. So a "tagger" script
could operate simply by doing a "notmuch search" for messages with the
"new" tag and could iterate over the filenames to process actual
messages. (We don't have support now for emitting just filenames from a
"notmuch search", but we have patches for that, and I'll be applying one
soon.)

So that's a taste for the "scriptability" I see in the current notmuch
system that makes it really much more flexible than any "hooks"
system. Additional flexibility comes from:

  * User can run a script like this at any point---not merely when
    messages are added.

  * User script isn't restricted to dealing only with "new" messages,
    but can act on any set of messages based on any search constraint,
    (or even all messages in the database).

The results can then be applied by simply calling "notmuch tag" as
needed.

And if there are any performance problems there we can fix them, (such
as, perhaps we'll end up wanting this script to be able to invoke a
single process for all of its tagging rather than calling "notmuch tag"
over and over).

>   A "search-filter" script may get search results from stdin and filter
>   them. Just my initial thoughts.

And how would this search functionality and filtering be different than
the search functionality provided by notmuch itself?

I can think of at least a couple of ways it might be different:

  1. It would be nice to be able to filter based on tags that are
     present in a thread, though perhaps not present in any message
     matching the original search.

     An obvious application of this is the "thread muting" feature,
     where once a message is tagged as "muted", no messages delivered to
     that thread in the future will appear in the inbox.

     This is a feature I'd like to put into the core of notmuch such
     that one passes a query to match messages and then also a second
     query to filter based on the collected tags in threads. Something
     like:

	notmuch search tag:inbox --filter="not tag:muted"

 2. There are other details available at the thread level that are not
    available at the level at which message-based searching happens.

    A simple example of this would be the ability to search for threads
    with a single message, (perhaps checking to ensure that all requests
    had gotten at least one reply). But one can imagine more complex
    things as well, "Show me all threads where ImportantPerson sent a
    message and where I never replied in the thread."

    For this kind of thing, I think we simply want to build on the
    output of "notmuch search". The current output isn't very usable for
    this, but with things like the structured json output, etc. (which,
    again, I hope to be merging soon), it would be quite easy to write
    new tools that accept that output and provide additional searching
    and filtering, etc. And that tool could provide lua-based scripting
    or whatever else is desired.

So my feeling is that if anything can live outside of notmuch, then it
should, and should simply build on top of notmuch output. (And we should
fix notmuch output to support that well.)

And anything that must live within notmuch (or is best supported there),
we should see if we can't just make that a core part of notmuch itself,
(such as the --filter option I showed above).

I'm *still* not wanting to squelch any experimentation with embedding
scripting languages inside notmuch, or anything else. I'm just still not
seeing anything that requires this.

Look at the amount of emacs-lisp code we've written, for example, and
the various things it does, (hiding away citations, etc.). That's all
"scripting code", but that sits easily on top of the existing notmuch
command-line tool.

I think I'd prefer to keep that nice clean boundary until we find
something that really requires changing that. But, show me something
really cool that requires it, and you might convince me. :-)

-Carl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100115/aaacd133/attachment.pgp>