[notmuch] Search results now start appearing "instantly"

[Carl Worth]

Here's a nice feature for any of you who have been suffering with
"notmuch search" taking forever before it would complete, (such as when
trying to display "notmuch search tag:inbox" with every message in your
collection having the inbox tag).

I've fixed the "notmuch search" command-line program to not do any of
the "chunking" that it was doing before, but to instead stream results
out as quickly as possible. So it takes maybe a second or two before the
results start appearing, but then it's a steady stream from then on.

I've also fixed the emacs interface to "notmuch search" which previously
would wait until the command completed and would then do the processing
it needs to on the buffer. The new implementation instead fires off
"notmuch search" as an asynchronous process and filters the results as
they come in.

The net effect is that searches now appear instantly and you can just
watch the scrollbar shrink as the results keep coming in, (and you can
navigate and read messages as much as you'd like while results keep
coming).

It's working out fairly well, (but for one minor bug which is that you
lose your current position when you refresh). And I hope people are
happy with it. For some it might take notmuch from "interesting, but
too slow to be usable" to "blisteringly fast, and where have you been
all my life".

The one thing that might still be undesirable is that the "notmuch
search" process will still continue to burn CPU until all of the results
are complete. We could probably take some clues from the user's actions
to ameliorate this. It would be easy to suspend the search process when
the user obviously isn't needing more results and then resume it
later. Detecting what the user needs is a little tricky, but in some
cases will be obvious, (such as the user switching away from the scroll
buffer or the user viewing a tiny fraction of the search results and not
scrolling through them).

So please update to the latest code and let me know what you think. I'll
be interested to hear if this helps people, and also to know if the CPU
usage is a problem for anybody.

And for the authors of the other interfaces, let us know when you've got
similar support for streaming searches, (or if you didn't get this
automatically as soon as "notmuch search" was fixed).

-Carl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20091125/bfdac07d/attachment-0001.pgp>