[emacs] optimizing notmuch-search to only parse displayed messages

[Tristan Cacqueray]

On Mon, Dec 17, 2018 at 07:59 Tristan Cacqueray wrote:
> Hi,
>
> I have been testing emacs as a mail client recently, so first, thank you for
> all the work on notmuch and the emacs mode, it's really neat :)
>
> My main issue is that, while notmuch-search is super fast to show the first
> messages, it seems like it is processing the whole query output in the
> background, which is resulting in a long cpu load for large search...
>
> Reading through the notmuch.el code, it seems like the search process output
> is ingested by a scratch buffer, and iiuc we can't control how fast the stdout
> gets processed. I tried adding some delay on the process-filter but that
> doesn't seems to work.
>
> I'm pretty much a lisp beginer, thus I don't know how doable this is, but
> can we replace that scratch buffer or the make-process usage by something
> that enables stdout's reading to be controled based on the window status
> (e.g. only read more when the window reaches the end of the buffer) ?
>
> As an alternative solution, I've looked into sending STOP and CONT signal to
> the process using the patch below. It's a bit buggy when changing windows,
> and it assumes the active window is the one being scrolled, but it does
> make the notmuch-search window reads and parses the message when needed...
>
> What would be the best way to improve notmuch-search efficiency?
> (beside adding search limit to the queries...)
>

Hi,

Here is a quick follow-up from the irc discussions.

The SIGSTOP trick doesn't work well with ssh wrapper, because the signal
doesn't propagate to the remote process with the standard ssh client.
Moreover, resuming a search after a db update may not work too. However
this works well for me, and below is an improved implementation as other
may find it useful.

Alternative solutions are:
- the query limit patch[0], but extending reload the whole search,
- use pagination, but this needs a get_msgs() patch.

Thank you all for the suggestions.
-Tristan

[0]: https://notmuchmail.org/pipermail/notmuch/2011/006297.html


-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-wip-add-notmuch-progressive-search-custom.patch
Type: text/x-diff
Size: 7973 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20190110/0d30d6f9/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 487 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20190110/0d30d6f9/attachment.sig>