[emacs] optimizing notmuch-search to only parse displayed messages

[Tristan Cacqueray]

Hi,

I have been testing emacs as a mail client recently, so first, thank you for
all the work on notmuch and the emacs mode, it's really neat :)

My main issue is that, while notmuch-search is super fast to show the first
messages, it seems like it is processing the whole query output in the
background, which is resulting in a long cpu load for large search...

Reading through the notmuch.el code, it seems like the search process output
is ingested by a scratch buffer, and iiuc we can't control how fast the stdout
gets processed. I tried adding some delay on the process-filter but that
doesn't seems to work.

I'm pretty much a lisp beginer, thus I don't know how doable this is, but
can we replace that scratch buffer or the make-process usage by something
that enables stdout's reading to be controled based on the window status
(e.g. only read more when the window reaches the end of the buffer) ?

As an alternative solution, I've looked into sending STOP and CONT signal to
the process using the patch below. It's a bit buggy when changing windows,
and it assumes the active window is the one being scrolled, but it does
make the notmuch-search window reads and parses the message when needed...

What would be the best way to improve notmuch-search efficiency?
(beside adding search limit to the queries...)

Cheers,
-Tristan


[PATCH] wip: stop notmuch-search process until window reach end of buffer

This change updates the process-filter function to send a SIG_STOP
signal to the notmuch-search process when the window is close to
the end of the buffer. Then a scroll-functions is used to send the
SIG_CONT when the window reaches the end of the buffer.
---
 emacs/notmuch.el | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 804e78ab..e42396e3 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -864,8 +864,29 @@ sets the :orig-tag property."
       (setq notmuch-search-target-thread "found")
       (goto-char pos))))

+;; notmuch-windows alist (window-name . process)
+(setq notmuch-windows nil)
+(defun notmuch-scroller (window window-start)
+  ;; (message "DEBUG: scroller position %d (%s)" (- (point-max) (window-end) 8192) (selected-window))
+  (if (<= (- (point-max) (window-end) 8192) 0)
+      ;; Window is near the end of the buffer
+      (progn
+        (let ((proc (alist-get (selected-window) notmuch-windows)))
+          (if (not (equal proc nil))
+              ;; Window is in notmuch-windows list
+              (progn
+                (message "DEBUG: Sending SIG_CONT signal to %d (%s)" (process-id proc) (selected-window))
+                ;; Resume the process
+                (signal-process proc 18)
+                ;; Remove it from the notmuch-windows list
+                (setq notmuch-windows (delq (assoc (selected-window) notmuch-windows) notmuch-windows))
+                ))))))
+;; Install the hook
+(add-hook 'window-scroll-functions 'notmuch-scroller)
+
 (defun notmuch-search-process-filter (proc string)
   "Process and filter the output of \"notmuch search\""
+  ;; (message "DEBUG: notmuch-search-process-filter [%d]" (length string))
   (let ((results-buf (process-buffer proc))
        (parse-buf (process-get proc 'parse-buf))
        (inhibit-read-only t)
@@ -877,7 +898,27 @@ sets the :orig-tag property."
          (goto-char (point-max))
          (insert string))
        (notmuch-sexp-parse-partial-list 'notmuch-search-append-result
-                                        results-buf)))))
+                                        results-buf))))
+
+  ;; (message "DEBUG: parser position %d" (- (window-end) (point-max) -16384))
+  (if (<= (- (window-end) (point-max) -16384) 0)
+      ;; Buffer is past the end of the window
+      (let ((old-proc (alist-get (selected-window) notmuch-windows)))
+        ;; Remove previous proc window association
+        (if (and old-proc (not (equal old-proc proc)))
+            (progn
+              (message "DEBUG: Removing stalled association")
+              (setq notmuch-windows (delq (assoc (selected-window) notmuch-windows) notmuch-windows))
+              ))
+        ;; Stop proc and associate to selected window
+        (if (not old-proc)
+            (progn
+              (add-to-list 'notmuch-windows `(,(selected-window) . ,proc))
+              (message "DEBUG: Sending SIG_STOP signal to %d (%s))" (process-id proc) (selected-window))
+              (signal-process proc 19)
+              ))))
+  )
+

 (defun notmuch-search-tag-all (tag-changes)
   "Add/remove tags from all messages in current search buffer.
--
2.19.2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20181217/4dda6d16/attachment.sig>