[PATCH 5/5] emacs: Use streaming S-expr parser for search

[Mark Walters]

This patch and patch 3/5 look good to me. Do you intend to take out the
async json parser immediately after this goes in? If not then I think it
would be good to tweak the arguments/error handling in the json parser
so that it matches that in this sexp parser.

Best wishes

Mark

On Sat, 18 May 2013, Austin Clements <amdragon at MIT.EDU> wrote:
> In addition to being the Right Thing to do, this noticeably improves
> the time taken to display the first page of search results, since it's
> roughly an order of magnitude faster than the JSON parser.
> Interestingly, it does *not* significantly improve the time to
> completely fill a large search buffer because for large search
> buffers, the cost of creating author invisibility overlays and
> inserting text (which slows down with more overlays) dominates.
> However, the time required to display the first page of results is
> generally more important to the user experience.
> ---
>  emacs/notmuch.el |   13 +++----------
>  test/emacs       |   10 +++-------
>  2 files changed, 6 insertions(+), 17 deletions(-)
>
> diff --git a/emacs/notmuch.el b/emacs/notmuch.el
> index b8d9c44..5a8c957 100644
> --- a/emacs/notmuch.el
> +++ b/emacs/notmuch.el
> @@ -58,6 +58,7 @@
>  (require 'notmuch-hello)
>  (require 'notmuch-maildir-fcc)
>  (require 'notmuch-message)
> +(require 'notmuch-parser)
>  
>  (defcustom notmuch-search-result-format
>    `(("date" . "%12s ")
> @@ -809,13 +810,6 @@ non-authors is found, assume that all of the authors match."
>  	(setq notmuch-search-target-thread "found")
>  	(goto-char beg)))))
>  
> -(defun notmuch-search-show-error (string &rest objects)
> -  (save-excursion
> -    (goto-char (point-max))
> -    (insert "Error: Unexpected output from notmuch search:\n")
> -    (insert (apply #'format string objects))
> -    (insert "\n")))
> -
>  (defun notmuch-search-process-filter (proc string)
>    "Process and filter the output of \"notmuch search\""
>    (let ((results-buf (process-buffer proc))
> @@ -829,8 +823,7 @@ non-authors is found, assume that all of the authors match."
>  	(save-excursion
>  	  (goto-char (point-max))
>  	  (insert string))
> -	(notmuch-json-parse-partial-list 'notmuch-search-show-result
> -					 'notmuch-search-show-error
> +	(notmuch-sexp-parse-partial-list 'notmuch-search-show-result
>  					 results-buf)))))
>  
>  (defun notmuch-search-tag-all (&optional tag-changes)
> @@ -933,7 +926,7 @@ Other optional parameters are used as follows:
>        (save-excursion
>  	(let ((proc (notmuch-start-notmuch
>  		     "notmuch-search" buffer #'notmuch-search-process-sentinel
> -		     "search" "--format=json" "--format-version=1"
> +		     "search" "--format=sexp" "--format-version=1"
>  		     (if oldest-first
>  			 "--sort=oldest-first"
>  		       "--sort=newest-first")
> diff --git a/test/emacs b/test/emacs
> index d38ae8c..7d42abf 100755
> --- a/test/emacs
> +++ b/test/emacs
> @@ -856,7 +856,7 @@ test_expect_success "Rendering HTML mail with images" \
>  test_begin_subtest "Search handles subprocess error exit codes"
>  cat > notmuch_fail <<EOF
>  #!/bin/sh
> -echo This is output
> +echo '()'
>  exit 1
>  EOF
>  chmod a+x notmuch_fail
> @@ -871,21 +871,19 @@ test_emacs "(let ((notmuch-command \"$PWD/notmuch_fail\"))
>  	       (test-output))"
>  sed -i -e 's/^\[.*\]$/[XXX]/' ERROR
>  test_expect_equal "$(cat OUTPUT; echo ---; cat MESSAGES; echo ---; cat ERROR)" "\
> -Error: Unexpected output from notmuch search:
> -This is output
>  End of search results.
>  ---
>  $PWD/notmuch_fail exited with status 1 (see *Notmuch errors* for more details)
>  ---
>  [XXX]
>  $PWD/notmuch_fail exited with status 1
> -command: $PWD/notmuch_fail search --format\=json --format-version\=1 --sort\=newest-first tag\:inbox
> +command: $PWD/notmuch_fail search --format\=sexp --format-version\=1 --sort\=newest-first tag\:inbox
>  exit status: 1"
>  
>  test_begin_subtest "Search handles subprocess warnings"
>  cat > notmuch_fail <<EOF
>  #!/bin/sh
> -echo This is output
> +echo '()'
>  echo This is a warning >&2
>  echo This is another warning >&2
>  exit 0
> @@ -903,8 +901,6 @@ test_emacs "(let ((notmuch-command \"$PWD/notmuch_fail\"))
>  	       (test-output))"
>  sed -i -e 's/^\[.*\]$/[XXX]/' ERROR
>  test_expect_equal "$(cat OUTPUT; echo ---; cat MESSAGES; echo ---; cat ERROR)" "\
> -Error: Unexpected output from notmuch search:
> -This is output
>  End of search results.
>  ---
>  This is a warning (see *Notmuch errors* for more details)
> -- 
> 1.7.10.4