[PATCH 07/11] emacs: Use notmuch tag --batch for large tag queries

[Austin Clements]

Quoth Mark Walters on Oct 09 at  8:38 am:
> 
> On Wed, 09 Oct 2013, Jani Nikula <jani at nikula.org> wrote:
> > On Tue, 08 Oct 2013, Austin Clements <amdragon at MIT.EDU> wrote:
> >> (Unfortunately, it's difficult to first demonstrate this problem with
> >> a known-broken test because modern Linux kernels have argument length
> >> limits in the megabytes, which makes Emacs really slow!)
> >> ---
> >>  emacs/notmuch-lib.el |  8 ++++++++
> >>  emacs/notmuch-tag.el | 12 ++++++++++--
> >>  test/emacs           |  8 ++++++++
> >>  3 files changed, 26 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
> >> index 22156f1..348112b 100644
> >> --- a/emacs/notmuch-lib.el
> >> +++ b/emacs/notmuch-lib.el
> >> @@ -261,6 +261,14 @@ user-friendly queries."
> >>    "Return a query that matches the message with id ID."
> >>    (concat "id:" (notmuch-escape-boolean-term id)))
> >>  
> >> +(defun notmuch-hex-encode (str)
> >> +  "Hex-encode STR (e.g., as used by batch tagging).
> >> +
> >> +This replaces spaces, percents, and double quotes in STR with
> >> +%NN where NN is the hexadecimal value of the character."
> >> +  (replace-regexp-in-string
> >> +   "[ %\"]" (lambda (match) (format "%%%02x" (aref match 0))) str))
> >> +
> >>  ;;
> >>  
> >>  (defun notmuch-common-do-stash (text)
> >> diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
> >> index 064cfa8..a4eec14 100644
> >> --- a/emacs/notmuch-tag.el
> >> +++ b/emacs/notmuch-tag.el
> >> @@ -242,6 +242,8 @@ from TAGS if present."
> >>  	   (error "Changed tag must be of the form `+this_tag' or `-that_tag'")))))
> >>      (sort result-tags 'string<)))
> >>  
> >> +(defconst notmuch-tag-argument-limit 1000)
> >> +
> >>  (defun notmuch-tag (query &optional tag-changes)
> >>    "Add/remove tags in TAG-CHANGES to messages matching QUERY.
> >>  
> >> @@ -268,8 +270,14 @@ notmuch-after-tag-hook will be run."
> >>  	tag-changes)
> >>    (unless (null tag-changes)
> >>      (run-hooks 'notmuch-before-tag-hook)
> >> -    (apply 'notmuch-call-notmuch-process "tag"
> >> -	   (append tag-changes (list "--" query)))
> >> +    (if (<= (length query) notmuch-tag-argument-limit)
> >> +	(apply 'notmuch-call-notmuch-process "tag"
> >> +	       (append tag-changes (list "--" query)))
> >> +      ;; Use batch tag mode to avoid argument length limitations
> >> +      (let ((batch-op (concat (mapconcat #'notmuch-hex-encode tag-changes " ")
> >> +			      " -- " query)))
> >> +	(message "Batch tagging with %s" batch-op)
> >
> > Why the message?
> >
> > Jani.
> 
> I had missed that: this message is presumably for debugging as it
> includes the (possibly megabyte sized) query. I think some message would
> be sensible as this could be a slow operation so it's probably worth
> telling the user why the interface has locked.

You're right, this was a leftover debug message.  While I wouldn't
mind a progress message for long tagging operations, query length
isn't a good approximation of how long it's going to take.  I was
planning to include messages on tagging operations in my undo series,
which I think can do it in a more principled way (assuming I can find
a clean way to update tags in the UI on undo).

> Best wishes
> 
> Mark