[RFC] Use JSON in emacs interface

[Austin Clements]

Quoth Chris Gray on Apr 29 at 10:22 am:
> Hi,
> 
> My thinking about this arises from the fact that there is a person on
> one of the lists that I read who puts a semicolon after his name.  Of
> course, this confuses the regex in notmuch-search-process-filter, which
> expects that the first semicolon in the string representing a thread is
> after all the authors.
> 
> I first thought of changing the regex so that it looked for the last
> semicolon in the string or something like that, but that would just move
> the problem.  (Semicolons are probably more frequent in subject lines
> than in author names.)  So it seems to me that what is needed is for
> notmuch and emacs to talk with each other in a format that is
> unambiguously parseable.  Since notmuch search already has the option of
> outputting to JSON, that seems like a natural fit.
> 
> Emacs has an existing JSON parser,
> <http://cvs.savannah.gnu.org/viewvc/*checkout*/emacs/lisp/json.el?root=emacs>,
> but it doesn't appear that it is able to parse progressively, meaning
> that it wouldn't be able to display results as they come in from notmuch
> search if used as-is.  My guess is that its parts could be hacked
> together to overcome this limitation though.
> 
> Anyway, if others think this is a good idea, I'm willing to do the
> coding.

This is definitely a good idea.  In fact, there's been quite a bit of
discussion about it in the past, and even some (very old and
outdated) implementation work:

  id:"878vs28dvo.fsf at praet.org"
    A rather lengthy discussion of another motivation for using JSON
    search.  Includes performance results for using JSON in search.

  id:"20120214152114.GQ27039 at mit.edu"
    Discussion of "framed" JSON that would be a valid JSON object, but
    could easily be consumed in a streaming fashion without hacking
    Emacs' JSON parser or resorting to paging.

  id:"1290777202-14040-1-git-send-email-dme at dme.org"
    The implementation from long ago.  Definitely outdated, but could
    be a good starting point.

It looks like most of the discussion about streaming JSON parsing has
been on the IRC channel, rather than the mailing list, but I'd be
happy to summarize it or dig out the IRC logs.  

I agree with Adam that it probably makes the most sense to get an
implementation working without streaming first and then add streaming
support.