[PATCH 10/10] test: use emacsclient(1) for Emacs tests

[Dmitry Kurochkin]

On Tue, 28 Jun 2011 15:14:04 -0700, Carl Worth <cworth at cworth.org> wrote:
> On Tue, 28 Jun 2011 08:45:11 +0400, Dmitry Kurochkin <dmitry.kurochkin at gmail.com> wrote:
> > Before the change, every Emacs test ran in a separate Emacs
> > instance.  Starting Emacs many times wastes considerable time and
> > it gets worse as the test suite grows.  The patch solves this by
> > using a single Emacs server and emacsclient(1) to run multiple
> > tests. 
> 
> Great, great stuff, Dmitry!
> 
> I've pushed everything earlier than this patch in this series.
> 
> And I'd be fine pushing this one as well, (if Austin cares strongly
> about not polling, I'll invite him to improve things from here).
> 
> Except that things don't actually work for me with this patch applied.
> 
> I'm not getting consistent results from the test suite, (I have seen
> both "2 tests failed" and "5 tests failed"). Here are the failures from
> a recent run with 5 failures:
> 
>  FAIL   decryption, --format=text
>  FAIL   decryption, --format=json
>  FAIL   decryption, --format=json, --part=4
>  FAIL   decrypt attachment (--part=5 --format=raw)
>  FAIL   decryption failure with missing key
> 
> In each case the actual output was either empty, or an empty JSON
> array. In some cases notmuch also gave an error:
> 
> Error: search term did not match precisely one message.
> 
> Are you not getting these same failures?
> 

Ouch.  I never saw these.  Do you get these in crypto tests only?

Regards,
  Dmitry

> Let me know what else I can do to investigate this.
> 
> -Carl
> 
> -- 
> carl.d.worth at intel.com