[PATCH v3 00/7] test: (hopefully) better test prerequisites

[Pieter Praet]

On Sun, 27 Nov 2011 22:36:12 +0400, Dmitry Kurochkin <dmitry.kurochkin at gmail.com> wrote:
> Changes:
> 
> v3 since v2:
> 
> * rebased the remaining patches on current master
> 
> v2 since v1:
> 
> * Add test_require_external_prereq function to explicitly check for
>   external dependencies, use it in test_emacs.
> 
> * Indenting fixes.
> 
> * Use $binary instead of $1 in test_declare_external_prereq.
> 
> Regards,
>   Dmitry
> 
> _______________________________________________
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Fantastic work Dmitry!  My inept dabblings are thoroughly humbled
by the elegance of this approach!

All that's left to go from three to four nines, is ensuring correct
$test_{count,success,fixed,broken,failure} reports (which can most
definitively be categorized as "non-trivial")...

  Eg. with and without emacs(1), emacsclient(1), dtach(1), gpg(1) :

    #+begin_example
      Notmuch test suite complete.
      All 347 tests behaved as expected (2 expected failures).
    #+end_example

    #+begin_example
      Notmuch test suite complete.
      266/312 tests passed.
      1 broken test failed as expected.
      5 tests failed.
      40 tests skipped.
    #+end_example

... as well as a minor issue re test_emacs(), which will be
addressed in a followup patch.


Peace

-- 
Pieter