[PATCH v3 0/5] Git-based modularization of test suite

[Carl Worth]

On Wed, 13 Oct 2010 14:44:23 +0200, Michal Sojka <sojkam1 at fel.cvut.cz> wrote:
> here is my experience with test_begin_subtest and test_expect_equal
> functions in test-lib.sh. I see there two significant differences from
> the git's test_expect_success and similar functions.
> 
> First, when the test fails, you only see the differences in the real
> output and the expected output. If you use test_expect_success you also
> see the actual commands that were executed.

Yes. For me, I never found the printing of the exact commands as
something useful. We probably should fix our test suite to behave
consistently regardless of which interface is used.

> Second, when some command between test_begin_subtest and
> test_expect_success fails (e.g. because of segfault) you may not detect
> it. You can add "set -e" the test-lib.sh (see the patch bellow), but in
> this case when a test fails you would see
> 
>   FATAL: Unexpected exit with code 1
> 
> without any additional information.

What do you recommend here?

Frankly, I'm fine if the test cases are easy to read, (and I think they
are easier without having many commands chained by "&&" and concatenated
into a single string). Then if any test fails, (whether a segfault or
simply incorrect output), it shouldn't be hard to just open the test
case and manually inspect the commands.

But if you've got some idea for anything more useful, (some magic action
with trap for example?), I'd be glad to hear it.

-Carl

-- 
carl.d.worth at intel.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20101013/4aca2474/attachment.pgp>