test suite: FIXED messages are misordered with tests

Daniel Kahn Gillmor dkg at fifthhorseman.net
Tue May 12 15:03:17 PDT 2020


I'm debugging/diagnosing/trying to clean up some "FIXED" known-broken
tests right now.

Sometimes, depending on circumstances i can't predict (race
conditions?), I see funny output like:

~~~
Use "make V=1" to see the details for passing and known broken tests.
INFO: using 2m timeout for tests
INFO: running tests with moreutils parallel
 FIXED  verify signed PKCS#7 subject (onepart-signed) signer User ID

T356-protected-headers: Testing Message decryption with protected headers
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc)
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc) signer User ID
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp)
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp) signer User ID
 BROKEN confirm encryption-protected PKCS#7 subject (enc+legacy-disp)
~~~


Clearly, that FIXED should come *after* the "T356-protected-headers:"
separator.

This is a minor bug, i suppose, but i confess i don't understand the
maze of shell functions in test-lib.sh well enough to see why this is
happening, let alone to fix it.

Anyone interested in fixing it should be able to do so by marking a good
test "known broken" and then re-running the test suite.  The above
output is taken from:

    make -j4 check NOTMUCH_TESTS=T356-protected-headers.sh

Sorry to send a bug report with no fixes!

           --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20200512/17982571/attachment.sig>


More information about the notmuch mailing list