Bug: counting messages twice after excluding tags yields different results

[Lucas Hoffmann]

Thank you Franz, sadly your reply did not convince me:

Quoting Franz Fellner (2016-08-31 11:21:18)
> Your problem: the example sucks ;)

No, I must object :(

> If the query searches for a tag you also have in exclude_tags (in your
> case: spam) the exclude gets ignored.

Is that documented?  Because it is not what I would expect.  You might
be right as I see this when counting messages for the first time after
excluding the tag.  But from the second run on I get a count of 0 which
is not explained by your comment.

> Change your query to just "is:inbox" and magically "spam" really gets
> excluded.

I know (and mentioned below, step 2) that this bug only appears when the
tag is also mentioned in the query.  (For the record it also happens
when the query is "is:inbox and not is:spam".)
I originally found the bug when writing a program that reads the default
config file and uses its search.exclude_tags value.  Additionally the
user might set a query and exclusion tags from the command line.
So even if I can fix the example the problem persists.  So I would
rather have the notmuch library be correctly documented or behave in an
sane way than manually parse and fix queries and exclude tags.  Please
note that my example is just that: an example to demonstrate the
behaviour.  So I don't think the correct answer to my report is "fix
your query/example".

> However it is better to create fresh query objects for each new query.
> I remember there are operations on query objects that are destructive.
> That's why users of the notmuch API usually create seperate queriy
> objects for counting messages/threads and getting the results.

Again: Is that documented?  Strangely enough counting is non destructive
before calling notmuch_query_add_tag_exclude.  And if any of these
operations is destructive I would expect any further calls to the same
function to return an error.  I updated the example c program to check
for this but did not see any error.  The new version is attached.

Hopefully I was able to clarify my concern.

Best wishes,
Lucas

> On Wed, 31 Aug 2016 10:23:59 +0200, Lucas <luc.lists at gmail.com> wrote:
> > Dear list members,
> >
> > I think I found a bug or at least undocumented behaviour in the notmuch
> > library.  I would like to report this here.  Originally I found the bug
> > in the python library but I attached a c program that shows the same
> > behaviour.  I am running notmuch version 0.22.1 from the Arch Linux
> > repositories.
> >
> > The setup:
> > 1. chose a query string, e.g. "is:inbox or is:spam"
> > 2. chose a tag to exclude that is matched by the query, e.g. "spam"
> > 3. open the database
> > 4. create a query
> > 5. check the message or thread count any number of times
> > 6. exclude the tag from the query
> > 7. check the message or thread count any number of times
> >
> > The result:
> > - In step 5 the result stays the same if I repeatedly call
> >   notmuch_query_count_messages_st or query.count_messages.
> > - In step 7 the count is different between the first call and all
> >   subsequent calls.  But neither seems correct to me.  I always get the
> >   same number as in step 5 for the first call and 0 afterwards.
> >
> > Expected result:
> > - subsequent calls to notmuch_query_count_messages_st or
> >   query.count_messages should yield the same result
> > - the exclusion should change the count to the actual amount (for
> >   "is:inbox or is:spam" I get 891 and for a plain "is:inbox" I get 58,
> >   which never shows up in step 7)
> >
> > Attached you can find a python and a c program that exhibit this
> > behaviour.  Please compile the c program with
> >
> > cc -DDB_PATH=\"/path/to/your/mail\" -lnotmuch test.c
> >
> > My question:
> > Is this documented somewhere?  Is it actually a bug or is it already
> > fixed in a newer version?
> >
> > Thank you for developing notmuch!
> >
> > Lucas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: example.c
Type: text/x-c
Size: 1529 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20160831/8bffa9d0/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: signature
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20160831/8bffa9d0/attachment.sig>