xapian exceptions not caught in python bindings?

[Patrick Totzke]

Hi all,

I know this issue is not easy to fix and that it's partly a problem
with xapians bad error handling. I just want to state another problem we run
into with xapians exceptions in combination with the python bindings.
If you run the following snippet, you notice that not only do we get
xapian-garbage on stderr but we don't really get any exceptions at the
position where it would make sense:

--------------------------------
from notmuch import Database
bad_querystring = "test AND"

# This should raise a NotmuchError and query should be NoneType
query = Database().create_query(bad_querystring)

# this prints to stderr but returns a Threads object.
threads = query.search_threads()

# this finally raises a NotmuchError(STATUS.NOT_INITIALIZED)
for t in threads: print t
--------------------------------

Although the querystring is invalid syntactically, the query object gets 
instantiated and can of course not behave as it should later on.
From a user point of view it is really quite strange that only later
(if one iterates over threads) an exception is raised.

One solution that comes to mind here is simply bending sys.sdterr to something internal,
interpreting its output and raising the correct exceptions accordingly.
The downside is, this would have to happen every time the library gets called.
Also, a fix like this would be python specific, so everybody else would still get
garbage printed to stderr occasionally.

Thoughts anyone?
/p



On Sun, Jun 26, 2011 at 09:27:33PM +0100, Patrick Totzke wrote:
> Hi all!
> It's me again, with some strange python behaviour :/
> 
> When I iterate over threads or messages only partially,
> then make changes to the index and continue the iteration,
> then I don't necessarily get NotmuchError's but the underlying
> libnotmuch seems to call terminate. I get the following two lines on
> stderr:
>   terminate called after throwing an instance of 'Xapian::DatabaseModifiedError'
>   Aborted
> then python aborts without an exception.
> This happens on current master with current python bindings. To reproduce
> use this to iterate over messages:
> 
> ------------------%<-------------------
> #it_read.py
> from notmuch import Query,Database
> msgs = Database().create_query('*').search_messages()
> 
> i=0
> for m in msgs:
>     i=i+1
>     if i%50==0:
>         raw_input()
>     readit = str(m)
> ------------------>%-------------------
> 
> start it, then use this to change the index:
> 
> ------------------%<-------------------
> #it_write.py 
> from notmuch import Query,Database
> db = Database(mode = Database.MODE.READ_WRITE)
> msgs = Query(db,'*').search_messages()
> 
> i=0
> for m in msgs:
>     i=i+1
>     if i%20==0:
>         m.add_tag('REMOVEME')
> ------------------>%-------------------
> 
> you have to alternate a bit between hitting enter in the
> read code and rerunning the write code.
> Read will fail at some point.
> 
> The strange thing is that its not consistent:
> I stumbled upon this while reading a list of threads from notmuch.threads on
> demand. If reading from the iterator catches a NotmuchError, i create a new one
> and update my list threads read so far. This does work, but: on every
> NotmuchError, I get this on stderr:
>  A Xapian exception occurred performing query: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
>  Query string was: thread:0000000000001876
> 
> After a couple of resets this does not happen again but the next time i read
> from an outdated index, the whole UI terminates with the "Aborted" message
> stated above.
> 
> To sum up, I think there are two bad things happening here:
> 1) should libnotmuch really print to stderr?
> 2) the python bindings are either not relaying all exceptions or libnotmuch is not relaying 
> all xapian exceptions in order for the bindings to intercept them
> 
> cheers,
> /p


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110717/7003bfa1/attachment.pgp>