[PATCH 2/2] python: annotate all calls into libnotmuch with types

[Justus Winter]

Hi :)

Quoting David Bremner (2014-01-24 14:17:27)
> Justus Winter <4winter at informatik.uni-hamburg.de> writes:
> 
> > Quoting Justus Winter (2011-12-07 19:49:31)
> 
> > And another one:
> [stack trace snipped]
> > #69 0x00000000004e10be in PyRun_SimpleFileExFlags (fp=0xb99c10, filename=<optimized out>, closeit=1, flags=0x7fffc13a1f60) at ../Python/pythonrun.c:936
> > #70 0x00000000004f10fd in Py_Main (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:599
> > #71 0x00007f72f2cbaead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, 
> >     rtld_fini=<optimized out>, stack_end=0x7fffc13a2078) at libc-start.c:228
> > #72 0x000000000041f199 in _start ()
> > (gdb) q
> >
> > with stderr saying:
> >
> > terminate called after throwing an instance of 'Xapian::DatabaseModifiedError'
> > Aborted (core dumped)
> >
> > This is an uncought c++ exception, right? If so I think it has to be
> > cought somewhere in libnotmuch and turned into the appropriate error
> > code (hm, there is only the generic XAPIAN_EXCEPTION, I thought there
> > was a way to indicate that the db has been modified?).
> 
> Can you still reproduce this bug?  If so, a small test case (python is
> fine, but ideally not requiring afew) would be helpful.

I'm sorry, I've no idea how to reproduce this. I have, however, some
more test cases that crash libnotmuch, like this one:

~~~ snap ~~~
import os
import notmuch

db_path = os.path.expanduser('~/Maildir')

db = notmuch.Database(db_path, mode=notmuch.Database.MODE.READ_WRITE)
directory = db.get_directory(os.path.join(db_path, 'cur'))
db._destroy(db._db)

try:
    directory.get_child_files()
except notmuch.errors.NotInitializedError as e:
    print('yeah')
else:
    assert False, "Hey, where's my exception?"
~~~ snap ~~~

% python destroy_minimal.py
[2]    3907 abort (core dumped)  python destroy_minimal.py

Program terminated with signal 6, Aborted.
#0  0x00007f996ad021d5 in __GI_raise (sig=sig at entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007f996ad021d5 in __GI_raise (sig=sig at entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007f996ad05388 in __GI_abort () at abort.c:90
#2  0x00007f99699d4486 in talloc_abort (reason=0x7f99699db648 "Bad talloc magic value - access after free") at ../talloc.c:317
#3  0x00007f99699d8097 in talloc_abort_access_after_free () at ../talloc.c:336
#4  talloc_chunk_from_ptr (ptr=0x23e6f20) at ../talloc.c:357
#5  talloc_chunk_from_ptr (ptr=0x23e6f20) at ../talloc.c:2064
#6  __talloc (size=17, context=0x23e6f20) at ../talloc.c:555
#7  talloc_vasprintf (t=<optimized out>, fmt=0x7f996a3a56ef "%s%u:", ap=0x7fffb612bff8) at ../talloc.c:2079
#8  0x00007f99699d8167 in talloc_asprintf (t=<optimized out>, fmt=<optimized out>) at ../talloc.c:2101
#9  0x00007f996a39c02e in notmuch_directory_get_child_files () from /home/teythoon/.local/lib/libnotmuch.so.3
#10 0x00007f996a5b1cfc in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#11 0x00007f996a5b162c in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi.so.6
#12 0x00007f996a7c86d0 in _ctypes_callproc () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#13 0x00007f996a7ca08e in PyCFuncPtr_call.3149.2622 () from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#14 0x000000000055f47a in PyEval_EvalFrameEx () at ../Objects/abstract.c:2529
#15 0x000000000055f7ba in PyEval_EvalFrameEx () at ../Python/ceval.c:4107
#16 0x0000000000566bfb in PyEval_EvalCode () at ../Python/ceval.c:3253
#17 0x0000000000469499 in run_mod.42569 () at ../Python/pythonrun.c:1370
#18 0x0000000000469819 in PyRun_FileExFlags () at ../Python/pythonrun.c:1356
#19 0x0000000000469d52 in PyRun_SimpleFileExFlags () at ../Python/pythonrun.c:948
#20 0x000000000046b65f in Py_Main () at ../Modules/main.c:640
#21 0x00007f996acee995 in __libc_start_main (main=0x46b71d <main>, argc=2, ubp_av=0x7fffb612ca58, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, 
    stack_end=0x7fffb612ca48) at libc-start.c:276
#22 0x0000000000573f2e in _start ()

This is a bit contrived b/c I'm destroying the db object by
hand. Never the less libnotmuch calls abort, and there is no way to
contain something like this in the python bindings.

I've seen your recent proposal to improve the error reporting for
libnotmuch functions. I think it's awesome, albeit a little late.

I've no idea how other frontends deal with libnotmuch, but alot has
moved libnotmuch to it's own process to contain any crashes (or
libnotmuch calling exit(3)), and to prevent libnotmuch from writing to
stderr destroying alots curses gui. I've always felt like libnotmuch
users were second-class citizens, the primary target of libnotmuch
being the notmuch binary.

Justus