[PATCH 2/2] python: annotate all calls into libnotmuch with types

Wed Dec 7 11:05:56 PST 2011

Quoting Justus Winter (2011-12-07 19:49:31)
>Quoting Justus Winter (2011-12-06 13:51:08)
>>I began running alot in gdb since I get segfaults within libnotmuch
>>from time to time and managed to get a stack trace pointing to
>>notmuch_database_begin_atomic, but I couldn't figure out what caused
>>it.

And another one:

Reading symbols from /usr/bin/python...Reading symbols from /usr/lib/debug/usr/bin/python2.7...done.
done.
[New LWP 15188]
[Thread debugging using libthread_db enabled]
Core was generated by `/usr/bin/python /home/teythoon/.local/bin/afew -v --tag --new'.
Program terminated with signal 6, Aborted.
#0  0x00007f72f2cce405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
	in ../nptl/sysdeps/unix/sysv/linux/raise.c
(gdb) bt
#0  0x00007f72f2cce405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007f72f2cd1680 in *__GI_abort () at abort.c:92
#2  0x00007f72f1380bfd in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007f72f137eda6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007f72f137edd3 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007f72f137eece in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007f72f16b792e in ChertTable::set_overwritten() const () from /usr/lib/libxapian.so.22
#7  0x00007f72f16ba7a6 in ChertTable::block_to_cursor(Cursor*, int, unsigned int) const () from /usr/lib/libxapian.so.22
#8  0x00007f72f16bcc35 in ChertTable::find(Cursor*) const () from /usr/lib/libxapian.so.22
#9  0x00007f72f1697431 in ChertCursor::find_entry(std::string const&) () from /usr/lib/libxapian.so.22
#10 0x00007f72f16c0858 in ?? () from /usr/lib/libxapian.so.22
#11 0x00007f72f16c0c69 in ?? () from /usr/lib/libxapian.so.22
#12 0x00007f72f16a649a in ?? () from /usr/lib/libxapian.so.22
#13 0x00007f72f1621001 in Xapian::Document::Internal::get_value(unsigned int) const () from /usr/lib/libxapian.so.22
#14 0x00007f72f162103c in Xapian::Document::get_value(unsigned int) const () from /usr/lib/libxapian.so.22
#15 0x00007f72f238ef23 in notmuch_message_get_header () from /home/teythoon/.local/lib/libnotmuch.so.2
#16 0x00007f72f25b7f70 in ffi_call_unix64 () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#17 0x00007f72f25b79eb in ffi_call () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#18 0x00007f72f25ab9c7 in _call_function_pointer (argcount=2, resmem=0x7fffc13a0310, restype=<optimized out>, atypes=<optimized out>, avalues=0x7fffc13a02f0, 
    pProc=0x7f72f238ee50 <notmuch_message_get_header>, flags=4353) at /home/packages/python/2.7/python2.7-2.7.2/Modules/_ctypes/callproc.c:827
#19 _ctypes_callproc (pProc=0x7f72f238ee50 <notmuch_message_get_header>, argtuple=0x0, flags=4353, argtypes=<optimized out>, restype=
    <_ctypes.PyCSimpleType at remote 0xd17c30>, checker=0x0) at /home/packages/python/2.7/python2.7-2.7.2/Modules/_ctypes/callproc.c:1174
#20 0x00007f72f25a3257 in PyCFuncPtr_call (self=<optimized out>, inargs=<optimized out>, kwds=0x0)
    at /home/packages/python/2.7/python2.7-2.7.2/Modules/_ctypes/_ctypes.c:3913
#21 0x000000000041d35a in PyObject_Call (func=<_FuncPtr(__name__='notmuch_message_get_header') at remote 0xe02530>, arg=<optimized out>, kw=<optimized out>)
    at ../Objects/abstract.c:2529
#22 0x00000000004b9b4e in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffc13a05f0, func=
    <_FuncPtr(__name__='notmuch_message_get_header') at remote 0xe02530>) at ../Python/ceval.c:4239
#23 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a05f0) at ../Python/ceval.c:4044
#24 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#25 0x00000000004b9d27 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffc13a0730, func=<function at remote 0xdfef50>)
    at ../Python/ceval.c:4107
#26 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a0730) at ../Python/ceval.c:4042
#27 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#28 0x00000000004bfc9d in PyEval_EvalCodeEx (co=0xcd28b0, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, 
    kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3253
#29 0x000000000044b36f in function_call (func=<function at remote 0xe03758>, arg=
    (<Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>,), kw=0x0) at ../Objects/funcobject.c:526
#30 0x000000000041d35a in PyObject_Call (func=<function at remote 0xe03758>, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2529
#31 0x0000000000432bdb in instancemethod_call (func=<function at remote 0xe03758>, arg=
    (<Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>,), kw=0x0) at ../Objects/classobject.c:2578
#32 0x000000000042668c in PyObject_Call (func=<instancemethod at remote 0x1192f00>, arg=<optimized out>, kw=0x0) at ../Objects/abstract.c:2529
#33 0x00000000004299fc in PyObject_CallFunctionObjArgs (callable=<instancemethod at remote 0x1192f00>) at ../Objects/abstract.c:2760
#34 0x0000000000466ebb in PyObject_Unicode (v=
    <Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>) at ../Objects/object.c:509
#35 0x0000000000494443 in unicode_new (type=0x853200, args=<optimized out>, kwds=<optimized out>) at ../Objects/unicodeobject.c:8722
#36 0x000000000047ef15 in type_call (type=0x853200, args=
    (<Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>,), kwds=0x0) at ../Objects/typeobject.c:721
#37 0x000000000041d35a in PyObject_Call (func=<type at remote 0x853200>, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2529
#38 0x00000000004b9b4e in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffc13a0e50, func=<type at remote 0x853200>) at ../Python/ceval.c:4239
#39 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a0e50) at ../Python/ceval.c:4044
#40 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#41 0x00000000004bfc9d in PyEval_EvalCodeEx (co=0xcd2830, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, 
    kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3253
#42 0x000000000044b36f in function_call (func=<function at remote 0xe036e0>, arg=
    (<Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>,), kw=0x0) at ../Objects/funcobject.c:526
#43 0x000000000041d35a in PyObject_Call (func=<function at remote 0xe036e0>, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2529
#44 0x0000000000432bdb in instancemethod_call (func=<function at remote 0xe036e0>, arg=
    (<Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>,), kw=0x0) at ../Objects/classobject.c:2578
#45 0x000000000041d35a in PyObject_Call (func=<instancemethod at remote 0xe710a0>, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2529
#46 0x00000000004b8cb6 in PyEval_CallObjectWithKeywords (func=<instancemethod at remote 0xe710a0>, arg=(), kw=<optimized out>) at ../Python/ceval.c:3890
#47 0x00000000004837c9 in slot_tp_str (self=
    <Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>) at ../Objects/typeobject.c:5341
#48 0x0000000000466848 in _PyObject_Str (v=
    <Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>) at ../Objects/object.c:430
#49 0x0000000000471ce9 in PyString_Format (format='Adding tags %s to %s', args=
    ('juggling', <Message(_parent=<Messages(_parent=<Query(sort=None, _db=<Database(_db=<LP_NotmuchDatabaseS at remote 0xdf8e60>) at remote 0xe6c9d0>, _query=<LP_NotmuchQueryS at remote 0xdf8f80>) at remote 0xe6ca50>, _msgs=<LP_NotmuchMessagesS at remote 0xe770e0>) at remote 0xe6c990>, _msg=<LP_NotmuchMessageS at remote 0xe77200>) at remote 0x11a1b10>)) at ../Objects/stringobject.c:4435
#50 0x00000000004bad8d in PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:1308
#51 0x00000000004bfc9d in PyEval_EvalCodeEx (co=0xbdf930, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, 
    kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3253
#52 0x00000000004b9a03 in fast_function (nk=<optimized out>, na=3, n=<optimized out>, pp_stack=0x7fffc13a17c0, func=<function at remote 0xe3b9b0>)
    at ../Python/ceval.c:4117
#53 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a17c0) at ../Python/ceval.c:4042
#54 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#55 0x00000000004b9d27 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffc13a1900, func=<function at remote 0xe65f50>)
    at ../Python/ceval.c:4107
#56 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a1900) at ../Python/ceval.c:4042
#57 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#58 0x00000000004b9d27 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, pp_stack=0x7fffc13a1a40, func=<function at remote 0xe3b8c0>)
    at ../Python/ceval.c:4107
#59 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a1a40) at ../Python/ceval.c:4042
#60 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#61 0x00000000004c0295 in PyEval_EvalCodeEx (co=0xe44730, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, 
    kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3253
#62 0x00000000004b9a03 in fast_function (nk=<optimized out>, na=2, n=<optimized out>, pp_stack=0x7fffc13a1c30, func=<function at remote 0xe70aa0>)
    at ../Python/ceval.c:4117
#63 call_function (oparg=<optimized out>, pp_stack=0x7fffc13a1c30) at ../Python/ceval.c:4042
#64 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:2666
#65 0x00000000004bfc9d in PyEval_EvalCodeEx (co=0xb4a8b0, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, 
    kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../Python/ceval.c:3253
#66 0x00000000004c0772 in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:667
#67 0x00000000004dfa22 in run_mod (mod=<optimized out>, filename=<optimized out>, globals=
    {'no_actions': 1, 'configured_filter_chain': [<SpamFilter(_remove_tags={}, _tag_blacklist=set([]), spam_tag='spam', _flush_tags=[], _tags_to_remove=[], _add_tags={}, _tags_to_add=[], db_path='/home/teythoon/Maildir') at remote 0xe6c6d0>, <ClassifyingFilter(_remove_tags={}, _tag_blacklist=set([]), _flush_tags=[], _tags_to_remove=[], _add_tags={'003f01ccb4fb$902a6430$b07f2c90$@gmx.de': set([u'juggling']), '20111203042836.GI27538 at rzssh1.informatik.uni-hamburg.de': set([u'juggling']), '4ED62052.4000109 at informatik.uni-hamburg.de': set([u'juggling']), 'A106C811-6B91-4E70-938A-403C22F24DB4 at mex21.net': set([u'juggling']), '4EDE321F.4040708 at ramdrive.org': set([u'juggling']), 'CAC2-jLG5wauWYVhFwxX8Rh1sbexx4OT2sks7EbZRAWn9=hPs3w at mail.gmail.com': set([u'juggling']), '4ED65C2C.9070805 at informatik.uni-hamburg.de': set([u'juggling']), '4EDCC624.3020509 at informatik.uni-hamburg.de': set([u'juggling']), 'D51F629D-9742-4F0A-807E-A7CF8102C9BD at robert-marquardt.com': set([u'juggling']), '005601ccb50a$91845110$b48cf330$@gmx.de': set(...(truncated), locals=
    {'no_actions': 1, 'configured_filter_chain': [<SpamFilter(_remove_tags={}, _tag_blacklist=set([]), spam_tag='spam', _flush_tags=[], _tags_to_remove=[], _add_tags={}, _tags_to_add=[], db_path='/home/teythoon/Maildir') at remote 0xe6c6d0>, <ClassifyingFilter(_remove_tags={}, _tag_blacklist=set([]), _flush_tags=[], _tags_to_remove=[], _add_tags={'003f01ccb4fb$902a6430$b07f2c90$@gmx.de': set([u'juggling']), '20111203042836.GI27538 at rzssh1.informatik.uni-hamburg.de': set([u'juggling']), '4ED62052.4000109 at informatik.uni-hamburg.de': set([u'juggling']), 'A106C811-6B91-4E70-938A-403C22F24DB4 at mex21.net': set([u'juggling']), '4EDE321F.4040708 at ramdrive.org': set([u'juggling']), 'CAC2-jLG5wauWYVhFwxX8Rh1sbexx4OT2sks7EbZRAWn9=hPs3w at mail.gmail.com': set([u'juggling']), '4ED65C2C.9070805 at informatik.uni-hamburg.de': set([u'juggling']), '4EDCC624.3020509 at informatik.uni-hamburg.de': set([u'juggling']), 'D51F629D-9742-4F0A-807E-A7CF8102C9BD at robert-marquardt.com': set([u'juggling']), '005601ccb50a$91845110$b48cf330$@gmx.de': set(...(truncated), flags=<optimized out>, arena=<optimized out>) at ../Python/pythonrun.c:1346
#68 0x00000000004e05b4 in PyRun_FileExFlags (fp=0xb99c10, filename=0x7fffc13a3346 "/home/teythoon/.local/bin/afew", start=<optimized out>, globals=
    {'no_actions': 1, 'configured_filter_chain': [<SpamFilter(_remove_tags={}, _tag_blacklist=set([]), spam_tag='spam', _flush_tags=[], _tags_to_remove=[], _add_tags={}, _tags_to_add=[], db_path='/home/teythoon/Maildir') at remote 0xe6c6d0>, <ClassifyingFilter(_remove_tags={}, _tag_blacklist=set([]), _flush_tags=[], _tags_to_remove=[], _add_tags={'003f01ccb4fb$902a6430$b07f2c90$@gmx.de': set([u'juggling']), '20111203042836.GI27538 at rzssh1.informatik.uni-hamburg.de': set([u'juggling']), '4ED62052.4000109 at informatik.uni-hamburg.de': set([u'juggling']), 'A106C811-6B91-4E70-938A-403C22F24DB4 at mex21.net': set([u'juggling']), '4EDE321F.4040708 at ramdrive.org': set([u'juggling']), 'CAC2-jLG5wauWYVhFwxX8Rh1sbexx4OT2sks7EbZRAWn9=hPs3w at mail.gmail.com': set([u'juggling']), '4ED65C2C.9070805 at informatik.uni-hamburg.de': set([u'juggling']), '4EDCC624.3020509 at informatik.uni-hamburg.de': set([u'juggling']), 'D51F629D-9742-4F0A-807E-A7CF8102C9BD at robert-marquardt.com': set([u'juggling']), '005601ccb50a$91845110$b48cf330$@gmx.de': set(...(truncated), locals=
    {'no_actions': 1, 'configured_filter_chain': [<SpamFilter(_remove_tags={}, _tag_blacklist=set([]), spam_tag='spam', _flush_tags=[], _tags_to_remove=[], _add_tags={}, _tags_to_add=[], db_path='/home/teythoon/Maildir') at remote 0xe6c6d0>, <ClassifyingFilter(_remove_tags={}, _tag_blacklist=set([]), _flush_tags=[], _tags_to_remove=[], _add_tags={'003f01ccb4fb$902a6430$b07f2c90$@gmx.de': set([u'juggling']), '20111203042836.GI27538 at rzssh1.informatik.uni-hamburg.de': set([u'juggling']), '4ED62052.4000109 at informatik.uni-hamburg.de': set([u'juggling']), 'A106C811-6B91-4E70-938A-403C22F24DB4 at mex21.net': set([u'juggling']), '4EDE321F.4040708 at ramdrive.org': set([u'juggling']), 'CAC2-jLG5wauWYVhFwxX8Rh1sbexx4OT2sks7EbZRAWn9=hPs3w at mail.gmail.com': set([u'juggling']), '4ED65C2C.9070805 at informatik.uni-hamburg.de': set([u'juggling']), '4EDCC624.3020509 at informatik.uni-hamburg.de': set([u'juggling']), 'D51F629D-9742-4F0A-807E-A7CF8102C9BD at robert-marquardt.com': set([u'juggling']), '005601ccb50a$91845110$b48cf330$@gmx.de': set(...(truncated), closeit=1, flags=0x7fffc13a1f60) at ../Python/pythonrun.c:1332
#69 0x00000000004e10be in PyRun_SimpleFileExFlags (fp=0xb99c10, filename=<optimized out>, closeit=1, flags=0x7fffc13a1f60) at ../Python/pythonrun.c:936
#70 0x00000000004f10fd in Py_Main (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:599
#71 0x00007f72f2cbaead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffc13a2078) at libc-start.c:228
#72 0x000000000041f199 in _start ()
(gdb) q

with stderr saying:

terminate called after throwing an instance of 'Xapian::DatabaseModifiedError'
Aborted (core dumped)

This is an uncought c++ exception, right? If so I think it has to be
cought somewhere in libnotmuch and turned into the appropriate error
code (hm, there is only the generic XAPIAN_EXCEPTION, I thought there
was a way to indicate that the db has been modified?).

Justus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: .signature
Type: application/octet-stream
Size: 17 bytes
Desc: not available
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111207/37160f77/attachment.obj>