segfault using python bindings

[David Čepelík]

Hello Notmuch devs,

I'm facing an issue trying to use the Python bindings. This trivial
piece of code segfaults:

    import notmuch

    database = notmuch.Database()
    threads = database.create_query('tag:inbox and not tag:killed').search_threads()
    
    for t in threads:
        print("Thread:", t)
        msgs = t.get_toplevel_messages()
        for m in msgs:
            print("Message:", m)
        msgs = t.get_toplevel_messages()
        next(msgs)

The problem is triggered by the call to next. Doing this instead works:

    database = notmuch.Database()

    threads = database.create_query('tag:inbox and not tag:killed').search_threads()
    for t in threads:
        print("Thread:", t)
        msgs = t.get_toplevel_messages()
        for m in msgs:
            print("Message:", m)

    threads = database.create_query('tag:inbox and not tag:killed').search_threads()
    for t in threads:
        print("Thread:", t)
        msgs = t.get_toplevel_messages()
        for m in msgs:
            print("Message:", m)

It seems that the problem is caused by calling get_toplevel_messages
twice on the same Threads object.

I've been able to narrow the problem down using gdb. The first few
frames of the stack-trace are:

    #0  0x000055555557da90 in  ()
    #1  0x00007ffff665db5a in Xapian::Document::Internal::get_value[abi:cxx11](unsigned int) const () at /usr/lib/libxapian.so.30
    #2  0x00007ffff665db91 in Xapian::Document::get_value[abi:cxx11](unsigned int) const () at /usr/lib/libxapian.so.30
    #3  0x00007ffff6e3165a in notmuch_message_get_header(notmuch_message_t*, char const*)
        (message=0x5555556e6920, header=0x7ffff7195f78 "from") at lib/message.cc:549
    #4  0x00007ffff6efb1c8 in ffi_call_unix64 () at /usr/lib/libffi.so.6

The () seems to denote C++'s tuple class:

    (gdb) frame 0
    #0  0x000055555557da90 in ?? ()
    (gdb) lis
    1	// <tuple> -*- C++ -*-
    2	
    3	// Copyright (C) 2007-2018 Free Software Foundation, Inc.
    4	//
    5	// This file is part of the GNU ISO C++ Library.  This library is free
    6	// software; you can redistribute it and/or modify it under the
    7	// terms of the GNU General Public License as published by the
    8	// Free Software Foundation; either version 3, or (at your option)
    9	// any later version.
    10	

Searching further, I've arrived at the following piece of Xapian code
(xapian-core/backends/documentinternal.h):

    390     std::string get_value(Xapian::valueno slot) const {
    391         if (values) {
    392             auto i = values->find(slot);
    393             if (i != values->end())
    394                 return i->second;
    395             return std::string();
    396         }
    397
    398         return fetch_value(slot);
    399     }

Since the invalid dereference indicates a tuple, I suspect the crash
stems from the use `second' on line 394. (The (->) dereference likely
does not cause the crash since otherwise we wouldn't arrive at the
tuple.)

When I use alot with this version of notmuch/python bindings, it works
just fine, but I suspect that's because alot wraps all the provided
objects to avoid this sort of bugs (and allow for repeated iteration
over Threads objects, etc), and hence avoid calling get_toplevel_messages()
twice.

Does the problem lie with my fundamental misunderstanding of how the
bindings work, or is this a bug?

Linux x1 4.18.16-arch1-1-ARCH #1 SMP PREEMPT Sat Oct 20 22:06:45 UTC 2018 x86_64 GNU/Linux
Python 3.7.1
built from upstream @ 7f726c6 (AUR: notmuch-git 3:0.28.2.7.g7f726c6e-1)


                                            Regards, David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: signature
URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20181109/f91a0591/attachment.sig>