segfault using python bindings

[Dirk Van Haerenborgh]

Hi,

I've encountered something very similar just today. But not with python,
but my rust bindings. I could very easily reproduce it using:

for (messages = notmuch_thread_get_messages (thread);
notmuch_messages_valid (messages);
notmuch_messages_move_to_next (messages))
{
notmuch_message_t *message = notmuch_messages_get (messages);
const char *mid = notmuch_message_get_message_id(message);
fprintf(stdout, "Message: %s\n", mid);
notmuch_message_destroy(message);
}

for (messages = notmuch_thread_get_messages (thread);
notmuch_messages_valid (messages);
notmuch_messages_move_to_next (messages))
{
notmuch_message_t *message = notmuch_messages_get (messages);
const char *mid = notmuch_message_get_message_id(message);
fprintf(stdout, "Message: %s\n", mid);
notmuch_message_destroy(message);
}

This is not a typo. I deliberately duplicated that bit.
The second time it runs that loop, it will always segfault, unless you
omit the first 'notmuch_message_destroy' call.

Given your example, I suspect the Python3 bindings to do something very
similar. I would think that this is the correct way to use the API?

Kind regards,
-Dirk


On Fri, 09 Nov 2018 15:49:13 +0100, David Čepelík wrote:

> Hello Notmuch devs,
>
> I'm facing an issue trying to use the Python bindings. This trivial
> piece of code segfaults:
>
> import notmuch
>
> database = notmuch.Database()
> threads = database.create_query('tag:inbox and not
> tag:killed').search_threads()
>
> for t in threads:
> print("Thread:", t)
> msgs = t.get_toplevel_messages()
> for m in msgs:
> print("Message:", m)
> msgs = t.get_toplevel_messages()
> next(msgs)
>
> The problem is triggered by the call to next. Doing this instead works:
>
> database = notmuch.Database()
>
> threads = database.create_query('tag:inbox and not
> tag:killed').search_threads()
> for t in threads:
> print("Thread:", t)
> msgs = t.get_toplevel_messages()
> for m in msgs:
> print("Message:", m)
>
> threads = database.create_query('tag:inbox and not
> tag:killed').search_threads()
> for t in threads:
> print("Thread:", t)
> msgs = t.get_toplevel_messages()
> for m in msgs:
> print("Message:", m)
>
> It seems that the problem is caused by calling get_toplevel_messages
> twice on the same Threads object.
>
> I've been able to narrow the problem down using gdb. The first few
> frames of the stack-trace are:
>
> #0 0x000055555557da90 in ()
> #1 0x00007ffff665db5a in
> Xapian::Document::Internal::get_value[abi:cxx11](unsigned int) const
> () at /usr/lib/libxapian.so.30 #2 0x00007ffff665db91 in
> Xapian::Document::get_value[abi:cxx11](unsigned int) const () at
> /usr/lib/libxapian.so.30 #3 0x00007ffff6e3165a in
> notmuch_message_get_header(notmuch_message_t*, char const*)
> (message=0x5555556e6920, header=0x7ffff7195f78 "from") at
> lib/message.cc:549
> #4 0x00007ffff6efb1c8 in ffi_call_unix64 () at /usr/lib/libffi.so.6
>
> The () seems to denote C++'s tuple class:
>
> (gdb) frame 0 #0 0x000055555557da90 in ?? ()
> (gdb) lis 1 // <tuple> -*- C++ -*-
> 2
> 3 // Copyright (C) 2007-2018 Free Software Foundation, Inc.
> 4 //
> 5 // This file is part of the GNU ISO C++ Library. This library is
> free 6 // software; you can redistribute it and/or modify it under
> the 7 // terms of the GNU General Public License as published by
the
> 8 // Free Software Foundation; either version 3, or (at your option)
> 9 // any later version.
> 10
>
> Searching further, I've arrived at the following piece of Xapian code
> (xapian-core/backends/documentinternal.h):
>
> 390 std::string get_value(Xapian::valueno slot) const {
> 391 if (values) {
> 392 auto i = values->find(slot);
> 393 if (i != values->end())
> 394 return i->second;
> 395 return std::string();
> 396 }
> 397 398 return fetch_value(slot);
> 399 }
>
> Since the invalid dereference indicates a tuple, I suspect the crash
> stems from the use `second' on line 394. (The (->) dereference likely
> does not cause the crash since otherwise we wouldn't arrive at the
> tuple.)
>
> When I use alot with this version of notmuch/python bindings, it works
> just fine, but I suspect that's because alot wraps all the provided
> objects to avoid this sort of bugs (and allow for repeated iteration
> over Threads objects, etc), and hence avoid calling
> get_toplevel_messages()
> twice.
>
> Does the problem lie with my fundamental misunderstanding of how the
> bindings work, or is this a bug?
>
> Linux x1 4.18.16-arch1-1-ARCH #1 SMP PREEMPT Sat Oct 20 22:06:45 UTC
> 2018 x86_64 GNU/Linux Python 3.7.1 built from upstream @ 7f726c6 (AUR:
> notmuch-git 3:0.28.2.7.g7f726c6e-1)
>
>
> Regards, David