Rethinking *_destroy()

[Ben Gamari]

As many might have noticed, there was recently a bit of a discussion on
this list concerning the state of memory management in libnotmuch,
especially regarding some classes of garbage collectors.

To summarize (someone correct me if I get something wrong),

 * Garbage collectors don't always dispose of objects in the order that
   they become unreachable (due to the fact that in the existence of
   cycles, this order is not well-defined).

 * Notmuch emulates a C-style free() function (which we call
   *_destroy()) on top of talloc

 * Calling *_destroy() on an object (e.g. Query) will also cause its
   children (e.g. Messages) to be freed

 * Calling *_destroy() on an object which has already been freed
   (not surprisingly) causes talloc to abort

Overall, this means that languages with cyclical garbage collectors
(Python, Haskell, and I'm sure others) can not bind libnotmuch
correctly. In the case of my work on the Haskell binding, I've found
that very often Query objects are released before Messages, quickly
causing talloc to abort. 

The solution as suggested my several people is for notmuch to
expose its reference counting mechanism (which even the problems with
*_destroy() notwithstanding, seems like a more natural means of memory
management, IMHO).

I can see at least two ways of doing this,

  1) Acknowledging that we use talloc and allowing users to use
     talloc_ref and talloc_unlink directly

  2) Wrapping talloc by adding a *_ref() and *_unref() to each object

Additionally, we need to decide to what extent we want to break the
libnotmuch API. While strictly speaking we could keep *_destroy() around
without breaking existing code, this will mean we will have two ways of
freeing an object. Perhaps a scheduled deprecation in a release or two
is in order here?

Anyways, I strongly believe that one of the above actions should be
taken as the current state of affairs is unacceptable for binding
writers.

Any and all feedback is desired.

Cheers,

- Ben