Notmuch DB Problems

[Jani Nikula]

On Tue, 04 Sep 2018, mueen at nawaz.org wrote:
> Hi,
>
> A few days ago I noticed notmuch new was no longer working (I have it
> as a cron job so it took a while to figure it out).
>
> It just freezes. I do have a Python hook, and it was freezing on the
> line that opens the database.
>
> I tried a notmuch dump. Same problem - freezes
>
> Based on some earlier threads, I tried a notmuch compact. Same problem
> - freezes.
>
> All these freezes seem to use no memory/CPU. 
>
> Interestingly, queries work fine - from both the command line and the
> Emacs interface. So I can read old stuff just fine. But all the
> commands above cause a freeze. 
>
> Currently using notmuch-0.24.2. I tried notmuch-0.27 - same problem.
>
> Results of a xapian check:
>
> docdata:
> blocksize=8K items=6 firstunused=3 revision=6442 levels=0 root=0
> B-tree checked okay
> docdata table structure checked OK
>
> termlist:
> blocksize=8K items=178562 firstunused=53441 revision=6442 levels=2
> root=46086
> /usr/bin/xapian-check: DatabaseError: 1 unused block(s) missing from
> the free list, first is 0
>
> What are my options? Unfortunately the last dump I have is many months
> old, so I'm a bit wary of deleting the database and rebuilding. Given
> that the show and search commands work, I was wondering if I can write
> a script to get all the message/thread ID's for all the tags and store
> them, and then rebuild the database and use that stored information to
> retag all my messages (all without using the dump command)?

It might be interesting to see an strace log to possibly get an idea
where it gets stuck.

Is the filesystem writable and working okay?

If search and show work, I'm guessing it gets stuck in trying to open
the database writable. One hackish idea is to patch notmuch dump to open
the database in read-only mode, and dump the tags. See below. The dump
command opens the database writable to prevent changes while
dumping. (Arguably this could be a command line option for cases like
yours.)

BR,
Jani.

diff --git a/notmuch-dump.c b/notmuch-dump.c
index ef2f02dfeb5c..d06dbcf50224 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -364,7 +364,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])
     int ret;
 
     if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
+			       NOTMUCH_DATABASE_MODE_READ_ONLY, &notmuch))
 	return EXIT_FAILURE;
 
     notmuch_exit_if_unmatched_db_uuid (notmuch);