notmuch ignoring alot of emails

[Alvaro Herrera]

On 2019-Mar-23, Alexei Gilchrist wrote:

> When I run notmuch I get a bunch (hundreds) of emails that are ignored with:
> 
> Note: Ignoring non-mail file: ...
> 
> The files are valid maildir files but have a paragraph somewhere in the body
> where someone has written "From ".

Yeah, that happens too when you attach patches generated with git
format-patch as plain text; this is extremely common in the
pgsql-hackers at lists.postgresql.org mailing list (you can download an
mbox from there for any month, convert it to a maildir, and give the
resulting maildir to notmuch -- you'll likely find a few dozen emails
that fail parsing).  This is a very annoying problem for me, see
201901181607.4rba4c5uyimv at alvherre.pgsql in this list earlier this year.

I worked around it by patching _notmuch_message_file_parse in
lib/message-file.c to set is_mbox = false unconditionally; but that's
not a real solution (and hence I didn't post as a patch here), and it
explodes real good if you have an actual mbox in the directory where the
mail is (since after the hack it won't skip it anymore).

I think a real solution is to parse the message header, look for the
Content-Length, and determine mbox-ness by looking for "From" only past
that many bytes; that seems to match what other mail parsing tools do.
However, I haven't gotten around to doing that.

-- 
Álvaro Herrera                            39°49'30"S 73°17'W
"La experiencia nos dice que el hombre peló millones de veces las patatas,
pero era forzoso admitir la posibilidad de que en un caso entre millones,
las patatas pelarían al hombre" (Ijon Tichy)