folder and path completely broken in HEAD?

[Mark Walters]

On Mon, 05 May 2014, Jani Nikula <jani at nikula.org> wrote:
> Hi Carl -
>
> On Tue, 06 May 2014, Carl Worth <cworth at cworth.org> wrote:
>> dm-list-email-notmuch at scs.stanford.edu writes:
>>> However, currently it seems strange that there are *two* different
>>> search terms (folder and path), and that neither one lets you search for
>>> a portion of your folder name.
>>
>> For what it's worth, I totally agree. I'm guilty as far as sitting out
>> of the detailed design discussions, (I don't use any sort of
>> folder-based searching, so I don't care too much). I was aware of the
>> problems of the original "folder:" code I wrote, and I was happy to hear
>> that people were addressing those problems.
>>
>> But it's terribly strange to me that notmuch now has two different
>> search terms that overlap so much in functionality. I know that I will
>> never trust myself to be able to distinguish/describe the folder:
>> vs. path: semantics without consulting the documentation. And that's
>> discouraging.
>
> The discussions about this were lengthy and tedious, and I was glad we
> eventually reached some consensus about what we wanted. It's always
> disappointing to find out one hasn't found the solution to satisfy
> everyone, but in the end I think I'm happier we were able to reach a
> decision, do something about the real issues people were facing with
> folder: and move on, rather than just grind to a halt. I think we were
> pretty close to everyone just dropping the ball and letting the folder:
> prefix be, warts and all.

I would just like to second what Jani said. There was a lot of
discussion and, at the time, the outcome covered all use cases that
anyone showed any sign of wanting. And these included things like
distinguishing between messages in cur or new or the toplevel of a maildir,
wanting to search all subdirectories of a particular path (if eg the
main mail folder is split into 256 subsirectories 00..ff).

> The idea of path: is that it's the exact filesystem directory, relative
> from the maildir root, with an rsync like ** syntax for recursive
> matching. Turns out people want folder: to hide maildir implementation
> details like cur and new. These are not compatible, or you need to add a
> syntax that's not easy or discoverable.
>
> path: is now pretty much complete, and allows one to do robust scripting
> that won't break in surprising ways. folder: is something we can still
> add new functionality into, for example fancier interpretations of
> maildir++, or anchoring if we ever get the custom query parser.
>
>> I think the original "folder:" shortcomings could have been addressed
>> without adding two terms, and also without losing some functionality,
>> (as shown in David's use case).
>>
>> I would have liked to have seen some explicit syntax for anchoring the
>> beginning and end of the directory name, (which could have then been
>> re-used for anchoring subject: or even some future header: prefix).
>
> As I understood it, that would have required writing a custom query
> parser, a significant effort. At least nobody came up with a scheme to
> do the anchoring without the parser while addressing the other issues
> with the old folder: prefix.
>
>> I've always thought that the "cur" and "new" directories were somewhere
>> on the spectrum between pointless and annoying. The idea with the
>> original "folder:" indexing was to implicitly ignore these, (when both
>> existed).

I think many of us would agree, but there were users who wanted to
distinguish new and cur, and in at least one case, the toplevel as well.


>>
>> It seems that the new "folder:" continues this idea, while the new
>> "path:" does not.
>
> Correct.
>
>> Meanwhile, the new "folder:" anchors the search to the beginning of
>> the directory, while "path:" does not.
>
> Incorrect (or I don't understand you).
>
>> And finally, "path:" adds a magic syntax to do hierarchical matching
>> while "folder:" does not.
>
> Correct.
>
>> That's an odd hodgepodge of non-orthogonal distinction in
>> functionality.

>
> I'm sorry to hear you think that way. One is verbatim filesystem path,
> the other hides mail store folder implementation details as a
> convenience.

I think it is unfortunate that we need two variants, but I think they do
do different things. Also, I think any single user will find one matches
their setup and use that one essentially exclusively: if everything is
in maildirs then you probably want folder:, if you want exact control
then use path:.

Indeed, it may be that a third option of roughly a maildir++: search term
might solve David's use case.

Best wishes

Mark