[RFC2 Patch 5/5] lib: iterator API for message properties

[Tomi Ollila]

On Wed, Jun 01 2016, Daniel Kahn Gillmor <dkg at fifthhorseman.net> wrote:

> On Tue 2016-05-31 21:12:21 -0400, David Bremner <david at tethera.net> wrote:
>> I was thinking a bit about how to dump/restore these.
>>
>> The most upwardly compatible way that i thought of is something like
>>
>> #= msg-id key=val key=val
>>
>> i.e. duplicate the msg-id for messages with properties
>>
>> This would be ignored by old notmuch-restore.
>>
>> Otherwise, maybe something like
>>
>> msg-id -- +tag +tag # key=val key=val
>>
>> I'm not sure. this might crash old notmuch-restore.
>>
>> How important is backward compatibility, and how important is minimizing
>> dump size? It's a bit hard to predict the things people might use
>> message properties for, but for thread surgery, I would expect a small
>> number of messages with properties.
>
> The other concern is our conception of how properties are unset/removed,
> right?
>
> With tags, it's possible to include -blah to remove the tag "blah".  how
> do we remove/clear/overwrite these tags?  what about using +key=val or
> -key=val to set/unset certain key/value combinations, and a value-less
> key= to remove all values matching a given key?
>
> alternately:
>
>  key=val (clears all values for "key", and sets a new value "val")
>  key+=val (appends a value "val" for "key")
>  key-=val (removes any "key" set to "val")
>  key= (clears all values for "key"

We'd have to distinct between key being empty and unset,
comparable to how notmuch config behaves...


$ notmuch config get built_with.compact
true
$
$ notmuch config get search.exclude_tag
$
$ notmuch config get search.exclude_tagsz 
Unknown configuration item: search.exclude_tagsz
zsh: exit 1     notmuch config get search.exclude_tagsz

>
> ---------
>
> However we resolve this particular decision, it'd be nice to have a
> stable, sane story about backward compatibility going forward, so that
> we don't have to worry about it in the future.
>
> For example, each dump file could start with a line like:
>
>   #version 1
>
> and notmuch restore would assume that without "#version n" as the first
> line, it's version 0.  then notmuch restore could decline to parse dump
> files of a version that it doesn't know about.
>
> Alternately, we could have the first line be something like:
>
>    #features config properties
>
> and if the first line is not #features, then we assume that no features
> are in place -- but if restore sees features it doesn't know about, it
> can offer to proceed while warning the user that we might miss something
> (or that something might break).

Currently dump output starts with (just run notmuch dump | less)

#notmuch-dump batch-tag:2 config,tags

perhaps this info could be put there -- is restore now (since a few notmuch
versions) already declining if this contains some strange data ?

of the 2 above suggestions I'd go w/ compatibilty version; it might be
challenging to get old notmuch parse relevant data from newer format...
... unless we also change the format to something more structured (jso^H^H^H^G
where only known data can be extracted (no, it is not SMOP, NO!)

> Thanks for working on this, David!  I think this is going to be really
> useful!

Öh, what is this feature for... >;) maybe I have to look into the series
deeper...

>
>     --dkg

Tomi